How do I abort a socket.recv() from another thread in Python

Question

I have a main thread that waits for connection. It spawns client threads that will echo the response from the client (telnet in this case). But say that I want to close down all sockets and all threads after some time, like after 1 connection.

How would I do it? If I do clientSocket.close() from the main thread, it won't stop doing the recv. It will only stop if I first send something through telnet, then it will fail doing further sends and recvs.

My code looks like this:

# Echo server program
import socket
from threading import Thread
import time

class ClientThread(Thread):
    def __init__(self, clientSocket):
            Thread.__init__(self)
            self.clientSocket = clientSocket

    def run(self):
            while 1:
                    try:
                            # It will hang here, even if I do close on the socket
                            data = self.clientSocket.recv(1024)
                            print "Got data: ", data
                            self.clientSocket.send(data)
                    except:
                            break

            self.clientSocket.close()

HOST = ''
PORT = 6000
serverSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serverSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
serverSocket.bind((HOST, PORT))
serverSocket.listen(1)

clientSocket, addr = serverSocket.accept()
print 'Got a new connection from: ', addr
clientThread = ClientThread(clientSocket)
clientThread.start()

time.sleep(1)

# This won't make the recv in the clientThread to stop immediately,
# nor will it generate an exception
clientSocket.close()

Solution 1

I know this is an old thread and that Samuel probably fixed his issue a long time ago. However, I had the same problem and came across this post while google'ing. Found a solution and think it is worthwhile to add.

You can use the shutdown method on the socket class. It can prevent further sends, receives or both.

socket.shutdown(socket.SHUT_WR)

The above prevents future sends, as an example.

See Python docs for more info.

Solution 2

I don't know if it's possible to do what you're asking, but it shouldn't be necessary. Just don't read from the socket if there is nothing to read; use select.select to check the socket for data.

change:

data = self.clientSocket.recv(1024)
print "Got data: ", data
self.clientSocket.send(data)

to something more like this:

r, _, _ = select.select([self.clientSocket], [], [])
if r:
    data = self.clientSocket.recv(1024)
    print "Got data: ", data
    self.clientSocket.send(data)

EDIT: If you want to guard against the possibility that the socket has been closed, catch socket.error.

do_read = False
try:
    r, _, _ = select.select([self.clientSocket], [], [])
    do_read = bool(r)
except socket.error:
    pass
if do_read:
    data = self.clientSocket.recv(1024)
    print "Got data: ", data
    self.clientSocket.send(data)

Solution 3

I found a solution using timeouts. That will interrupt the recv (actually before the timeout has expired which is nice):

# Echo server program
import socket
from threading import Thread
import time


class ClientThread(Thread):
    def __init__(self, clientSocke):
        Thread.__init__(self)
        self.clientSocket = clientSocket

    def run(self):
        while 1:
            try:
                data = self.clientSocket.recv(1024)
                print "Got data: ", data
                self.clientSocket.send(data)
            except socket.timeout: 
                # If it was a timeout, we want to continue with recv
                continue
            except:
                break

        self.clientSocket.close()

HOST = ''
PORT = 6000
serverSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serverSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
serverSocket.bind((HOST, PORT))
serverSocket.listen(1)

clientSocket, addr = serverSocket.accept()
clientSocket.settimeout(1)

print 'Got a new connection from: ', addr
clientThread = ClientThread(clientSocket)
clientThread.start()

# Close it down immediatly 
clientSocket.close()

Solution 4

I must apologize for the comments below. The earlier comment by @Matt Anderson works. I had made a mistake when trying it out which led to my post below.

Using timeout is not a very good solution. It may seem that waking up for an instant and then going back to sleep is no big deal, but I have seen it greatly affect the performance of an application. You have an operation that for the most part wants to block until data is available and thus sleep forever. However, if you want to abort for some reason, like shutting down your application, then the trick is how to get out. For sockets, you can use select and listen on two sockets. Your primary one, and a special shutdown one. Creating the shutdown one though is a bit of a pain. You have to create it. You have to get the listening socket to accept it. You have to keep track of both ends of this pipe. I have the same issue with the Synchronized Queue class. There however, you can at least insert a dummy object into the queue to wake up the get(). This requires that the dummy object not look like your normal data though. I sometimes wish Python had something like the Windows API WaitForMultipleObjects.

View more solutions

Comments

[Samuel Skånberg]

But the same thing will happen with select. If I close down the socket it will complain about bad file descriptor when doing select.select(). I saw that your solution appeared before I posted my solution. What do you think of the way I solved it there, using timeouts?

[Samuel Skånberg]

I tried the same with select now. It works as good as with timeouts, but only if I close down the socket immediatly after I start the thread. If I do a time.sleep(1) it will fail.

[sunsay]

In C# there is a very useful method like WaitAny which i've been using for exactly this purpose (to stop waiting data from socket if there is another event to work with). Python really lacks such functionality :( Voted up for your variant. But i think it loads CPU a little if there are many threads (and timeout is only one second long)...