Python socket error on UDP data receive. (10054)

Question

I currently have a problem using UDP and Python socket module. We have a server and clients. The problem occurs when we send data to a user. It's possible that user may have closed their connection to the server through a client crash, disconnect by ISP, or some other improper method. As such, it is possible to send data to a closed socket.

Of course with UDP you can't tell if the data really reached or if it's closed, as it doesn't care (atleast, it doesn't bring up an exception). However, if you send data and it is closed off, you get data back somehow (???), which ends up giving you a socket error on sock.recvfrom. [Errno 10054] An existing connection was forcibly closed by the remote host. Almost seems like an automatic response from the connection.

Although this is fine, and can be handled by a try: except: block (even if it lowers performance of the server a little bit). The problem is, I can't tell who this is coming from or what socket is closed. Is there anyway to find out 'who' (ip, socket #) sent this? It would be great as I could instantly just disconnect them and remove them from the data. Any suggestions? Thanks.

Server:

import socket

class Server(object):
    def __init__(self):
        self.socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.connected = {}

    def connect(self):
        self.socket.bind(('127.0.0.1', 5579))

    def find_data(self):
        while 1:
            data, address = self.socket.recvfrom(1024)
            self.got_data(data,address)
            if self.connected.has_key(address):
                pass
            else:
                self.connected[address] = None

    def got_data(self, data, address):
        print "GOT",data,"FROM",address
        for people in self.connected:
            print people
            self.send_data('hi', people)

    def send_data(self, data, address):
        self.socket.sendto(data,address)


if __name__ == '__main__':
    server = Server()
    server.connect()
    print "NOW SEARCHING FOR DATA"
    server.find_data()

Client:

import socket, time

class Client(object):
    def __init__(self):
        self.socket = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)

    def connect(self):
        self.socket.connect(('127.0.0.1', 5579))

    def send_data(self):
        self.socket.sendto('hi',('127.0.0.1', 5579))

    def got_data(self, data, address):
        print "GOT",data,"FROM",address


if __name__ == '__main__':
    client = Client()
    client.connect()
    while 1:
        client.send_data()
        time.sleep(5)

Solution 1

Firstly this is possibly platform specific and you don't mention the platform that you're running on; however, 10054 is WSAECONNRESET so I'm guessing a Windows platform of some kind.

Secondly as previously pointed out there is no connection with UDP. Your call to Connect() in the client simply causes the networking code on your client machine to allow you to initiate Send() calls rather than SendTo() calls and simply default the address that you are sending data to when you issue Send() calls to the address supplied to the call to Connect().

Thirdly I'm surprised that you're getting WSAECONNRESET and not ERROR_PORT_UNREACHABLE; however the underlying reason is probably the same. The UDP stack on the remote machine will likely be sending a ICMP Port Unreachable error if there's no socket open on the port that you are sending to. So, if your client sends data and then closes the socket and then your server sends data back to the client address you'll get a port unreachable and some versions of windows might be translating that into a connection reset error...

The problem with these ICMP port unreachable errors, is that they're reported via the Winsock code by failing a pending UDP Recv/RecvFrom call. As I explain here and question here the UDP stack obviously knows the address that generated the port unreachable but it doesn't pass that information on to the caller so there's nothing that you can do to map these messages to something useful. It's possible that you're running on a version of Windows prior to Vista and the UDP stack is doing something useful with the address and it IS reporting the error to the correct socket, but don't bet on it.

Finally you have a problem anyway; the ICMP port unreachable error isn't delivered reliably so you can't know for sure that you WILL get an error if you try and send UDP data to a client that has gone away. IMHO this means that you shouldn't rely on it even if it works sometimes.

You are obviously attempting to build some kind of connection oriented protocol on top of UDP (why else would your server hold on to the addresses of the clients). You'll have to do a lot more to create a viable pseudo connection over UDP and one of the first things to realise is that the only way you can know when a client has gone away is to set your own timeout and 'disconnect' your pseudo connection if you don't hear from them within a certain period of time.

Of course this doesn't answer your question; how do you avoid the exception. I expect you can't. The cause of the exception is likely a 'failure' return code from a Recv() or RecvFrom() call and the python network code is probably converting all such failure returns into exceptions for you.

Solution 2

The problem is far simpler than it looks. Use socket.recv() rather than socket.recvfrom() - I made this change locally and your code then works.

Solution 3

Well, it seems obvious.

1. UDP doesn't have connections, so Client.connect is wrong
2. You're storing the client address in the Server.connected dict. When the client is closed, nobody will be there to receive what you're sending.

Getting network communication right is hard, since socket library it is too low-level (it is a thin wrapper around C sockets). A lot of details have been left out on your code. I suggest trying a higher level library, like twisted. Here's some example on UDP to get you started.

Comments

[Admin]

Thanks for the informative and huge response. :) I also suspected we couldn't avoid the exception. To answer a few of questions: I am using Windows 7. Also the reason we are trying to separate the IP and socket # is if we have 5 different clients, all of them will have separate data and accounts (server side). How else is the server to distinguish who is who, data wise, without that information? Maybe we are approaching this the wrong way?

[Len Holgate]

Why are you not using TCP for this? With TCP you'll have real connections with notification of when the connection is no longer open. Staying with UDP means that you have to build your own connection, so either you rely on the client address and port not changing (which can be OK but might not be with some NATs) or you embed a 'connection id' in your datagram. With that you have an unreliable connection oriented system, but you have no way to know when the client goes away; so you need timeouts...

[Admin]

Unfortunately TCP was too slow for what we needed, which is a game. Perhaps the solution is somewhere in-between? TCP for handling login information and UDP to handle most of the data sending to the game server.

[Len Holgate]

There's nothing to stop you using UDP, but perhaps you should take a look at the various protocols that have already been built on UDP to add various elements of TCP back into the mix whilst improving performance; I have a question here: stackoverflow.com/questions/107668/… which lists several alternatives... Just bear in mind that there are various issues to address and that others have often already been there and done that... (ENet is quite useful for games, but I'm not sure if there are python bindings).

[xaxxon]

udp doesn't have connections, but connect has a behavior for udp sockets that emulates them.