Python: Binding Socket: "Address already in use"

Question

I have a question regarding client socket on TCP/IP network. Let's say I use

try:

    comSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    comSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

except socket.error, msg:

    sys.stderr.write("[ERROR] %s\n" % msg[1])
    sys.exit(1)

try:
    comSocket.bind(('', 5555))

    comSocket.connect()

except socket.error, msg:

    sys.stderr.write("[ERROR] %s\n" % msg[1])

    sys.exit(2)

The socket created will be bound to port 5555. The problem is that after ending the connection

comSocket.shutdown(1)
comSocket.close()

Using wireshark, I see the socket closed with FIN,ACK and ACK from both sides, I can't use the port again. I get the following error:

[ERROR] Address already in use

I wonder how can I clear the port right away so that next time I still can use that same port.

comSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

setsockopt doesn't seem to be able to resolve the problem Thank you!

Solution 1

Try using the SO_REUSEADDR socket option before binding the socket.

comSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

Edit: I see you're still having trouble with this. There is a case where SO_REUSEADDR won't work. If you try to bind a socket and reconnect to the same destination (with SO_REUSEADDR enabled), then TIME_WAIT will still be in effect. It will however allow you to connect to a different host:port.

A couple of solutions come to mind. You can either continue retrying until you can gain a connection again. Or if the client initiates the closing of the socket (not the server), then it should magically work.

Solution 2

Here is the complete code that I've tested and absolutely does NOT give me a "address already in use" error. You can save this in a file and run the file from within the base directory of the HTML files you want to serve. Additionally, you could programmatically change directories prior to starting the server

import socket
import SimpleHTTPServer
import SocketServer
# import os # uncomment if you want to change directories within the program

PORT = 8000

# Absolutely essential!  This ensures that socket resuse is setup BEFORE
# it is bound.  Will avoid the TIME_WAIT issue

class MyTCPServer(SocketServer.TCPServer):
    def server_bind(self):
        self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.socket.bind(self.server_address)

Handler = SimpleHTTPServer.SimpleHTTPRequestHandler

httpd = MyTCPServer(("", PORT), Handler)

# os.chdir("/My/Webpages/Live/here.html")

httpd.serve_forever()

# httpd.shutdown() # If you want to programmatically shut off the server

Solution 3

According to this link

Actually, SO_REUSEADDR flag can lead to much greater consequences: SO_REUSADDR permits you to use a port that is stuck in TIME_WAIT, but you still can not use that port to establish a connection to the last place it connected to. What? Suppose I pick local port 1010, and connect to foobar.com port 300, and then close locally, leaving that port in TIME_WAIT. I can reuse local port 1010 right away to connect to anywhere except for foobar.com port 300.

However you can completely avoid TIME_WAIT state by ensuring that the remote end initiates the closure (close event). So the server can avoid problems by letting the client close first. The application protocol must be designed so that the client knows when to close. The server can safely close in response to an EOF from the client, however it will also need to set a timeout when it is expecting an EOF in case the client has left the network ungracefully. In many cases simply waiting a few seconds before the server closes will be adequate.

I also advice you to learn more about networking and network programming. You should now at least how tcp protocol works. The protocol is quite trivial and small and hence, may save you a lot of time in future.

With netstat command you can easily see which programs ( (program_name,pid) tuple) are binded to which ports and what is the socket current state: TIME_WAIT, CLOSING, FIN_WAIT and so on.

A really good explanation of linux network configurations can be found https://serverfault.com/questions/212093/how-to-reduce-number-of-sockets-in-time-wait.

Solution 4

In case you face the problem using TCPServer or SimpleHTTPServer, override SocketServer.TCPServer.allow_reuse_address (python 2.7.x) or socketserver.TCPServer.allow_reuse_address (python 3.x) attribute

class MyServer(SocketServer.TCPServer):
    allow_reuse_address = True

server = MyServer((HOST, PORT), MyHandler)
server.serve_forever()

Solution 5

You need to set the allow_reuse_address before binding. Instead of the SimpleHTTPServer run this snippet:

Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", PORT), Handler, bind_and_activate=False)
httpd.allow_reuse_address = True
httpd.server_bind()
httpd.server_activate()
httpd.serve_forever()

This prevents the server from binding before we got a chance to set the flags.

View more solutions

Comments

[Tu Hoang]

Still cannot reuse it. The time I have to wait before being able to reuse the same port is 1 minute 30 seconds :(

[lunixbochs]

did you call setsockopt before bind? was the first socket created with SO_REUSEADDR, or just the failed one? the waiting socket must have SO_REUSEADDR set for this to work

[Tu Hoang]

Yes, I did include comSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR,1) but still doesn't work.

[lunixbochs]

please paste your exact new code, and clarify whether you've had a successful socket open since you changed the code

[Tu Hoang]

@lunixboschs: I posted the code on the original post since I cannot reply to my own thread. I still have to wait for around 1min30secs before I can actually reuse that port.

[Tu Hoang]

tcp 0 0 98c5-9-134-71-1:freeciv mobile-166-132-02:2345 TIME_WAIT

[Bryan]

@Tanner: Updated my answer. I think that is what you are experiencing.

[Tu Hoang]

Thank you :) But anyways, I just use a static port so that the Sys Admins create firewall rule for the connection. Since I only need to restart it once every 5 minutes so that wouldn't be an issue.

[jcoffland]

SO_REUSEADDR is for server sockets. Noobie is creating a client socket and should not be using bind() at all.

[Rustem K]

I think it is not your issue. Please see my answer below.

[Rustem K]

Also, you should be careful with your code. If your code still on development and some exceptions occured, the connection might be not closed properly especially from server side.

[HelloWorld]

You should be fair and quote the sentences you copy & pasted from Tom's Network Guide. Please correct your answer.

[Andrei]

It seems much easier to subclass TCPServer and override the attribute, see my answer for example

[Chris Merck]

Unless the process is already killed and it just left the port open to be cleaned up by the os.

[Androbin]

Even after httpd.shutdown() you still wanna call httpd.server_close() to fully release all resources.

[Androbin]

Also, consider using the atexit module for any unexpected exists.

[user207421]

@jcoffland I agree he shouldn't be using bind(), but SO_REUSEADDR is for all sockets including not only TCP server sockets but also TCP client sockets and UDP sockets. Don't post misinformation here.

[user207421]

You could just as well omit the bind, but the OP states he must use port 5555,and this answer doesn't.

[user207421]

bind() and shutdown() have exactly nothing to do with each other, and the suggestion that 'they don't seem to play well together' is baseless. You have missed the part where he needs to use local port 5555.

[user207421]

Killing processes does not affect TIME_WAIT in any way.

[user207421]

Killing processes does not affect TIME_WAIT in any way.

[Felipe Cruz]

he changed the content of the question: stackoverflow.com/revisions/6380057/4 - check the revisions before next time :)

[qwerty_so]

Why is it that this answer propose a solution that according to the question explicitely does not work? And it's marked as correct??

[Swift - Friday Pie]

@jcoffland that's incorrect. calling bind on outgoing socket is legal for port to port interaction - many well-known services do that. but there is indeed a problem with reusing port uf communication wasn't shutdown properly

[Aliaksandr S.]

This does not really answer the question. If you have a different question, you can ask it by clicking Ask Question. To get notified when this question gets new answers, you can follow this question. Once you have enough reputation, you can also add a bounty to draw more attention to this question. - From Review