Address already in use with boost asio acceptor

Question

I wrote a server that is listening for incomming TCP connections and clients connecting to it. When I shut down the server and restart it on the same port, I sometimes get the error message EADDRINUSE when calling bind(...) (error code: 98 on Linux). This happens even though I am setting the option to reuse the socket.

The error does not happen all the time, but it seems that it happens more often when clients are connected to the server and sending data while it shuts down. I guess the problem is that there are still pending connections while the server is shut down (related topic: https://stackoverflow.com/questions/41602/how-to-forcibly-close-a-socket-in-time-wait).

On the server side, I am using boost::asio::ip::tcp::acceptor. I initialize it with the option "reuse_address" (see http://beta.boost.org/doc/libs/1_38_0/doc/html/boost_asio/reference/basic_socket_acceptor.html). Here is the code snippet:

using boost::asio::ip::tcp;
acceptor acceptor::acceptor(io_service);
endpoint ep(ip::tcp::v4(), port);
acceptor.open(ep.protocol());
acceptor.set_option(acceptor::reuse_address(true));
acceptor.bind(ep);
acceptor.listen();

The acceptor is closed with:

acceptor.close();

I also tried using acceptor.cancel() before that, but it had the same effect. When this error occurred, I cannot restart the server on the same port for quite some time. Restarting the network helps, but is not a permanent solution.

What am I missing?

Any help would be greatly appreciated! :)

Solution 1

These were originally a comment to the question.

---

does your server fork child processes? Also, are you sure the socket is in TIME_WAIT state? You might want to grab the netstat -ap output when this happens

Solution 2

When you solve these problems "by force", it seems you are calling problems on your head, do not you?

There is a reason the default behavior requires you to wait, otherwise the network could for example confuse the ACK from the previous connection to be ACK for the new connection.

I would not allow this "solution" to be included in release builds in my team.

Remember, when the probability of error is very low, testing is extremely difficult!

Comments

[Sam Miller]

does your server fork child processes?

[Sam Miller]

also, are you sure the socket is in TIME_WAIT state? You might want to grab the netstat -ap output when this happens.

[Alexander]

Sam, thank you very much for your input! It helped me to find the solution to my problem. It turned out that I did set the reuse address option for the acceptor, but not for all the other connections. Again, thank you very much for your help!

[Alexander]

Thank you for your reply. I split my answer due to space limitations. I recap the situation first: A server was running and listened on a fixed port for incoming connections. Then, clients connected to it on that port. At some point in time, I stopped the server which led to all clients closing their connection to the server. (I am pretty sure that I closed the connections both on the server and client side correctly.)

[Alexander]

Then, I started the server again. However, the server could not begin listening on the same port because the clients' connections did not have the reuse option and the port was still reserved. The question now is how I can reuse an address that is already marked in use by the operating system even though all connections were closed correctly without using the reuse option?

[Alexander]

Just accepting to wait (which can take quite long) is not an acceptable solution for a server. I believed (and still believe) that setting the reuse option is appropriate in this case. However, I am very interested in an alternative solution and would be happy if you could provide or hint at one. (On a side note: The proposed solution above works very well. We use the server extensively and did not encounter any problems related with the connections. Of course, this does not guarantee that it is 100% error free.)

[ulidtko]

The FIN_WAIT timeout is customizable; it's a common practice to reduce it from the default 120 seconds to, like 5 seconds. This will not alleviate the problem completely — you still have to handle the "address already in use" situation (as it has other valid reasons to occur) — but obviates the need for SO_REUSEADDR.