Meaning of "java.io.IOException: Connection timed out" after connect phase

Question

Could be related: Difference between Connection timed out and Read timed out

I have written a java server application using nio.

I connected a client to my server application and unplugged the network cable of the client. On the server side, I didn't get any exception immediately but after some time (8 minutes or so), I got a "IOException: Connection timed out"

Here is a partial stack trace:

java.io.IOException: Connection timed out
    at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
    at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
    at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225)
    at sun.nio.ch.IOUtil.read(IOUtil.java:198)
    at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375)
........

Till this time, when I saw the netstat output, I see that the socket state of this particular client connection is shown as ESTABLISHED.

Questions are:

1. Is this timeout configurable?

2. Why does the netstat output show the socket state as ESTABLISHED? Ideally it should be CLOSE_WAIT (as the client got disconnected)

Solution 1

1. No it is not configurable. It is the result of retransmit timeouts. It wouldn't happen at all unless the application kept writing, or had pending writes when the disconnect happened.

2. It shouldn't be CLOSE_WAIT, as no FIN had been received. Ergo it should be ESTABLISHED.

Solution 2

That timeout is generally not configurable as it depends on the possibilities offered by the operating system. Unix in general does not allow a process to fix the connection timeout and generally it is fixed to around two minutes. Perhaps some versions of linux/BSD systems allow this to be configured, but that's not portable and normally is not allowed to fix it to the user (only the administrator). This has to do with the number of retransmissions and the timeouts used for each try, and is under the exclusive control of the TCP implementation.

When you finish a connection you pass through two states (FIN_WAIT and TIME_WAIT) that are not timeout states. The first of two is to get the other end's response (you can close your side of the connection telling the other side you are not going to send more data, but you have to wait for the other end to do the same thing) The TIME_WAIT is a special state that the kernel maintains for a closed connection to process (and discard) all the possible retransmissions of the last frames that can be in course after the connection is closed. They have nothing to do with timeouts.

A tcp connection has no timeout implicit. Two machines can pass weeks without interchanging any info if they have nothing to transmit. You can control the use of some kind of heartbeat between silenting connections to check their liveness with one socket option (SO_KEEPALIVE) This option makes the tcps at both sides to interchange empty packets to know if the other side is still alive. Again, you can only control the use of this packets, not the frequency or the number of lost frames that closes the connection (this can be configured in linux, but touching the kernel configuration only in administrator mode)

Note 1 (answer to @Krishna Chaitanya P)

If you unplugged the cable and got an exception some time later, it can be one of two reasons for that to happen:

1. You continued writing to that connection and the sending buffer filled up without being acknowledged in time (this is rare, as normally your process get blocked in write(2) system call when this happens) and some timeout (in the java implementation of socket) did occur.
2. Your java implementation of tcp socket uses the SO_KEEPALIVE option (the most probable thing). As I said before, you have boolean control to use or not use it, but you cannot adjust the time between keepalives or the number of them that drops your connection. Try to call getKeepAlive()/setKeepAlive(boolean) methods on the Socket class to control this feature. I have not seen in the documentation if the connected socket is, by default, keepalived or not. This is, by far, a commonly used option in a server, as it allows to disconnect the clients that lose connections without telling to the server.

Comments

[user207421]

No. A port in CLOSE_WAIT state has already receive FIN from the peer and is waiting for the local application to close.

[Krishna Chaitanya P]

The timeout is configurable, not in the application, but at the OS level - Look at the Linux man page of tcp for parameters tcp_retries1 and tcp_retries2.

[Krishna Chaitanya P]

As I already mentioned, I unplugged the cable manually. I do not have a firewall in between the client and server, they are directly connected.

[Luis Colorado]

This TO is definitely not configurable. If the process that is being awaited to close doesn't do it, why to interrupt it with a timeout event? There's no possible loss of information in a close(2) call to make it necessary to inform a process that it has not closed the socket to make that timeout necessary.

[Luis Colorado]

In that case, the firewall is sending empty tcp segments with a proper sequence number (in both directions) and a RST flag, so the connection is actively reset by it's intervention. This can occur in case a NAT entry in table timeouts, but it's not frequent to see it implemented.