java.net.SocketTimeoutException: Read timed out under Tomcat

Question

I have a Tomcat based web application. I am intermittently getting the following exception,

Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:150)
    at java.net.SocketInputStream.read(SocketInputStream.java:121)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:532)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501)
    at org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:563)
    at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:124)
    at org.apache.coyote.http11.AbstractInputBuffer.doRead(AbstractInputBuffer.java:346)
    at org.apache.coyote.Request.doRead(Request.java:422)
    at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:290)
    at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:431)
    at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:315)
    at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:200)
    at java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)

Unfortunately I don't have access to the client, so I am just trying to confirm on various reasons this can happen,

1. Server is trying to read data from the request, but its taking longer than the timeout value for the data to arrive from the client. Timeout here would typically be Tomcat connector → connectionTimeout attribute.

2. Client has a read timeout set, and server is taking longer than that to respond.

3. One of the threads I went through, said this can happen with high concurrency and if the keepalive is enabled.

For #1, the initial value I had set was 20 sec, I have bumped this up to 60sec, will test, and see if there are any changes.

Meanwhile, if any of you guys can provide you expert opinion on this, that'l be really helpful. Or for that matter any other reason you can think of which might cause this issue.

Solution 1

Server is trying to read data from the request, but its taking longer than the timeout value for the data to arrive from the client. Timeout here would typically be tomcat connector -> connectionTimeout attribute.

Correct.

Client has a read timeout set, and server is taking longer than that to respond.

No. That would cause a timeout at the client.

One of the threads i went through, said this can happen with high concurrency and if the keepalive is enabled.

That is obviously guesswork, and completely incorrect. It happens if and only if no data arrives within the timeout. Period. Load and keepalive and concurrency have nothing to do with it whatsoever.

It just means the client isn't sending. You don't need to worry about it. Browser clients come and go in all sorts of strange ways.

Solution 2

Here are the basic instructions:-

1. Locate the "server.xml" file in the "conf" folder beneath Tomcat's base directory (i.e. %CATALINA_HOME%/conf/server.xml).
2. Open the file in an editor and search for <Connector.
3. Locate the relevant connector that is timing out - this will typically be the HTTP connector, i.e. the one with protocol="HTTP/1.1".
4. If a connectionTimeout value is set on the connector, it may need to be increased - e.g. from 20000 milliseconds (= 20 seconds) to 120000 milliseconds (= 2 minutes). If no connectionTimeout property value is set on the connector, the default is 60 seconds - if this is insufficient, the property may need to be added.
5. Restart Tomcat

Solution 3

Connection.Response resp = Jsoup.connect(url) //
                .timeout(20000) //
                .method(Connection.Method.GET) //
                .execute();

actually, the error occurs when you have slow internet so try to maximize the timeout time and then your code will definitely work as it works for me.

Comments

[fge]

I don't believe it is point 2; however, to confirm you should also set up an AccessLogValve and try and correlate this exception with a specific request.

[Victor]

For #1 as i said, i have bumped up the connection timeout value of the tomcat connector from 20sec to 60sec. If the timeouts reduce then i'll try changing it to -1 (infinite) for testing and eventually set a more appropriate value. For #2, if suppose the client times out what error/exception should i be expecting on the server?

[user207421]

If the client times out reading the response, the server either won't see anything at all or else may see a 'connection reset'. I wouldn't put the server read timeout too high: it ties up a thread. If a client opens a connection to the server and doesn't send anything immediately it is misbehaving pretty badly.

[Victor]

I do get a "connection reset" once in a while, which i attributed to client timing out (client does have a timeout set, need to figure out what is that). However i have been getting far too many "read timeouts". For which i have bumped up the connection timeout to 60sec. will monitory today if "read timeouts" reduce. My max threads is about 190 and accept count is set to 400, which i plan to reduce to 200. Since no point in having a larger queue, it will just increase the waiting time and hence the latency. Any other changes that you think i can try and might help?

[user207421]

Vicky, the objective isn't to get a clean log file. The objective is to service the clients that aren't misbehaving. The longer you make the read timeout, the longer that thread is tied up waiting for the misbehaving client, so the fewer threads are available to service behaving clients.

[Blake Russo]

Excellent explanation of the general problem, thank you! Does anyone know what file for the tomcat connector property to change that timeout?

[user207421]

@tuxworld.net server.xml.

[user207421]

This is client code for a server-side problem, and actually it occurs when the client is slow sending the complete request.

[jazeb007]

I am also getting this issue fo several days but cannot fix it