Changing the TCP RTO value in Linux

Question

I want to alter the TCP RTO (retransmission timeout) value for a connection, and some reading I have done suggests that I could do this, but does not reveal where and how to change it.

I have looked at the /proc/sys/net/ipv4 variables, but none of the variables is related to RTO. I would appreciate it if someone can tell me how to alter this value.

Solution

The reason you can't alter the RTO specifically is because it is not a static value. Instead (except for the initial SYN, naturally) it is based on the RTT (Round Trip Time) for each connection. Actually, it is based on a smoothed version of RTT and the RTT variance with some constants thrown into the mix. Hence, it is a dynamic, calculated value for each TCP connection, and I highly recommend this article which goes into more detail on the calculation and RTO in general.

Also relevant is RFC 6298 which states (among a lot of other things):

Whenever RTO is computed, if it is less than 1 second, then the RTO SHOULD be rounded up to 1 second.

Does the kernel always set RTO to 1 second then? Well, with Linux you can show the current RTO values for your open connections by running the ss -i command:

State       Recv-Q Send-Q                                                  Local Address:Port     Peer Address:Port
ESTAB       0      0                                                           10.0.2.15:52861   216.58.219.46:http
     cubic rto:204 rtt:4/2 cwnd:10 send 29.2Mbps rcv_space:14600
ESTAB       0      0                                                           10.0.2.15:ssh          10.0.2.2:52586
     cubic rto:201 rtt:1.5/0.75 ato:40 cwnd:10 send 77.9Mbps rcv_space:14600
ESTAB       0      0                                                           10.0.2.15:52864   216.58.219.46:http
     cubic rto:204 rtt:4.5/4.5 cwnd:10 send 26.0Mbps rcv_space:14600

The above is the output from a VM which I am logged into with SSH and has a couple of connections open to google.com. As you can see, the RTO is in fact set to 200-ish (milliseconds). You will note that is not rounded to the 1 second value from the RFC, and you may also think that it's a little high. That's because there are min (200 milliseconds) and max (120 seconds) bounds in play when it comes to RTO for Linux (there is a great explanation of this in the article I linked above).

So, you can't alter the RTO value directly, but for lossy networks (like wireless) you can try tweaking F-RTO (this may already be enabled depending on your distro). There are actually two related options related to F-RTO that you can tweak (good summary here):

net.ipv4.tcp_frto
net.ipv4.tcp_frto_response

Depending on what you are trying to optimize for, these may or may not be useful.

EDIT: following up on the ability to tweak the rto_min/max values for TCP from the comments.

You can't change the global minimum RTO for TCP (as an aside, you can do it for SCTP - those are exposed in sysctl), but the good news is that you can tweak the minimum value of the RTO on a per-route basis. Here's my routing table on my CentOS VM:

ip route
10.0.2.0/24 dev eth0  proto kernel  scope link  src 10.0.2.15 
169.254.0.0/16 dev eth0  scope link  metric 1002 
default via 10.0.2.2 dev eth0

I can change the rto_min value on the default route as follows:

ip route change default via 10.0.2.2 dev eth0 rto_min 5ms

And now, my routing table looks like this:

ip route
10.0.2.0/24 dev eth0  proto kernel  scope link  src 10.0.2.15 
169.254.0.0/16 dev eth0  scope link  metric 1002 
default via 10.0.2.2 dev eth0  rto_min lock 5ms

Finally, let's initiate a connection and check out ss -i to see if this has been respected:

ss -i
State       Recv-Q Send-Q                                               Local Address:Port                                                   Peer Address:Port   
ESTAB       0      0                                                        10.0.2.15:ssh                                                        10.0.2.2:50714   
     cubic rto:201 rtt:1.5/0.75 ato:40 cwnd:10 send 77.9Mbps rcv_space:14600
ESTAB       0      0                                                        10.0.2.15:39042                                                 216.58.216.14:http    
     cubic rto:15 rtt:5/2.5 cwnd:10 send 23.4Mbps rcv_space:14600

Success! The rto on the HTTP connection (after the change) is 15ms, whereas the SSH connection (before the change) is 200+ as before.

I actually like this approach - it allows you to set the lower value on appropriate routes rather than globally where it might screw up other traffic. Similarly (see the ip man page) you can tweak the initial rtt estimate and the initial rttvar for the route (used when calculating the dynamic RTO). While it's not a complete solution in terms of tweaking, I think most of the important pieces are there. You can't tweak the max setting, but I think that is not going to be as useful generally in any case.

Comments

[Admin]

This doesn't work for me. At the very minimum including 'ms' with the rto time gives me an error! I did get the command to work, but ss -i says otherwise. Furthermore, the 2 sysctl vars don't exist. I'm running on a 4.4 kernel

[Admin]

I tried on a more recent distro and had success doing ip route replace instead - seems the ip route syntax has changed a little. I was able to make the modification successfully though. Just to note, you should comment on the answer, not the question if you want to ping me - I basically saw this by accident, pure luck that it was recent :)

[srinivas rao]

I believe they are constants in the code, and I am not aware of a way to set them dynamically but will dig a bit and see since I assume altering the code and compiling your own kernel is a bit much :)

[est]

Why aren't there anything like rto_max ? How do we set a global maximum timeout ?

[srinivas rao]

If you set the minimum (or accept the default) and then alter the number of retries (net.ipv4.tcp_retries1 and net.ipv4.tcp_retries2 or similar IIRC) allowed, I think you can get to an equivalent of an RTO maximum.