Why is my UDP so slow?

Question

I'm using Iperf on two VMs and when using TCP I find the performance is as follows:

notroot@ubuntu:~$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001 TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.1.29 port 5001 connected with 192.168.1.13 port 52478
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 2.22 GBytes 1.90 Gbits/sec

UDP on the other hand is awful:

notroot@ubuntu:~$ iperf -s -u
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size: 208 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.29 port 5001 connected with 192.168.1.13 port 33775
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0-10.0 sec 1.25 MBytes 1.05 Mbits/sec 0.284 ms 0/ 893 (0%)

I was reading this article

Questions:

1. What do you think of the results?
2. How can I manipulate the datagram size in case it is a fragmentation issue? Though that said I can confirm that the Iperf client is sending 1470 byte datagrams.

Thanks.

Solution 1

In this case, the problem is that iperf's default speed for UDP is very slow (see the other answers). But for less extreme speed differences, check this question on serverfault.com.

Following a quote from pehrs's answer

Each frame goes through several buffers as you send it: The application buffer, The Protocol Buffer, The Software interface buffer and the Hardware interface buffer. As you start stressing the stack by sending high speed data you will fill up these buffers and either block or lose data. You also have strategies for timeliness and polling that can impact your performance. For example, by using a larger buffer and poll less often you can get much better performance while sacrificing latency.

TCP is optimized for high speed bulk transfers while UDP is optimized for low latency in the Linux kernel. This has an impact on buffer sizes and how data is polled and handed over. In addition to this, you frequently have offloading to hardware for TCP. I would expect considerably better performance for TCP compared to UDP.

Note that sending high speed data over UDP is usually a bad idea, unless you implement your own congestion control. TCP protects your network from congestion collapses. Use UDP when you have small amounts of data or high timeliness requirements.

Solution 2

iPerf uses a default of 1Mb/sec for UDP tests. Use the -b flag on the iperf client to specify the UDP bandwidth you want to transmit at e.g.

iperf -c 10.79.175.219 -u -f m -b 100M

Solution 3

As Nick answered, iPerf uses a default of 1Mbit/sec for UDP.

In order to use the same bandwidth as TCP while performing a UDP test, simply provide -b flag with the value of 0.

iperf3 --udp -b 0 -f m -c <server ip>

form the documentation,

-b, --bandwidth #[KMG][/#]
target bandwidth in bits/sec

(0 for unlimited)

(default 1 Mbit/sec for UDP, unlimited for TCP)

(optional slash and packet count for burst mode)

-f in the above command provide type of formatting (m if for MBits),

-f, --format [kmgKMG] format to report: Kbits, Mbits, KBytes, MBytes

Comments

[xryl669]

This is the actual reason and should be the correct answer.

[xryl669]

This does not explain a 2000x rate difference. Even an order of magnitude faster for TCP would not be explainable this way.

[Telegrapher]

I downvoted this answer because it is not correct. I've performed such test and the performance difference from UDP to TCP exists, but as @xryl669 says, is way smaller. In a gigabit link I went from 928Mbit/s (TCP) to 812Mbit/s (UDP)

[Telegrapher]

I've tested this and this is the correct answer.

[Satish]

I am getting 14.4 KBytes when i use -b 0

[Raman]

Works for me with iperf3. 1 Mbit/s before, 943 Mbit/s after -b 0 (on a gigabit link).

[Melroy van den Berg]

Indeed. Use 0 as bit-rate for unlimited.

[Admin]

Except this answer is for iperf3 only, not iperf 2.