linux KVM guests network performance is slower than host

linux kvm-virtualization

21,545

Solution 1

I suppose you already using virtio driver in your VMs, its a must to have decent network performance.

You can also try using quite new guest netdev model, which uses some advanced memory mapping techniques to achieve even better network performance. You VMs should use the same configuration as with virtio drivers; kvm parameters to set start VM should be like this (fix them for your needs):

-netdev tap,vhost=on,ifname=vnet0,id=guest0,script=/path/to/custom/tap-ifup,downscript=no \
-device virtio-net-pci,netdev=guest0,mac=00:16:3e:47:aa:53

With this configuration I've achieved 6 Gb/s.

Solution 2

Try changing ethernet driver settings.

/sbin/ethtool --offload eth0 gso off tso off sg off gro off

See

https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Host_Configuration_and_Guest_Installation_Guide/ch10s04.html

" If you experience low performance with the para-virtualized network drivers, verify the setting for the GSO and TSO features on the host system. The para-virtualized network drivers require that the GSO and TSO options are disabled for optimal performance."

21,545

Author by

Moaz Refat

Updated on September 18, 2022

Comments

Moaz Refat almost 2 years
i have a KVM linux based virtual machines, as the network performance of the host is running perfect with 1 Gbps (on the internal network) and using iperf tools it show that the network is running perfect on the host, but guests have slow network performance as iperf result was so strange as it is not 100 Mbps and is not 1 Gbps, it's between 350 and 500 Mbps maximum. This is the result of runnin iperf -c servername -P 15:
```
[ ID] Interval       Transfer     Bandwidth
[ 17]  0.0-10.0 sec  68.6 MBytes  57.5 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  18.6 MBytes  15.5 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec  61.9 MBytes  51.9 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[ 11]  0.0-10.0 sec  16.0 MBytes  13.4 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[ 10]  0.0-10.0 sec  24.3 MBytes  20.4 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[ 15]  0.0-10.0 sec  20.2 MBytes  16.9 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[ 12]  0.0-10.0 sec  30.2 MBytes  25.3 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  36.2 MBytes  30.3 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[  8]  0.0-10.0 sec  23.6 MBytes  19.8 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[  9]  0.0-10.0 sec  24.2 MBytes  20.3 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[ 14]  0.0-10.0 sec  31.0 MBytes  25.9 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[  7]  0.0-10.0 sec  16.6 MBytes  13.9 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[  6]  0.0-10.0 sec  18.8 MBytes  15.7 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[ 13]  0.0-10.0 sec  13.3 MBytes  11.2 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[ 16]  0.0-10.0 sec  29.6 MBytes  24.8 Mbits/sec
[SUM]  0.0-10.0 sec    433 MBytes    363 Mbits/sec
```
I tested different network models and the best result was with the virtio model any one please can help to get the full performance of network and be 1 Gbps.

I would like to add that when i execute the command brctl show I got the fbridge name
```
bridge id       STP enabled interfaces
br0     8000.d067e5fb7dec   no      eth0
                            vnet0
                            vnet1
                            vnet2
br2     8000.d067e5fb7dee   no      eth1
virbr0      8000.000000000000   yes
```
can this affect the network speed of the guest?
- Devator over 11 years
  
  What drivers are you using for your guest? Try virtio if you are not doing so already.
- Heis Spiter over 11 years
  
  Did you already look at this?
- Danie over 11 years
  
  Can you elaborate on the kvm setup, are you using attached storage via network, etc. is the network slow between kvms on different hardware hosts?
- Moaz Refat over 11 years
  
  @Devator the guest is Dell poweredge server with network interface Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) and the host is working with 1GB/s but the guests which runs with range between 350MB/s and 500MB/s i don't know why.
- Devator over 11 years
  
  You are specifiying the host's hardware. The guests are the VM's. You need to set the network adapter (of the guests) to virtio.
- mgorven over 11 years
  
  What's the host OS and KVM version?
- Moaz Refat over 11 years
  
  @Devator i already did , and set the network adapter to virtio,
- Moaz Refat over 11 years
  
  @mgorven Operating system linux / Ubuntu 10.04 64-bit, KVM version is qemu-kvm-0.12.3
- Axel over 10 years
  
  You should edit the extra information from your comments into your question so it is easier for new viewers (and possible providers of useful answers) to see everything relevant. Also: you seem to have missed Danie's spec information question. Does this machine's hardware differ at all to the other one you mentioned testing the VM in?
- David Corsalini about 10 years
  
  Ubuntu 10? Any ubuntu, but especially older versions are known for problematic virtualization performance. Try with a fresh Fedora or RHEL build.
Moaz Refat over 11 years

i copied the virtual machine to the server whose network performance is perfect and he network perofamance of the guest work perfect and running with 1GB/s so the problem is a host problem, can you advice what it may to be ?
Lari Hotari over 10 years

see also serverfault.com/questions/272483/…
Lari Hotari over 10 years

Xen related, but partly applies to KVM/virtio: cloudnull.io/2012/07/xenserver-network-tuning