KVM: High host CPU load after upgrading VM to windows 10 1803

Question

I'm running an Ubuntu 16.04 server with KVM/libvirt/qemu and Windows 10 virtual machine. After upgrading Windows 10 to the latest 1803 version (April 2018 Update), i noticed a slowdown in the VM responsiveness and a too high CPU load in the host. The tests I did:

• "top" used on the host reports that the qemu-system-x86 process is taking 38-44% of CPU, while the VM is idle and reporting 4-6% of load inside windows process manager. When windows shows 50% of CPU load, the host shows 150-200% cpu load for the qemu-system-x86 process. This 8-10x increase in host CPU load does not occur with the old windows 10 1709 VM.
• "dstat" on the host reports a lot of system->csw, about 13k.
• I tested another system with the same setup with an AMD FX-4300, Ubuntu 17.10 dekstop + kvm, upgraded windows 10 to 1803 and I have the same problem of high host CPU and dstat-csw also on this machine.
• I took a third machine with an intel core i7, installed a new Ubuntu 18.04 desktop + kvm, installed a fresh new Windows 10, installed Virtio drivers available from here and I have the same high cpu problem.

What can I do? Am I using the right virtio drivers?

Thank you

Solution 1

Found someone with the same issue and a possible fix for it here: https://forum.proxmox.com/threads/high-cpu-load-for-windows-10-guests-when-idle.44531/

Tested changing my own config, and setting hpet to yes in the vm xml fixed it for me.

  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='yes'/>
    <timer name='hypervclock' present='yes'/>
  </clock>

hpet part is important.

Solution 2

Got the same issue with Windows 10 21H1 and qemu-kvm-4.2.0. When windows machine was idle, I got around 30% cpu usage. After the fix, I get about 5%.

Got the issue resolved by adding hv_synic & hv_stimer hyperv enlightments. You might get stuck with the following error

error: Failed to start domain mymachine.local
error: internal error: process exited while connecting to monitor: Hyper-V synthetic timers (hv-stimer) requires Hyper-V clocksources (hv-time)
Hyper-V synthetic interrupt controller (hv-synic) requires Hyper-V VP_INDEX MSR (hv-vpindex)
2021-09-22T20:30:06.440656Z qemu-kvm: kvm_init_vcpu failed: Function not implemented

In order to get the above enlightments to work, you have to add hv_time (which is translated to hypervclock in libvirt) and hv_vpindex enlightments too.

My final libvirt XML file looks like

  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vpindex state='on'/>
      <synic state='on'/>
      <stimer state='on'/>
    </hyperv>
    <smm state='on'/>
  </features>

  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='discard'/>
    <timer name='hpet' present='yes'/>
    <timer name='hypervclock' present='yes'/>
  </clock>

Disclaimer: This has been done under CentOS 8, but should work under any libvirt controlled KVM.

Comments

[giox069]

It works! Host CPU load is reduced a lot. I have also found another post on reddit which suggested to remove the following parameters from qemu commandline: -no-hpet -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard which translates into a slightly different .xml for libvirtd: <timer name='rtc' tickpolicy='delay'/> <timer name='pit' tickpolicy='catchup'/> <timer name='hpet' present='yes'/> Also with the above rtc/pit settings host cpu usage is reduced. VM general performances seems to be a bit lower than windows 1709.

[giox069]

It also seems that <timer name='pit' tickpolicy='discard'/> (instead of 'delay') improves general VM performances. Ubuntu 16.04 does not support it. I upgraded to 18.04.

[Costin Gușă]

for me enabling <synic state='on'> and <stimer state='on'> lowered the cpu usage to under 3%, with almost all cycles used by kernel kvm threads (looking at threads with top, qemu-kvm-ev uses zero cpu on rhel7.5 and kernel 4.16 from fedora28 for a q35 uefi win10 vm); source bugs.launchpad.net/qemu/+bug/1775702