VGA passthrough with QEMU and KVM - Nothing on screen

Question

These last few days I've been trying to set up a windows virtual machine with VGA passthrough, using this guide as my main reference: https://www.pugetsystems.com/labs/articles/Multiheaded-NVIDIA-Gaming-using-Ubuntu-14-04-KVM-585/

My pc uses an intel i7-4790 with an integrated intel HD 4600, which i use as my primary card, and an NVIDIA GTX 960 which is the one i want to pass throug.

I installed the NVIDIA drivers, and i managed to bind the nvidia card to the vfio-pci controller without any problem, after disabling hybrid graphics in the nvidia settings.

I tried to blacklist the nvidia drivers and use pci-stub to "steal" the GPU from them, so i could avoid any possible errors, but it didn't work, so i manually unload the nvidia modules after the OS starts, using this script(in console mode):

#!/bin/bash
sudo service lightdm stop
sudo rmmod nvidia_uvm
sudo rmmod nvidia_drm
sudo rmmod nvidia_modeset
sudo rmmod nvidia
sudo service lightdm start

I checked that this worked by removing the device(gpu), and verifying that the card was being claimed by vfio-pci instead of nvidia drivers when i rescanned.

After i had all the drivers set up, i started the VM with this script:

#!/bin/bash

sudo qemu-system-x86_64 -enable-kvm -M q35 -m 4096 -cpu host,kvm=off \
-smp 4,sockets=1,cores=4,threads=1 \
-bios /usr/share/seabios/bios.bin -vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,x-vga=on \
-device virtio-scsi-pci,id=scsi \
-drive file=/home/xanty/windows1.img,id=disk,format=raw,cache=none,if=none -device scsi-hd,drive=disk \
-drive file=/home/xanty/Descargas/windows7pro.iso,id=isocd,cache=none,if=none -device scsi-cd,drive=isocd \
-boot menu=on

exit 0

And i finally got it to run, and when checking the pci info on the qemu monitor (pic here), everything looks fine, but the second monitor i have attached to my nvidia card's DVI-I port doesn't show anything. The monitor doesn't seem to receive any signal.

I need some help to figure out how to solve this, or at least find a viable alternative to this method.

As additional info, i'm using Ubuntu Studio 16.04, my first monitor is a Samsung monitor plugged to the Intel DVI-I port, and my second monitor is a BENQ with one VGA and one DVI-I port, and i have it plugged to the GTX 960 DVI-I port (this model doesn't have VGA port)

I really appreciate any help you can provide.

EDIT 1:

Usually, before i try to launch the vm, and after i remove the nvidia modules, i remove the device and rescan it:

echo > 1 /sys/bus/pci/devices/0000:01:00.0/remove
echo > 1 /sys/bus/pci/devices/0000:01:00.1/remove
echo > 1 /sys/bus/pci/rescan

If i do this, the vm seems to run without major problems, but when i don't, the qemu monitor freezes when i try to start it, and i can't close it or stop the process unless i "kill -9" it. This is because of the line:

-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,x-vga=on \

If i remove the "x-vga=on" parameter it still works fine.

EDIT 2: After some further testing and researching on this issue, i discovered that tha vfio-pci driver wasn't enabling the gpu when i launch kvm.

After starting kvm, "dmesg|grep -i vfio" command returns these new lines:

[ 1572.975051] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1e@0x258
[ 1572.975063] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900

There should be another line similar to this one :

[ 3182.192258] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)

But it's missing.

EDIT3: I just downgraded to Ubuntu Studio 14.04, and i finally was able to make pci-stub grab the gpu before the nvidia/nouveau drivers. I can bind the gpu to vfio-pci drivers easily and without crashes, and the output after launching kvm looks like this:

[  130.988082] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[  130.988186] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1e@0x258
[  130.988196] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900

But after all this, the monitor still doesn't get any signal. I really need some help. My only option left is to try downgrading my kernel from 4.2.0-27-lowlatency to 4.1.x, and i'm worried it will screw up my OS.

Solution

SOLVED.

After much researching, i noticed 2 main problems:

1- There was a problem with my integrated graphics card i915 drivers, and i found out that i either had to patch my kernel with an unnofficial patch, or install my virtual machine on UEFI mode, and simulate a bios startup. For this i had to instal ovmf.

apt-get install ovmf

2- The windows nvidia drivers were detecting that i was running windows on kvm and where blocking my gpu.

In the end, after installing ovmf, i followed this tutorial on how to set it up using virt-manager and ovmf: http://vfio.blogspot.com.es/2015/05/vfio-gpu-how-to-series-part-4-our-first.html

It also explains how to edit the xml configuration of the machine to obfuscate kvm from the nvidia drivers.

I also found out a funny feature:

If i select an vga startup option (VNC server) to emulate the machine on a window, once windows starts, it detects the window as a second physical monitor, allowing me to capture mouse and keyboard without the need to passthrough the usb controllers.

Comments

[elbarna]

Thanks,please can you put the correct command line with vnc server?

[Xanty]

I ended up using virt-manager, like the tutorial says. I just added a basic vnc server with default settings, a virtual graphics card (cirrus) and obviously virtual mouse and keyboard. This only served to create a window in which i could capture my mouse. The virtual machine would just detect this window as a second monitor. This method can cause some bugs if you try to press keys and move the mouse simultaneously, so i would advise you to capture your usb controller directly instead if you want to play videogames or something similar.

[elbarna]

I made a thing similiar but the second monitor(the monitor with "3d") show only the desktop background..and every thing i open is opened but i cannot see it(is like is over the screen!)

[elbarna]

Another thing,on youtube and similar i see the only success are with intel integrated on host,i have nvidia nouveau on host,maybe is possible works only with intel?Do you know a good mobo with intel + iommu +...amd cpu?

[Xanty]

You need 2 GPUs, as one of the cards is reserved by the machine for the VGA passthrough. You can do it with 2 PCIe cards or one integrated and one PCIe. In this last case, you have to save the integrated card for the host and save the other for the VM. For this to work it is necessary to have the integrated GPU as your primary GPU for the host system (the monitor of your host must be connected to the integrated card port) and check that your integrated card isn't making use of the other card (nvidia x server settings->PRIME profiles->Nvidia mode must be unchecked).

[Xanty]

If you try to use your integrated card for the VM, or you are using your integrated card for the host, but your configuration allows it to make use of the other GPU instead of using integrated graphics, then your virtual machine and the host will "fight" for the control of that card, and it can result in either the machine or your system freezing or crashing. With the above configuration i used for the official nvidia drivers it worked fine for me, but if you use nouveau you will have to blacklist it to prevent it from grabbing the nvidia GPU during boot time.

[elbarna]

I use two different cards,all pci-e one is nvidia geforce 210 for host other is ati amd 5000 hd for guest