Problems with KVM+GPU passthrough

If none of the more specific forums is the right place to ask

Problems with KVM+GPU passthrough

Postby jen140 » 2017-12-22 13:22

Hello.

Please read everything before you answer. Thank you.
And yes, I've updated initramfs(update-initramfs -u), updated the grub config(grub-mkconfig -o /boot/grub/grub.cfg) and rebooted the machine multiple times.

I've been trying to setup KVM with GPU passthrough but I'm unable to do so.

Hardware:
Processor: Intel i7-7700K
Motherboard: Asus ex-8250-v7
GPU's: GeForce GTX 970, GeForce GTX 660

In the bios the only option that I have about vt-x/vt-d is "Intel VT-x Thchnology" that is Supported.

Software:
Debian 9
qemu-kvm 1:2.8

What I want to do:
I'd like to have this machine running with kvm and two windows 10 VM's, both having each one of the GPU's. I'm not interested in running any desktop manager on the host(and I don't have xorg installed).

The problem:
virt-manager says "unsupported configuration: host doesn't support passthrough of host PCI devices".

/etc/defaults/grub:
Code: Select all
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on vfio-pci.ids=10de:13c2,10de:0fbb,10de:11c0,10de:0e0b"
GRUB_CMDLINE_LINUX=""


/etc/modules:
Code: Select all
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd


/etc/modprobe.d/nouveau.conf:
Code: Select all
blacklist nouveau


/etc/modprobe.d/vfio.conf:
Code: Select all
options vfio-pci ids=10de:13c2,10de:0fbb,10de:11c0,10de:0e0b


/etc/initramfs-tools/modules:
Code: Select all
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd


egrep -q '^flags.*(svm|vmx)' /proc/cpuinfo && echo virtualization extensions available:
Code: Select all
virtualization extensions available


ls -liah /sys/kernel/iommu_groups/:
Code: Select all
total 0
2708 drwxr-xr-x  2 root root 0 Dec 22 14:09 .
2676 drwxr-xr-x 10 root root 0 Dec 22 13:14 ..


lspci -nn:
Code: Select all
00:00.0 Host bridge [0600]: Intel Corporation Device [8086:591f] (rev 05)
00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe Controller (x16) [8086:1901] (rev 05)
00:14.0 USB controller [0c03]: Intel Corporation Device [8086:a2af]
00:16.0 Communication controller [0780]: Intel Corporation Device [8086:a2ba]
00:17.0 SATA controller [0106]: Intel Corporation Device [8086:a282]
00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:a294] (rev f0)
00:1d.0 PCI bridge [0604]: Intel Corporation Device [8086:a298] (rev f0)
00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a2c8]
00:1f.2 Memory controller [0580]: Intel Corporation Device [8086:a2a1]
00:1f.3 Audio device [0403]: Intel Corporation Device [8086:a2f0]
00:1f.4 SMBus [0c05]: Intel Corporation Device [8086:a2a3]
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-V [8086:15b8]
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)
03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK106 [GeForce GTX 660] [10de:11c0] (rev a1)
03:00.1 Audio device [0403]: NVIDIA Corporation GK106 HDMI Audio Controller [10de:0e0b] (rev a1)


lspci -k:
Code: Select all
00:00.0 Host bridge: Intel Corporation Device 591f (rev 05)
   Subsystem: ASUSTeK Computer Inc. Device 8694
00:01.0 PCI bridge: Intel Corporation Skylake PCIe Controller (x16) (rev 05)
   Kernel driver in use: pcieport
   Kernel modules: shpchp
00:14.0 USB controller: Intel Corporation Device a2af
   Subsystem: ASUSTeK Computer Inc. Device 8694
   Kernel driver in use: xhci_hcd
   Kernel modules: xhci_pci
00:16.0 Communication controller: Intel Corporation Device a2ba
   Subsystem: ASUSTeK Computer Inc. Device 8694
   Kernel driver in use: mei_me
   Kernel modules: mei_me
00:17.0 SATA controller: Intel Corporation Device a282
   Subsystem: ASUSTeK Computer Inc. Device 8694
   Kernel driver in use: ahci
   Kernel modules: ahci
00:1c.0 PCI bridge: Intel Corporation Device a294 (rev f0)
   Kernel driver in use: pcieport
   Kernel modules: shpchp
00:1d.0 PCI bridge: Intel Corporation Device a298 (rev f0)
   Kernel driver in use: pcieport
   Kernel modules: shpchp
00:1f.0 ISA bridge: Intel Corporation Device a2c8
   Subsystem: ASUSTeK Computer Inc. Device 8694
00:1f.2 Memory controller: Intel Corporation Device a2a1
   Subsystem: ASUSTeK Computer Inc. Device 8694
00:1f.3 Audio device: Intel Corporation Device a2f0
   Subsystem: ASUSTeK Computer Inc. Device 86ee
   Kernel driver in use: snd_hda_intel
   Kernel modules: snd_hda_intel
00:1f.4 SMBus: Intel Corporation Device a2a3
   Subsystem: ASUSTeK Computer Inc. Device 8694
   Kernel driver in use: i801_smbus
   Kernel modules: i2c_i801
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-V
   Subsystem: ASUSTeK Computer Inc. Ethernet Connection (2) I219-V
   Kernel driver in use: e1000e
   Kernel modules: e1000e
01:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
   Subsystem: Micro-Star International Co., Ltd. [MSI] GM204 [GeForce GTX 970]
   Kernel modules: nouveau
01:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1)
   Subsystem: Micro-Star International Co., Ltd. [MSI] GM204 High Definition Audio Controller
   Kernel driver in use: snd_hda_intel
   Kernel modules: snd_hda_intel
03:00.0 VGA compatible controller: NVIDIA Corporation GK106 [GeForce GTX 660] (rev a1)
   Subsystem: Micro-Star International Co., Ltd. [MSI] GK106 [GeForce GTX 660]
   Kernel modules: nouveau
03:00.1 Audio device: NVIDIA Corporation GK106 HDMI Audio Controller (rev a1)
   Subsystem: Micro-Star International Co., Ltd. [MSI] GK106 HDMI Audio Controller
   Kernel driver in use: snd_hda_intel
   Kernel modules: snd_hda_intel


dmesg |grep -i vfio:
Code: Select all
[    1.862810] VFIO - User Level meta-driver version: 0.3
[    1.870620] vfio-pci: probe of 0000:01:00.0 failed with error -22
[    1.870627] vfio_pci: add [10de:13c2[ffff:ffff]] class 0x000000/00000000
[    1.870642] vfio-pci: probe of 0000:01:00.0 failed with error -22
[    1.870652] vfio-pci: probe of 0000:01:00.1 failed with error -22
[    1.870655] vfio_pci: add [10de:0fbb[ffff:ffff]] class 0x000000/00000000
[    1.870666] vfio-pci: probe of 0000:01:00.0 failed with error -22
[    1.870675] vfio-pci: probe of 0000:01:00.1 failed with error -22
[    1.870685] vfio-pci: probe of 0000:03:00.0 failed with error -22
[    1.870687] vfio_pci: add [10de:11c0[ffff:ffff]] class 0x000000/00000000
[    1.870699] vfio-pci: probe of 0000:01:00.0 failed with error -22
[    1.870707] vfio-pci: probe of 0000:01:00.1 failed with error -22
[    1.870716] vfio-pci: probe of 0000:03:00.0 failed with error -22
[    1.870726] vfio-pci: probe of 0000:03:00.1 failed with error -22
[    1.870729] vfio_pci: add [10de:0e0b[ffff:ffff]] class 0x000000/00000000


dmesg | grep -e DMAR -e IOMMU:
Code: Select all
[    0.000000] DMAR: IOMMU enabled
[    0.477982] AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.de>
[    0.477982] AMD IOMMUv2 functionality not available on this system



driver/unbind:
Code: Select all
echo '0000:01:00.0' | /sys/bus/pci/devices/0000\:01\:00.0/driver/unbind:
bash: /sys/bus/pci/devices/0000:01:00.0/driver/unbind: No such file or directory

echo '0000:01:00.1' | /sys/bus/pci/devices/0000\:01\:00.1/driver/unbind:
bash: /sys/bus/pci/devices/0000:01:00.1/driver/unbind: Permission denied

echo '0000:03:00.0' | /sys/bus/pci/devices/0000\:03\:00.0/driver/unbind
bash: /sys/bus/pci/devices/0000:03:00.0/driver/unbind: No such file or directory

echo '0000:03:00.1' | /sys/bus/pci/devices/0000\:03\:00.1/driver/unbind
bash: /sys/bus/pci/devices/0000:03:00.1/driver/unbind: Permission denied


lsmod | grep kvm:
Code: Select all
kvm_intel             192512  0
kvm                   589824  1 kvm_intel
irqbypass              16384  2 kvm,vfio_pci


lsmod | grep vfio:
Code: Select all
vfio_pci               45056  0
irqbypass              16384  2 kvm,vfio_pci
vfio_virqfd            16384  1 vfio_pci
vfio_iommu_type1       20480  0
vfio                   28672  2 vfio_iommu_type1,vfio_pci


Full dmesg:
https://paste.ee/p/Osq7K

Thanks for your attention.
jen140
 
Posts: 15
Joined: 2009-10-03 20:46

Return to General Questions

Who is online

Users browsing this forum: No registered users and 8 guests

fashionable