Forked from krzys-h/Hyper-V GPU-PV with Linux guest.md
Created
August 8, 2025 16:25
-
-
Save onixus74/b5d1cec9339d66430b4cd6e4192ed331 to your computer and use it in GitHub Desktop.
Revisions
-
krzys-h revised this gist
Nov 26, 2021 . 1 changed file with 9 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -7,16 +7,20 @@ Well... not quite. I managed to get it to run... but not well. ## How to do it? 1. Verify driver support Run `Get-VMHostPartitionableGpu` in PowerShell. You should see your graphics card listed, if you get nothing, update your graphics drivers and try again. 2. Create a new VM in Hyper-V Manager. Make sure to: * Use Generation 2 * DISABLE dynamic memory (it interferes with vGPU on Windows so it probably won't work on Linux either, I didn't check this yet though) * DISABLE automatic snapshots (they are not supported with vGPU and will only cause problems) * DISABLE secure boot (we'll need custom kernel drivers, and I never tried to make this work with secure boot) * Don't forget to add more CPU cores because the stupid wizard still adds only one vCPU... 3. Add GPU-PV adapter From PowerShell running as administrator: ```powershell Set-VM -VMName <vmname> -GuestControlledCacheTypes $true -LowMemoryMappedIoSpace 1GB -HighMemoryMappedIoSpace 32GB @@ -26,6 +30,7 @@ Well... not quite. I managed to get it to run... but not well. 4. Install Ubuntu 21.04 in the VM like you would usually 5. Build the `dxgkrnl` driver Until Microsoft upstreams this driver to the mainline Linux kernel, you will have to build it manually. Use the following script I made to get the driver from the WSL2-Linux-Kernel tree, patch it for out-of-tree build and add it to DKMS: ```sh #!/bin/bash -e @@ -64,6 +69,7 @@ Well... not quite. I managed to get it to run... but not well. ``` 6. Copy GPU drivers from your host system Now you will also need to copy some files from the host machine: the closed-source D3D12 implementation provided by Microsoft, as well as Linux parts of the graphics driver provided by your GPU vendor. If you ever tried to run GPU-PV with a Windows guest, this part should look familiar. Figuring out how to transfer the files into the VM is left as an exercise to the reader, I'll just assume that your Windows host volume is available at `/mnt` for simplicity: ```sh mkdir -p /usr/lib/wsl/{lib,drivers} @@ -74,12 +80,14 @@ Well... not quite. I managed to get it to run... but not well. Note: You will need to repeat this step every time you update Windows or your graphics drivers 7. Set up the system to be able to load libraries from `/usr/lib/wsl/lib/`: ```sh echo "/usr/lib/wsl/lib" > /etc/ld.so.conf.d/ld.wsl.conf ldconfig # (if you get 'libcuda.so.1 is not a symbolic link', just ignore it) ``` 8. Workaround a bug in the D3D12 implementation (it assumes that the `/usr/lib/wsl/lib/` mount is case-insensitive... just Windows things...) ```sh ln -s /usr/lib/wsl/lib/libd3d12core.so /usr/lib/wsl/lib/libD3D12Core.so ``` -
krzys-h revised this gist
Nov 26, 2021 . 1 changed file with 8 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -8,19 +8,23 @@ Well... not quite. I managed to get it to run... but not well. 1. Verify driver support Run `Get-VMHostPartitionableGpu` in PowerShell. You should see your graphics card listed, if you get nothing, update your graphics drivers and try again. 2. Create a new VM in Hyper-V Manager. Make sure to: * Use Generation 2 * DISABLE dynamic memory (it interferes with vGPU on Windows so it probably won't work on Linux either, I didn't check this yet though) * DISABLE automatic snapshots (they are not supported with vGPU and will only cause problems) * DISABLE secure boot (we'll need custom kernel drivers, and I never tried to make this work with secure boot) * Don't forget to add more CPU cores because the stupid wizard still adds only one vCPU... 3. Add GPU-PV adapter From PowerShell running as administrator: ```powershell Set-VM -VMName <vmname> -GuestControlledCacheTypes $true -LowMemoryMappedIoSpace 1GB -HighMemoryMappedIoSpace 32GB Add-VMGpuPartitionAdapter -VMName <vmname> ``` 4. Install Ubuntu 21.04 in the VM like you would usually 5. Build the `dxgkrnl` driver Until Microsoft upstreams this driver to the mainline Linux kernel, you will have to build it manually. Use the following script I made to get the driver from the WSL2-Linux-Kernel tree, patch it for out-of-tree build and add it to DKMS: ```sh @@ -58,6 +62,7 @@ Well... not quite. I managed to get it to run... but not well. dkms build dxgkrnl/$VERSION dkms install dxgkrnl/$VERSION ``` 6. Copy GPU drivers from your host system Now you will also need to copy some files from the host machine: the closed-source D3D12 implementation provided by Microsoft, as well as Linux parts of the graphics driver provided by your GPU vendor. If you ever tried to run GPU-PV with a Windows guest, this part should look familiar. Figuring out how to transfer the files into the VM is left as an exercise to the reader, I'll just assume that your Windows host volume is available at `/mnt` for simplicity: ```sh @@ -67,15 +72,18 @@ Well... not quite. I managed to get it to run... but not well. chmod -R 0555 /usr/lib/wsl ``` Note: You will need to repeat this step every time you update Windows or your graphics drivers 7. Set up the system to be able to load libraries from `/usr/lib/wsl/lib/`: ```sh echo "/usr/lib/wsl/lib" > /etc/ld.so.conf.d/ld.wsl.conf ldconfig # (if you get 'libcuda.so.1 is not a symbolic link', just ignore it) ``` 8. Workaround a bug in the D3D12 implementation (it assumes that the `/usr/lib/wsl/lib/` mount is case-insensitive... just Windows things...) ```sh ln -s /usr/lib/wsl/lib/libd3d12core.so /usr/lib/wsl/lib/libD3D12Core.so ``` 9. Reboot the VM If you've done everything correctly, `glxinfo | grep "OpenGL renderer string"` should display `D3D12 (Your GPU Name)`. If it does not, here are some useful commands for debugging: -
krzys-h renamed this gist
Nov 26, 2021 . 1 changed file with 0 additions and 0 deletions.There are no files selected for viewing
File renamed without changes. -
krzys-h created this gist
Nov 26, 2021 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,91 @@ # Ubuntu 21.04 VM with GPU acceleration under Hyper-V...? Modern versions of Windows support GPU paravirtualization in Hyper-V with normal consumer graphics cards. This is used e.g. for graphics acceleration in Windows Sandbox, as well as WSLg. In some cases, it may be useful to create a normal VM with GPU acceleration using this feature, but this is not officially supported. People [already figured out how to do it with Windows guests](https://mu0.cc/2020/08/25/hyperv-gpupv/) though, so why not do the same with Linux? It should be easy given that WSLg is open source and [reasonably well documented](https://devblogs.microsoft.com/directx/directx-heart-linux/), right? Well... not quite. I managed to get it to run... but not well. ## How to do it? 1. Verify driver support Run `Get-VMHostPartitionableGpu` in PowerShell. You should see your graphics card listed, if you get nothing, update your graphics drivers and try again. 2. Create a new VM in Hyper-V Manager. Make sure to: * Use Generation 2 * DISABLE dynamic memory (it interferes with vGPU on Windows so it probably won't work on Linux either, I didn't check this yet though) * DISABLE automatic snapshots (they are not supported with vGPU and will only cause problems) * DISABLE secure boot (we'll need custom kernel drivers, and I never tried to make this work with secure boot) * Don't forget to add more CPU cores because the stupid wizard still adds only one vCPU... 3. Add GPU-PV adapter From PowerShell running as administrator: ```powershell Set-VM -VMName <vmname> -GuestControlledCacheTypes $true -LowMemoryMappedIoSpace 1GB -HighMemoryMappedIoSpace 32GB Add-VMGpuPartitionAdapter -VMName <vmname> ``` 4. Install Ubuntu 21.04 in the VM like you would usually 5. Build the `dxgkrnl` driver Until Microsoft upstreams this driver to the mainline Linux kernel, you will have to build it manually. Use the following script I made to get the driver from the WSL2-Linux-Kernel tree, patch it for out-of-tree build and add it to DKMS: ```sh #!/bin/bash -e BRANCH=linux-msft-wsl-5.10.y if [ "$EUID" -ne 0 ]; then echo "Swithing to root..." exec sudo $0 "$@" fi apt-get install -y git dkms git clone -b $BRANCH --depth=1 https://github.com/microsoft/WSL2-Linux-Kernel cd WSL2-Linux-Kernel VERSION=$(git rev-parse --short HEAD) cp -r drivers/hv/dxgkrnl /usr/src/dxgkrnl-$VERSION mkdir -p /usr/src/dxgkrnl-$VERSION/inc/{uapi/misc,linux} cp include/uapi/misc/d3dkmthk.h /usr/src/dxgkrnl-$VERSION/inc/uapi/misc/d3dkmthk.h cp include/linux/hyperv.h /usr/src/dxgkrnl-$VERSION/inc/linux/hyperv_dxgkrnl.h sed -i 's/\$(CONFIG_DXGKRNL)/m/' /usr/src/dxgkrnl-$VERSION/Makefile sed -i 's#linux/hyperv.h#linux/hyperv_dxgkrnl.h#' /usr/src/dxgkrnl-$VERSION/dxgmodule.c echo "EXTRA_CFLAGS=-I\$(PWD)/inc" >> /usr/src/dxgkrnl-$VERSION/Makefile cat > /usr/src/dxgkrnl-$VERSION/dkms.conf <<EOF PACKAGE_NAME="dxgkrnl" PACKAGE_VERSION="$VERSION" BUILT_MODULE_NAME="dxgkrnl" DEST_MODULE_LOCATION="/kernel/drivers/hv/dxgkrnl/" AUTOINSTALL="yes" EOF dkms add dxgkrnl/$VERSION dkms build dxgkrnl/$VERSION dkms install dxgkrnl/$VERSION ``` 6. Copy GPU drivers from your host system Now you will also need to copy some files from the host machine: the closed-source D3D12 implementation provided by Microsoft, as well as Linux parts of the graphics driver provided by your GPU vendor. If you ever tried to run GPU-PV with a Windows guest, this part should look familiar. Figuring out how to transfer the files into the VM is left as an exercise to the reader, I'll just assume that your Windows host volume is available at `/mnt` for simplicity: ```sh mkdir -p /usr/lib/wsl/{lib,drivers} cp -r /mnt/Windows/system32/lxss/lib/* /usr/lib/wsl/lib/ cp -r /mnt/Windows/system32/DriverStore/FileRepository/nv_dispi.inf_amd64_* /usr/lib/wsl/drivers/ # this may be different for different GPU vendors, refer to tutorials for Windows guests if needed chmod -R 0555 /usr/lib/wsl ``` Note: You will need to repeat this step every time you update Windows or your graphics drivers 7. Set up the system to be able to load libraries from `/usr/lib/wsl/lib/`: ```sh echo "/usr/lib/wsl/lib" > /etc/ld.so.conf.d/ld.wsl.conf ldconfig # (if you get 'libcuda.so.1 is not a symbolic link', just ignore it) ``` 8. Workaround a bug in the D3D12 implementation (it assumes that the `/usr/lib/wsl/lib/` mount is case-insensitive... just Windows things...) ```sh ln -s /usr/lib/wsl/lib/libd3d12core.so /usr/lib/wsl/lib/libD3D12Core.so ``` 9. Reboot the VM If you've done everything correctly, `glxinfo | grep "OpenGL renderer string"` should display `D3D12 (Your GPU Name)`. If it does not, here are some useful commands for debugging: ```sh sudo lspci -v # should list the vGPU and the dxgkrnl driver ls -l /dev/dxg # should exist if the dxgkrnl /usr/lib/wsl/lib/nvidia-smi # should be able to not fail :P ``` ## The problems 1. The thing is UNSTABLE. Just running `glxgears` crashes GNOME, spectacularly. I'd recommend switching to a simple window manager like i3 for testing. 2. GPU acceleration doesn't seem to be picked up everywhere, sometimes it falls back to software rendering with llvmpipe for no apparent reason 3. While when it works you can clearly see that the GPU is working from the FPS counter... I didn't figure out a good way to get these frames from the VM at a reasonable rate yet! The Hyper-V virtual display is slow, and even if you get enhanced session to work it's just RDP under the hood which is not really designed for high FPS output either. On Windows, you can simply use something like Parsec to connect to the VM, but all streaming solutions I know of don't work on a Linux host at all.