Skip to content

Instantly share code, notes, and snippets.

@silvanshade

silvanshade/qemu-vulkan-virtio.md

Forked from peppergrayxyz/qemu-vulkan-virtio.md
Created July 19, 2025 16:53
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save silvanshade/21777cc779abb2f0f3692c2ffd9f5aa0 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save silvanshade/21777cc779abb2f0f3692c2ffd9f5aa0 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @peppergrayxyz peppergrayxyz revised this gist Dec 18, 2024. 1 changed file with 92 additions and 9 deletions.
    101 changes: 92 additions & 9 deletions qemu-vulkan-virtio.md
    View file
    Original file line number Diff line number Diff line change
    @@ -31,6 +31,8 @@ Make sure you have the proper version installed on the host:
    - linux kernel >= 6.13 built with CONFIG_UDMABUF
    - working Vulkan and kvm setup
    - qemu >= 9.2.0
    - virglrenderer with enabled venus support
    - mesa >= 24.2.0

    You can verify this like so:
    ```
    @@ -45,14 +47,15 @@ QEMU emulator version 9.2.0
    Copyright (c) 2003-2024 Fabrice Bellard and the QEMU Project developers
    ```

    For Vulkan to work you need the proper drivers to be installed for your graphics card. To verfiy your setup, install `vulkan-tools`:
    ```
    $ vulkaninfo --summary
    ==========
    VULKANINFO
    ==========
    Check your distros package sources how they build virglrenderer.

    For Vulkan to work you need the proper drivers to be installed for your graphics card.

    Vulkan Instance Version: ...
    To verfiy your setup, install `vulkan-tools`. Make sure mesa >= 24.2.0 and test vkcube:
    ```
    $ vulkaninfo --summary | grep driverInfo
    driverInfo = Mesa 24.2.3-1ubuntu1
    driverInfo = Mesa 24.2.3-1ubuntu1 (LLVM 19.1.0)
    ...
    $ vkcube
    Selected GPU x: ..., type: ...
    @@ -76,7 +79,8 @@ mkdir build && cd build
    make -j4
    ```

    The configuration step will throgh errors if packages are missing. Check the qemu wiki for further info what to install: https://wiki.qemu.org/Hosts/Linux
    The configuration step will throgh errors if packages are missing.
    Check the qemu wiki for further info what to install: https://wiki.qemu.org/Hosts/Linux


    ## Create and run an image for QEMU
    @@ -134,12 +138,91 @@ Selected GPU x: ..., type: ...

    If the deive is ```llvmpipe``` somehting is wrong. The device should be ```virgl (...)```.

    #### Troubleshooting
    ## Troubleshooting

    - (host) add ```-d guest_errors``` to show error messages from the guest
    - (guest) try installing vulkan virtio drivers and mesa
    - check the original [blog post](https://www.collabora.com/news-and-blog/blog/2021/11/26/venus-on-qemu-enabling-new-virtual-vulkan-driver/)

    ### Ubuntu 24.10

    This is how you do it on Ubuntu

    #### kernel

    Install mainline: https://github.com/bkw777/mainline

    ```
    sudo add-apt-repository ppa:cappelikan/ppa
    sudo apt update
    sudo apt install mainline
    ```

    find the latest kernel (>= 6.13), at the time of writing 6.13 is a release candidate, so include those:

    `$ mainline check --include-rc`

    Install kernel:

    `$ sudo mainline install 6.13-rc1`

    Verfify installed kernels:
    ```
    $ mainline list-installed
    mainline 1.4.10
    Installed Kernels:
    linux-image-6.11.0-13-generic
    linux-image-generic-hwe-24.04
    linux-image-unsigned-6.13.0-061300rc1-generic
    mainline: done
    ```

    reboot into new kernel

    verify running kernel
    ```
    $ uname -r
    6.13.0-061300rc1-generic
    ```

    #### virglrenderer

    the ubuntu package is not compiled with the proper flags.

    If installed remove it: `$ sudo apt-get remove libvirglrenderer-dev`

    download, build & install from source with venus enabled
    ```
    wget https://gitlab.freedesktop.org/virgl/virglrenderer/-/archive/1.1.0/virglrenderer-1.1.0.tar.gz
    sudo apt-get install python3-full ninja-build libvulkan-dev libva-dev
    python3 -m venv venv
    venv/bin/pip install meson
    venv/bin/meson build -Dvideo=true -Dvenus=true
    ninja -C build
    ninja install
    ```

    #### qemu

    install qemu >= 9.2.0, at the time of writing ubuntu has not yet packaged it

    Install build depdencies: https://wiki.qemu.org/Hosts/Linux
    ```
    sudo apt-get install build-essential pip libslirp-dev slirp
    sudo apt-get install git libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev ninja-build
    sudo apt-get install git-email
    sudo apt-get install libaio-dev libbluetooth-dev libcapstone-dev libbrlapi-dev libbz2-dev
    sudo apt-get install libcap-ng-dev libcurl4-gnutls-dev libgtk-3-dev
    sudo apt-get install libibverbs-dev libjpeg8-dev libncurses5-dev libnuma-dev
    sudo apt-get install librbd-dev librdmacm-dev
    sudo apt-get install libsasl2-dev libsdl2-dev libseccomp-dev libsnappy-dev libssh-dev
    sudo apt-get install libvde-dev libvdeplug-dev libvte-2.91-dev libxen-dev liblzo2-dev
    sudo apt-get install valgrind xfslibs-dev
    sudo apt-get install libnfs-dev libiscsi-dev
    ```

    build and run as described

    ## virt-manager

    -- work in progress --
  2. @peppergrayxyz peppergrayxyz created this gist Dec 11, 2024.
    174 changes: 174 additions & 0 deletions qemu-vulkan-virtio.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,174 @@
    # QEMU with VirtIO GPU Vulkan Support

    With its latest reales qemu added the Venus patches so that virtio-gpu now support venus encapsulation for vulkan. This is one more piece to the puzzle towards full Vulkan support.

    An outdated blog post on [clollabora](https://www.collabora.com/news-and-blog/blog/2021/11/26/venus-on-qemu-enabling-new-virtual-vulkan-driver/) described in 2021 how to enable 3D acceleration of Vulkan applications in QEMU through the Venus experimental Vulkan driver for VirtIO-GPU with a local development environment. Following up on the outdated write up, this is how its done today.

    ## Definitions

    Let's start with the brief description of the projects mentioned in the post & extend them:

    - QEMU is a machine emulator
    - VirGL is an OpenGL driver for VirtIO-GPU, available in Mesa.
    - Venus is an experimental Vulkan driver for VirtIO-GPU, also available in Mesa.
    - Virglrenderer is a library that enables hardware acceleration to VM guests, effectively translating commands from the two drivers just mentioned to either OpenGL or Vulkan.
    - libvirt is an API for managing platform virtualization
    - virt-manager is a desktop user interface for managing virtual machines through libvirt

    Merged Patches:
    - 2024-08-14 [venus: make cross-device optional](https://gitlab.freedesktop.org/mesa/mesa/-/commit/087e9a96d13155e26987befae78b6ccbb7ae242b) merged in [mesa 24.2](https://www.phoronix.com/news/Mesa-24.2-Released)
    - 2024-11-25 [KVM: Stop grabbing references to PFNMAP'd pages](https://lore.kernel.org/all/[email protected]/) merged in [linux 6.13](https://www.phoronix.com/news/Linux-6.13-KVM)
    - 2024-11-12 [Support blob memory and venus on qemu](https://lists.gnu.org/archive/html/qemu-devel/2024-08/msg03288.html) merged in [qemu 9.2.0](https://www.phoronix.com/news/QEMU-9.2-Released)

    Work in progress:
    - libvirt [Add support for more virtio-vga-gl arguments #638](https://gitlab.com/libvirt/libvirt/-/issues/638)
    - virt-manager [Add support for Venus / Vulkan VirtIO-GPU driver #362](https://github.com/virt-manager/virt-manager/issues/362)

    ## Prerequisites

    Make sure you have the proper version installed on the host:

    - linux kernel >= 6.13 built with CONFIG_UDMABUF
    - working Vulkan and kvm setup
    - qemu >= 9.2.0

    You can verify this like so:
    ```
    $ uname -r
    6.13.0
    $ ls /dev/udmabuf
    /dev/udmabuf
    $ ls /dev/kvm
    /dev/kvm
    $ qemu-system-x86_64 --version
    QEMU emulator version 9.2.0
    Copyright (c) 2003-2024 Fabrice Bellard and the QEMU Project developers
    ```

    For Vulkan to work you need the proper drivers to be installed for your graphics card. To verfiy your setup, install `vulkan-tools`:
    ```
    $ vulkaninfo --summary
    ==========
    VULKANINFO
    ==========
    Vulkan Instance Version: ...
    ...
    $ vkcube
    Selected GPU x: ..., type: ...
    ```

    #### Building qemu

    If your distro doesn't (yet) ship and updated version of qemu, you can build it yourself from source:

    ```
    wget https://download.qemu.org/qemu-9.2.0.tar.xz
    tar xvJf qemu-9.2.0.tar.xz
    cd qemu-9.2.0
    mkdir build && cd build
    ../configure --target-list=x86_64-softmmu \
    --enable-kvm \
    --enable-opengl \
    --enable-virglrenderer \
    --enable-gtk \
    --enable-sdl
    make -j4
    ```

    The configuration step will throgh errors if packages are missing. Check the qemu wiki for further info what to install: https://wiki.qemu.org/Hosts/Linux


    ## Create and run an image for QEMU

    Create an image & fetch the distro of your choice:

    ### Host

    ```bash
    ISO=ubuntu-24.10-desktop-amd64.iso
    wget https://releases.ubuntu.com/oracular/ubuntu-24.10-desktop-amd64.iso

    IMG=ubuntu-24-10.qcow2
    qemu-img create -f qcow2 $IMG 16G
    ```

    Run a live version or install the distro

    ```
    qemu-system-x86_64 \
    -enable-kvm \
    -M q35 \
    -smp 4 \
    -m 4G \
    -cpu host \
    -net nic,model=virtio \
    -net user,hostfwd=tcp::2222-:22 \
    -device virtio-vga-gl,hostmem=4G,blob=true,venus=true \
    -vga none \
    -display gtk,gl=on,show-cursor=on \
    -usb -device usb-tablet \
    -object memory-backend-memfd,id=mem1,size=4G \
    -machine memory-backend=mem1 \
    -hda $IMG \
    -cdrom $ISO
    ```

    Adjust the parameters accordingly:
    - smp: number of cpu cores
    - m: RAM
    - hostmem,size: VRAM

    ### Guest

    Install ```mesa-utilites``` and ```vulkan-tools``` to test the setup:

    ```
    $ glxinfo -B
    ```

    ```
    $ vkcube
    Selected GPU x: ..., type: ...
    ```

    If the deive is ```llvmpipe``` somehting is wrong. The device should be ```virgl (...)```.

    #### Troubleshooting

    - (host) add ```-d guest_errors``` to show error messages from the guest
    - (guest) try installing vulkan virtio drivers and mesa
    - check the original [blog post](https://www.collabora.com/news-and-blog/blog/2021/11/26/venus-on-qemu-enabling-new-virtual-vulkan-driver/)

    ## virt-manager

    -- work in progress --

    Currently this is work in progress, so there is no option to add vulkan support in virt-manager. There are no fields to configure this. Also xml doesnt work, because libvirt doesn't know about these options either, so xml validation fails. There is however an option for [QEMU command-line passthrough](https://libvirt.org/kbase/qemu-passthrough-security.html) which bypasses the validation.

    If you setup a default machine with 4G of memory, you can do this:

    ```xml
    <qemu:commandline>
    <qemu:arg value="-device"/>
    <qemu:arg value="virtio-vga-gl,hostmem=4G,blob=true,venus=true"/>
    <qemu:arg value="-object"/>
    <qemu:arg value="memory-backend-memfd,id=mem1,size=4G"/>
    <qemu:arg value="-machine"/>
    <qemu:arg value="memory-backend=mem1"/>
    <qemu:arg value="-vga"/>
    <qemu:arg value="none"/>
    </qemu:commandline>
    ```

    Which gives this error:
    ```
    qemu-system-x86_64: virgl could not be initialized: -1
    ```

    Changing the number from 4G to 4194304k (same as memory) leds to this error:
    ```
    qemu-system-x86_64: Spice: ../spice-0.15.2/server/red-qxl.cpp:435:spice_qxl_gl_scanout: condition `qxl_state->gl_draw_cookie == GL_DRAW_COOKIE_INVALID' failed
    ```

    to be further investigated.