- hardware
Dell Poweredge R720
Nvidia Tesla P40 24GB
-
follow below link and stop prior section "Configuring the VM (Windows 10)", note the modifications listed below. https://gist.github.com/qubidt/64f617e959725e934992b080e677656f
-
in proxmox web interface, select vm, for hostpci - check
All Functions,ROM-Bar,PCI-Express -
for vm BIOS, use
Default (SeaBIOS). -
for vm Machine, use
q35. -
edit vm conf
/etc/pve/qemu-server/${VM_ID}.confper below
-
cpu: host,hidden=1,flags=+pcid
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
# below is based on needs
cores: 10
memory: 262144
scsi0: local-lvm:vm-100-disk-0,size=768G
boot: order=scsi0;net0
scsihw: virtio-scsi-pci
hostpci0: 0000:42:00,pcie=1
-
turn on the vm.
-
when installing ubuntu, don't install the driver.
-
boot up vm
-
check if gpu is present
lspci | grep 01:00
- install driver
sudo apt-add-repository -r ppa:graphics-drivers/ppa
sudo apt update
sudo apt remove nvidia*
sudo apt autoremove
sudo ubuntu-drivers autoinstall
- run
nvidia-smiand get complaints, run below.
sudo rmmod nouveau
sudo modprobe nvidia
# https://unix.stackexchange.com/questions/219059/remove-nouveau-driver-nvidia-without-rebooting
- run
nvidia-smito confirm presense of gpu.
--
-
enable "3rd Party Card fan behavior" - this allowed the GPU temp to hover at around 60C - while being used, Fan hovers in the range of 4200 to 7000 RPM. (typical GPU temp in a server room with AC will be around 55C under load). The Inlet and Exhaust Temp at 31 and 38C respectively (server is placed in the garage with no AC, with outdoor temp at 29C).
-
Lastly, please be mindful when purchasing the gpu power cable, there are 2 kinds of gpu power cables for dell servers PCI risers, one for the Nvidia Telsas and one for consumer "general purpose" GPUs. This is a step that SHALL NOT GO WRONG or else you may fry your server&GPU! read more here: https://kenmoini.com/post/2021/03/fun-with-servers-and-gpus