amontalban · July 28, 2023 16:31 · Feb 19, 2023 · Oct 27, 2020 · Oct 27, 2020
diff --git a/prevent-host-driver-in-pci-pass.md b/prevent-host-driver-in-pci-pass.md
@@ -30,6 +30,7 @@ There are two prerequisites:
 **The first is simple:**
  - add `vfio-pci` to `/etc/initramfs-tools/modules`
  - update initramfs: `update-initramfs -u -k $(uname -r)`
+ - **Proxmox on UEFI:** *if you're using Proxmox 7 booted using UEFI mode you also need to run `proxmox-boot-tool refresh`* 
  - it will place the module in `initramfs` disk (in `/etc/conf/modules`)
 
 **The second is more complicated:**
@@ -45,6 +46,7 @@ There are two prerequisites:
      - `chmod +x /usr/share/initramfs-tools/scripts/init-top/load_vfio-pci`
      - edit `/usr/share/initramfs-tools/scripts/init-top/udev` and change `PREREQS=""` to `PREREQS="load_vfio-pci"`
  - update initramfs: `update-initramfs -u -k $(uname -r)`
+ - **Proxmox on UEFI:** *if you're using Proxmox 7 booted using UEFI mode you also need to run `proxmox-boot-tool refresh`*
  - note: this **will not** work if placed in "standard place" (`/etc/initramfs-tools/scripts...`) as dependencies are not cross-directory and [`/usr/share` comes first](http://manpages.ubuntu.com/manpages/xenial/man8/initramfs-tools.8.html#boot%20scripts)
  
 ## Verify

diff --git a/prevent-host-driver-in-pci-pass.md b/prevent-host-driver-in-pci-pass.md
@@ -24,7 +24,7 @@ However, these will not work if your device is handled by something loaded very
 
 ## Solution
 There are two prerequisites:
-1. `vfio-pci` must load be availbale before rootfs is attached
+1. `vfio-pci` must be availbale before rootfs is attached
 2. `vfio-pci` must load before `ahci` loads
 
 **The first is simple:**

diff --git a/prevent-host-driver-in-pci-pass.md b/prevent-host-driver-in-pci-pass.md
@@ -0,0 +1,69 @@
+## Scenario
+You're running a KVM-based virtualization. You want to do PCI/PCIe passthrough of some device. You don't want it to attach to the host OS at all.
+
+Your device looks like that:
+
+```
+00:1f.2 SATA controller [0106]: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller [8086:1c02] (rev 05)
+	Subsystem: Hewlett-Packard Company 6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller [103c:330d]
+	Kernel driver in use: ahci
+	Kernel modules: ahci
+```
+
+
+## Problem
+Usually the solutions are simple:
+
+1. If you have only one device listing some module in `Kernel modules` (e.g. `nvidiafb`) you can add it to `/etc/modprobe.d/some-file.conf` as `blacklist nvidiafb`
+2. If you have multiple and they're normal devices you just add `options vfio-pci ids=8086:1c02` to some file in`/etc/modprobe.d/` (make sure to use the id in `[...]` and **not** pci location `00:1f.2`)
+
+However, these will not work if your device is handled by something loaded very very VERY early... like a driver for your second SATA controller.
+1. You cannot blacklist `ahci` (like in example here) because you will prevent all controllers from working (=no boot volume)
+2. You cannot use `modprobe.d` to set options because `vfio-pci` loads waaaaay too late.
+
+
+## Solution
+There are two prerequisites:
+1. `vfio-pci` must load be availbale before rootfs is attached
+2. `vfio-pci` must load before `ahci` loads
+
+**The first is simple:**
+ - add `vfio-pci` to `/etc/initramfs-tools/modules`
+ - update initramfs: `update-initramfs -u -k $(uname -r)`
+ - it will place the module in `initramfs` disk (in `/etc/conf/modules`)
+
+**The second is more complicated:**
+ - entry in `/etc/initramfs-tools/modules` will load `vfio-pci` before the rootfs is mounted
+ - however, `/etc/conf/modules` from ramdisk is loaded after some scripts (see `/init` in ramdisk)
+ - these scripts (`scripts/init-top/`) load some drivers... and `udev`... and `udev` loads `ahci`
+ - solution:
+     - create `/usr/share/initramfs-tools/scripts/init-top/load_vfio-pci` with
+       ```shell script
+       #!/bin/sh
+       modprobe vfio-pci ids=8086:1c02
+       ```
+     - `chmod +x /usr/share/initramfs-tools/scripts/init-top/load_vfio-pci`
+     - edit `/usr/share/initramfs-tools/scripts/init-top/udev` and change `PREREQS=""` to `PREREQS="load_vfio-pci"`
+ - update initramfs: `update-initramfs -u -k $(uname -r)`
+ - note: this **will not** work if placed in "standard place" (`/etc/initramfs-tools/scripts...`) as dependencies are not cross-directory and [`/usr/share` comes first](http://manpages.ubuntu.com/manpages/xenial/man8/initramfs-tools.8.html#boot%20scripts)
+
+## Verify
+Without the mod:
+```
+# lspci -knn
+...
+00:1f.2 SATA controller [0106]: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller [8086:1c02] (rev 05)
+	Subsystem: Hewlett-Packard Company 6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller [103c:330d]
+	Kernel driver in use: ahci
+	Kernel modules: ahci
+```
+
+With the mod:
+```
+# lspci -knn
+...
+00:1f.2 SATA controller [0106]: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller [8086:1c02] (rev 05)
+	Subsystem: Hewlett-Packard Company 6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller [103c:330d]
+	Kernel driver in use: vfio-pci
+	Kernel modules: ahci
+```