# How To Set Up a Raspberry Pi 4 with Archlinux 64-bit (AArch64) and Full Disk Encryption (+SSH unlock), USB Boot (No SD-Card) and btrfs Written by: *XSystem* First published on: *20 Dec 2020* Last updated on: *20 Dec 2020* ## [0] Introduction ### Overview In this guide you will learn how to set up a Raspberry Pi 4 Model B with the following features: - **64 bit Archlinux ARM (AArch64)** - **Full USB Boot** - This allows you to ditch the SD card entirely and boot from a thumb drive or SSD connected through a SATA-to-USB adapter. - **Full Disk Encryption + SSH Unlock** - This applies to the root filesystem, not the boot partition. - You will be able to unlock the system **locally** using monitor and keyboard or **remotely** by connecting to the Pi through an SSH connection we will set up. - **btrfs as root filesystem** Of course, you can omit certain steps in this guide if you're only interested in parts of the setup. Maybe you're content with ext4 as the root filesystem, or would like to set up Full Disk Encryption on the SD card and not a USB drive. All steps in this guide are explained, so you should be able to discern what is necessary for your specific use case. #### Caveats As the guide is written right now, it is assumed you are using the **ethernet port** for networking. With the way the initramfs will be set up, the **onboard WiFi interface is not detected** anymore. I believe this can be fixed, but I haven't been able to do that yet. ### Motivation and Special Thanks I decided to write this guide because I was able to find resources online documenting parts of the process, but getting everything to work together took some effort, and I wanted to share my results. Lots of information presented in this guide is based on already existing guides / resources. Specifically, I'd like to thank and reference: - *gea0*'s guide on setting up Full Disk Encryption + SSH Unlock on a Raspberry Pi 3: - https://gist.github.com/gea0/4fc2be0cb7a74d0e7cc4322aed710d38 - *lightirius*' post over on the ArchlinuxARM forums on how to set up full USB boot by manually updating u-boot: - https://archlinuxarm.org/forum/viewtopic.php?t=14672#p64313 ### What You Will Need - An **SD Card** in order to get Archlinux running on the Pi. We will use this installation to set everything up and copy it over to the USB drive once we're ready. - Your **USB boot device**. This can be a thumb drive, SSD or HDD, connected via USB or a USB-to-SATA bridge. - The Raspberry Pi can be finnicky when it comes to the USB-to-SATA adapters it supports. Do some research to find out which ones are worth picking up. - A way to flash OS images to an SD card. - Some third host from which you would like to remotely unlock the Pi. We will set up an SSH key + config there. - A **Raspberry Pi 4 Model B**, of course. ## [1] Set up the Raspberry Pi's EEPROM to support USB booting. You need to update the Pi's EEPROM in order for it to support full USB booting. Since the EEPROM is a piece of memory directly integrated on the Pi's SoC this change will persist even if you swap out all storage media attached to the Pi. If you have already done this for your Pi, you can skip this section altogether. There are already a number of guides out on the web on how to accomplish this, so I will keep this section brief. Install the latest version of *Raspberry Pi OS* (Yes, *Raspberry Pi OS*, not Archlinux) onto your SD card. - Refer to the official guides for more information on this: https://www.raspberrypi.org/software/ - Note that Raspberry Pi OS Lite is sufficient, a desktop environment is not required. - Remember to place an empty file called 'ssh' into the boot partition if you'd like to use a headless setup. More info here: https://www.raspberrypi.org/documentation/remote-access/ssh/README.md Once you're up and running, update your system: ```bash sudo apt update sudo apt full-upgrade ``` Edit the configuration file governing what firmware updates you receive: ```bash sudo nano /etc/default/rpi-eeprom-update ``` In this file the `FIRMWARE_RELEASE_STATUS` variable should currently be set to `critical`. Change it to `stable` in order to receive the latest updates. The file should then look like this: ``` FIRMWARE_RELEASE_STATUS="stable" ``` Update the firmware of the Pi: ```bash sudo rpi-eeprom-update -d -a ``` If required, perform a system reboot now. At the time of writing (20 Dec 2020), the updated VL805 EEPROM version reads '000138a1'. Now that the EEPROM has been updated, you can shut down the system. The installation of Raspberry Pi OS is no longer needed and will be replaced with Arch in the next step. ## [2] Install 64 bit Archlinux onto the SD card. We will set up a basic Arch installation on the SD card without any of the special features (no btrfs, no encryption) which will be cloned to the USB drive once everything has been prepared. Therefore, use `ext4` for the root filesystem at this point, we will set up `btrfs` later. Simply follow the guide on the Archlinux ARM website on how to do this: https://archlinuxarm.org/platforms/armv8/broadcom/raspberry-pi-4 Don't forget to perform the steps listed in the **AArch64** section, i.e. don't forget to pull the 64 bit image and don't forget to update `/etc/fstab` so that it points to the right block device. Additionally, you may want to choose a larger size for the boot partition. 200 MiB are sufficient for our setup, but a little more headroom can't hurt. Once setup, boot into the system and perform basic installation steps. Please refer to the Archwiki's Installation page for guidance: https://wiki.archlinux.org/index.php/installation_guide Note that all software you install at this point will carry over to the final, encrypted system. I'd recommend at least performing the following steps: - As described on the Archlinux ARM installation page, set up the package signing keys with `pacman-key --init` and `pacman-key --populate archlinuxarm`. - Perform a full system update with `pacman -Syu`. - Set up networking so that the machine retains internet access across reboots without manual setup on each boot. - Configure and generate locales. - Set up (a new) user account and change the passwords, `sudo` is recommended as well. - Install a text editor of your choice. ## [3] Install packages required for the special setup. The following packages are required: | Package | Description | | :-- | :-- | | dosfstools | Required to set up a `vfat` partition on the USB drive. | | btrfs-progs | Required to maintain `btrfs` filesystems. Not necessary if you're sticking with `ext4`. | | rsync | Will be used to clone the prepared system to the USB drive and to transfer files between the Pi and your third system from which you will remotely unlock the Pi. | | unzip | What it says on the tin, really. Used to decompress zip-archives. | | base-devel | Required to build user packages. | | uboot-tools | Required to update the U-Boot boot script. | | mkinitcpio-utils | See below. | | mkinitcpio-netconf | See below. | | mkinitcpio-dropbear | These three packages set up networking and an SSH shell during boot to allow remotely unlocking the root filesystem. | To install all of them at once, run: ```bash sudo pacman -S dosfstools btrfs-progs rsync unzip base-devel \ uboot-tools mkinitcpio-utils mkinitcpio-netconf mkinitcpio-dropbear ``` ## [4] Swap out the stable U-Boot for the release candidate. To boot the generic / mainline 64 bit Linux kernel, Archlinux uses a bootloader called *Das U-Boot*. At the time of writing, the version of the bootloader supplied in the repositories does not support full USB booting. There is, however, a release candidate available that supports full USB booting. We will now download the package files of the u-boot package installed on the Pi, modify them so that they install the release-candidate instead of the stable version, and then swap out the bootloader on the Pi. First, acquire the package files for the package `uboot-raspberrypi`. These can be found here: https://github.com/archlinuxarm/PKGBUILDs/tree/master/alarm/uboot-raspberrypi You can use a tool like DownGit in order to download just the specific folder of the repository: - Visit https://downgit.github.io/ - Paste the link to the specific folder given above. - Download the zip archive. You now need to transfer the zip archive to the Pi. (Assuming you performed the steps above on a third system and not on the Pi itself.) This can for instance be achieved using rsync over ssh: ```bash # Perform this on the third host, assuming the Pi's username and hostname # are still called 'alarm' and the Pi is connected to the same network. rsync uboot-raspberrypi.zip alarm@alarm:/home/alarm/ ``` Once the zip archive is available on the Pi, unzip it and change into the directory: ```bash # Make sure you are not root, as root cannot / should not install user packages. # Change into the home directory, if not already. cd # Unzip the archive. All files are already contained within a folder within the archive. unzip uboot-raspberrypi.zip # Change into the directory. cd uboot-raspberrypi ``` Next, we need to perform some edits on the PKGBUILD in order to use the release candidate instead of the stable version. ```bash vim PKGBUILD ``` Perform the following three replacements: | Variable | Before | After | | :-- | :-- | :-- | | `pkgname` | `uboot-raspberrypi` | `uboot-raspberrypi-rc` | | `pkgver` | `2020.07` | `2020.10rc2` | | First value in `md5sums` | `86e51eeccd15e658ad1df943a0edf622` | `bae5280c7ce49961c3722fa9019535bf` | The PKGBUILD should then look something like this: ```bash # U-Boot: Raspberry Pi # Maintainer: Kevin Mihelich buildarch=12 pkgname=uboot-raspberrypi-rc pkgver=2020.10rc2 pkgrel=2 pkgdesc="U-Boot for Raspberry Pi" arch=('armv7h' 'aarch64') url='http://www.denx.de/wiki/U-Boot/WebHome' license=('GPL') backup=('boot/boot.txt' 'boot/boot.scr' 'boot/config.txt') makedepends=('bc' 'dtc' 'git') conflicts_armv7h=('linux-raspberrypi') _commit=f4b58692fef0b9c16bd4564edb980fff73a758b3 source=("ftp://ftp.denx.de/pub/u-boot/u-boot-${pkgver/rc/-rc}.tar.bz2" "https://github.com/raspberrypi/firmware/raw/${_commit}/boot/bcm2710-rpi-3-b.dtb" "https://github.com/raspberrypi/firmware/raw/${_commit}/boot/bcm2710-rpi-3-b-plus.dtb" "https://github.com/raspberrypi/firmware/raw/${_commit}/boot/bcm2710-rpi-cm3.dtb" "https://github.com/raspberrypi/firmware/raw/${_commit}/boot/bcm2711-rpi-4-b.dtb" '0001-rpi-increase-space-for-kernel.patch' 'boot.txt.v2' 'boot.txt.v3' 'mkscr') md5sums=('bae5280c7ce49961c3722fa9019535bf' '0c56f6b8fde06be1415b3ff85b5b5370' 'e4b819439961514c7441473d4733a1b4' '38cab92f98944f0492c5320cf8b36870' '04f2dd06c65cd7ad2932041cbe220a13' '728c4a0a542db702b8d88ffe1994660c' '69e883f0b8d1686b32bdf79684623f06' 'be8abe44b86d63428d7ac3acc64ee3bf' '021623a04afd29ac3f368977140cfbfd') # ... ``` Now, build the package but do not install it just yet: ```bash # Make sure you are cd'd into the PKGBUILD's directory. makepkg -s ``` This should install the necessary build-dependencies, download the release candidate and build the bootloader. Now we're going to swap the stable bootloader with the release-candidate. Make sure to perform both steps in a single session (i.e. don't reboot the system inbetween), as your system is briefly left with no bootloader. ```bash # Uninstall the stable bootloader. sudo pacman -R uboot-raspberrypi # Afterwards, while still in the uboot-raspberrypi directory, # install our freshly built release-candidate. makepkg -si ``` `makepkg -si` will inform you the package has already been built, and will simply proceed to install it. Once this is done, reboot the system. This is to ensure that swapping out the bootloader worked on your system. ## [5] Configure dropbear to allow early SSH access during boot. These steps are simply adapted from gea0's guide for the Pi 3 which was already linked in the beginning: https://gist.github.com/gea0/4fc2be0cb7a74d0e7cc4322aed710d38 Set up an RSA SSH key on the system from which you would like to remotely unlock your pi: ```bash ssh-keygen -t rsa -b 4096 -a 100 -f ~/.ssh/pi_unlock_key ``` Transfer it to the pi: ```bash rsync ~/.ssh/pi_unlock_key.pub alarm@alarm:/home/alarm/ ``` Additionally, set up the configuration on this host system: ``` vim ~/.ssh/config -------------------------------------------------------------------------------- Host pi-unlock HostName pi-locked User root IdentityFile ~/.ssh/pi_unlock_key ``` Of course, you can choose a different hostname here, we will set it up in a later section when we update the `/boot/boot.txt` file. If you decide to go with a static IP setup, you can also replace `pi-locked` with the static IP. The user has to be `root` because this is the user used to connect during the early unlock stage. This doesn't mean that we enable actual root access over ssh for the booted system. The next steps will now be **performed on the Pi** and not the host used for unlocking. Make the copied key dropbear's root key. ```bash sudo mv ~/pi_unlock_key.pub /etc/dropbear/root_key ``` With this setup, the generated key will **only** be used to unock the Pi during boot, and not for actually connecting with the Pi over ssh once it has booted. Of course, feel free to set this up differently. For instance, you could use one key for both unlocking and connecting with the booted system. Finally, regenerate the RSA host key with the `-m PEM` option. This is due to a bug in `dropbear` which will cause errors in the next section when running the `dropbear`-hook during `mkinitcpio`. ```bash cd /etc/ssh sudo rm ssh_host_rsa_key* sudo rm ssh_host_dsa_key* sudo rm ssh_host_ecdsa_key* # Regenerate it with the '-m PEM' option. sudo ssh-keygen -t rsa -b 4096 -f ssh_host_rsa_key -N "" -m PEM < /dev/null sudo ssh-keygen -t ecdsa -f ssh_host_dsa_key -N "" -m PEM < /dev/null sudo ssh-keygen -t ecdsa -f ssh_host_ecdsa_key -N "" -m PEM < /dev/null ``` Now that dropbear is set up, we can move on to the initramfs. ## [6] Update the initramfs to support btrfs, full USB boot and full disk encryption. We will now perform a number of changes to the system's initramfs in order to ensure support for the three essential features (btrfs, usb boot and disk encryption + SSH unlock) during boot. Edit `mkinitcpio.conf`: ```bash sudo vim /etc/mkinitcpio.conf ``` We now need to add three additional modules: | module | description | | :-- | :-- | | `btrfs` | Necessary to allow booting from a `btrfs` filesystem. Not necessary when using `ext4`. | | `pcie_brcmstb` | Necessary to allow booting from a USB device. | `broadcom` | The netconf-hook would get stuck during boot if I didn't add this module. It's therefore a necessary module for remotely unlocking the machine. | Also add the following binary: | binary | description | | :-- | :-- | | `/usr/lib/libgcc_s.so.1` | Decryption would otherwise fail with the error that `pthread_cancel` is not available when using the `encryptssh` hook. | Finally, we need the following additional hooks: | hooks | insert after | description | | :-- | :-- | :-- | | `keyboard keymap` | `autodetect` | Loads the keyboard early in order to allow entry of the passphrase using monitor and keyboard. | | `sleep netconf dropbear encryptssh` | `block` | The four hooks necessary to set up early networking, the ssh shell and encryption. `sleep` ensures all devices are online before setting up networking. | Your `/etc/mkinitcpio.conf` should then look something like this: ``` # ... MODULES=(btrfs pcie_brcmstb broadcom) # ... BINARIES=(/usr/lib/libgcc_s.so.1) # ... FILES=() # ... HOOKS=(base udev autodetect keyboard keymap modconf block sleep netconf dropbear encryptssh filesystems fsck) ``` Now, rebuild the initramfs: ```bash sudo mkinitcpio -P ``` ## [7] Prepare the USB device. We will now set up the USB device using full-disk encryption. Note that you should now perform some special steps on the USB drive to ensure full security, which often involves overriding either the entire drive or the second partition with random bytes or zero bytes. It may also be advisable to perform a SATA Secure Erase on the drive before partitioning. Instead of providing all the details here, you are strongly advised to consider the corresponding page on the Archwiki before proceeding: https://wiki.archlinux.org/index.php/Dm-crypt/Drive_preparation Now, on to the actual partitioning of the drive. Plug in the USB device to the pi and identify it with `sudo fdisk -l`. It will probably have been assigned the `/dev/sda`-identifier. Once identified, reformat it similarly to the installation guide: ```bash sudo fdisk /dev/sda # Create a new MBR table. o # Create a new primary boot partition. n [Enter] (picks the default 'p') [Enter] (picks the default '1') [Enter] (picks the default '2048') +200M # Set its type correctly. t c # Create the second partition, which we will encrypt momentarily. n [Enter] (picks the default 'p') [Enter] (picks the default '2') [Enter] (picks the default first sector) [Enter] (picks the default last sector) # Print the pending changes and ensure everything looks good. p # Apply and exit. w ``` Next, set up the appropriate filesystems and encryption: ```bash # Set up the correct filesystem for the boot partition. sudo mkfs.vfat /dev/sda1 # Set up encryption on the second partition. sudo cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha512 --use-random -i 1000 /dev/sda2 sudo cryptsetup luksOpen /dev/sda2 root # And format the btrfs filesystem for it. sudo mkfs.btrfs /dev/mapper/root # Alternatively, if you prefer ext4 for the root filesystem: sudo mkfs.ext4 /dev/mapper/root ``` If you're using `btrfs`, now is the time to set up the subvolume structure. How you do this is ultimately up to you, therefore the section below is just an example. We will later define `rootfs` as the root subvolume when adjusting the kernel command line parameters. Another option can be to set the default subvolume using `btrfs subvolume set-default`. ```bash # Mount the btrfs subvolume. sudo mount /dev/mapper/root /mnt # Create a subvolume for the root filesystem. sudo btrfs subvolume create /mnt/rootfs # Unmount for now. sudo umount /mnt sudo cryptsetup close root ``` ## [8] Prepare the boot.txt with the correct kernel command line arguments. For now, create a copy of the boot.txt and then edit this copy. ```bash sudo cp /boot/boot.txt /boot/boot.txt.new sudo vim /boot/boot.txt.new ``` In it, comment out the line that reads `part uuid ...` by placing a \#-symbol in front of it. Next, we will focus our attention on the `setenv`-line that defines the command line arguments of the kernel. Replace the section that reads `root=PARTUUID=${uuid}` with the following: ``` ip=::::pi-locked:eth0:dhcp cryptdevice=/dev/sda2:root root=/dev/mapper/root rootflags=subvol=rootfs ``` The first argument tells the `netconf`-hook how exactly it is supposed to set up networking. This is the place where the hostname we defined earlier during ssh configuration comes into play. Generally, the `ip` argument has the following pattern: ``` ip=:::::::: ``` For more information and further examples on how to set this up, refer to the corresponding section in the Archwiki: https://wiki.archlinux.org/index.php/Mkinitcpio#Using_net In my example I have explicitly set the hostname to `pi-locked` and told netconf to autoconfigure the ethernet interface using DHCP. The next two arguments set up full disk encryption, using `/dev/sda2` as the encrypted blockdevice and mapping it to `/dev/mapper/root`. Finally, `rootflags=subvol=rootfs` ensures that the `btrfs` subvolume we set up earlier is used as the root filesystem. If you're using `ext4` you can omit this parameter. The `boot.txt.new` should then look something like this: ``` # After modifying, run ./mkscr # Set root partition to the second partition of boot device #part uuid ${devtype} ${devnum}:2 uuid setenv bootargs console=ttyS1,115200 console=tty0 ip=::::pi-locked:eth0:dhcp cryptdevice=/dev/sda2:root root=/dev/mapper/root rootflags=subvol=rootfs rw rootwait smsc95xx.macaddr="${usbethaddr}" # ... ``` **Note**: We will run `./mkscr` later, which will apply the changes. If applied now, you wouldn't be able to reboot the system anymore. ## [9] Clone the system. We will now clone our prepared system to the USB drive. First, mount both partitions of the USB drive. One way to achieve this can look like this: ```bash # Create a new folder in your home-directory for this. mkdir ~/pi-setup cd ~/pi-setup # Create folders for the two partitions. mkdir usb-boot mkdir usb-root # Identify the disks. sudo fdisk -l # Assuming your USB drive was detected as /dev/sda # Open the encrypted partition sudo cryptsetup luksOpen /dev/sda2 root # Mount both USB partitions sudo mount /dev/sda1 usb-boot/ # Make sure you mount the BTRFS subvolume for the root filesystem. sudo mount -t btrfs -o subvol=rootfs /dev/mapper/root usb-root/ # Alternatively, when using ext4: sudo mount /dev/mapper/root usb-root/ ``` **Note**: Double check that you mounted the right partitions in the right folders. It is easy to make a mistake here, and you may corrupt the bootstrap-system we've set up so far with the next few commands if you're not careful. Now, actually clone the system. ```bash sudo rsync --info=progress2 -axHAX /boot/ usb-boot/ sudo rsync --info=progress2 -axHAX / usb-root/ # Ensure the cache is empty. sudo sync ``` Now, we need to perform two final adjustments. First, adjust the `fstab` of the cloned system by replacing `/dev/mmcblk1p1` with `/dev/sda1`: ```bash sudo vim usb-root/etc/fstab -------------------------------------------------------------------------------- /dev/sda1 /boot vfat defaults 0 0 ``` Secondly, apply the changes we made to the bootloader. ```bash cd ~/pi-setup/usb-boot sudo mv boot.txt.new boot.txt sudo ./mkscr ``` Finally, unmount the two partitions. ```bash cd ~/pi-setup sudo umount usb-boot usb-root sudo cryptsetup close root ``` And that's it. Shutdown your system and remove the SD card. ## [10] Finish! Now, connect the USB drive and **only** the USB drive to the Pi and power it on. You should now be able to unlock the Pi locally through both monitor and keyboard as well as remotely via ssh by connecting to the Pi from your other host with the host-config that we've set up: ```bash ssh pi-unlock ``` ## Known Issues This setup assumes you're using ethernet for your networking. The initramfs lacks the appropriate kernel modules to set up WiFi, which unfortunately makes it unavailable in the booted system. I'm sure this can be remedied by including the correct module in the initramfs, but I don't know which modules those are. Moreover, the system consistently prints an error message in the audit-log about a failing hardware interrupt on mmc1. The system consistently scans for the SD card which, of course, isn't there. I read that this can be turned off by passing an appropriate kernel parameter, but this appeared to have no effect on the problem. ### Feedback is welcome! If you have ideas on how to fix the known issues described above, noticed some other mistake in the guide or if everything worked out fine for you, feel free to let me know in the comments below. This is the first time I've written a larger guide like this, I hope it will prove helpful.