Running Ubuntu 16.04.1 armhf on Qemu 10.x under macOS

This is a writeup about how to install Ubuntu 16.04.1 Xenial Xerus for the 32-bit hard-float ARMv7 (armhf) architecture on a Qemu VM via Ubuntu netboot.

The setup will create a Ubuntu VM with LPAE extensions (generic-lpae) enabled. However, this writeup should also work for non-LPAE (generic) kernels.

The performance of the resulting VM is quite good, and it allows VMs with >1G ram (compared to 256M on versatilepb and 1G on versatile-a9/versatile-a15). It also supports virtio disks whereas versatile-a9/versatile-a15 only support SD cards via the -sd argument.

Get netboot files

The netboot files are available on the official Ubuntu mirror. The following commands will download the kernel (vmlinuz) and initrd (initrd.gz) in a new directory called netboot:

mkdir netboot
cd netboot
wget -r -nH -nd -np -R "index.html*" --quiet http://ports.ubuntu.com/ubuntu-ports/dists/xenial/main/installer-armhf/current/images/generic-lpae/netboot/

Create a image file

The following command creates a image file that will be used as a disk for the Ubuntu system:

qemu-img create -f qcow2 ubuntu.qcow2 16G

Start the netboot installation

From the netboot directory (containing the netboot files vmlinuz and initrd.gz), run the following command:

qemu-system-arm \
  -kernel vmlinuz \
  -initrd initrd.gz \
  -append "root=/dev/ram" \
  -no-reboot \
  -nographic \
  -m 1024 \
  -M virt \
  -device virtio-blk-device,drive=hd0 \
  -drive if=none,file=ubuntu.qcow2,id=hd0 \
  -device virtio-net-device,netdev=usernet \
	-netdev user,id=usernet,hostfwd=tcp::5555-:22

The kernel should now boot into the Ubuntu installer within the terminal window where the command has been executed.

Extract the new kernel

After the installation process has been finished, extract the kernel and initrd files from the new installed Ubuntu system. Mount the Ubuntu image:

qemu-img convert -f qcow2 -O raw ubuntu.qcow2 ubuntu-raw.img
sudo losetup /dev/loop0 ubuntu-raw.img
OFFSET=$(($(sudo fdisk -l /dev/loop0 |grep /dev/loop0p1 |awk '{print $3}')*512))
sudo mount -o loop,offset=$OFFSET /dev/loop0 /mnt

Create a new directory boot for the new kernel and initrd files and copy them into it:

mkdir boot
cp /mnt/initrd.img-4.4.0-38-generic-lpae
cp /mnt/vmlinuz-4.4.0-38-generic-lpae boot

Cleanup:

sudo umount /mnt
sudo losetup -d /dev/loop0
rm ubuntu-raw.img

TODO: Add working solution that does not require to convert the image file.

Start the Ubuntu VM

The Ubuntu VM directory should now have the following structure:

├── boot
│   ├── initrd.img-4.4.0-38-generic-lpae
│   └── vmlinuz-4.4.0-38-generic-lpae
├── netboot
│   ├── initrd.gz
│   └── vmlinuz
└── ubuntu.img

Start Qemu:

qemu-system-arm \
  -kernel boot/vmlinuz-4.4.0-38-generic-lpae \
  -initrd boot/initrd.img-4.4.0-38-generic-lpae \
  -append "root=/dev/vda2 rootfstype=ext4" \
  -no-reboot \
  -nographic \
  -m 1024 \
  -M virt \
  -serial stdio \
  -monitor telnet:127.0.0.1:9000,server,nowait \
  -net nic \
  -net tap,ifname=tap0,script=no,downscript=no \
  -drive file=ubuntu.img,if=virtio

The following script can be used for starting the VM:

#!/bin/sh
NET_IF=tap0
BRIDGE=br0
if ! grep --quiet $NET_IF /proc/net/dev;then
    echo "Creating ${NET_IF} device"
    sudo ip tuntap add dev $NET_IF mode tap
    sudo ip l s up dev $NET_IF
fi
if grep --quiet $BRIDGE /proc/net/dev;then
    sudo ip l s $NET_IF master $BRIDGE
else
    echo "${BRIDGE} does not exist!"
    exit
fi
qemu-system-arm \
  -kernel boot/vmlinuz-4.4.0-38-generic-lpae \
  -initrd boot/initrd.img-4.4.0-38-generic-lpae \
  -append "root=/dev/vda2 rootfstype=ext4" \
  -no-reboot \
  -nographic \
  -m 1024 \
  -M virt \
  -serial stdio \
  -monitor telnet:127.0.0.1:9000,server,nowait \
  -net nic \
  -net tap,ifname=tap0,script=no,downscript=no \
  -drive file=ubuntu.img,if=virtio