How To Run Yggdrasil In Docker
==============================

Want to run Yggdrasil in a Docker container?  This is how you can do it.

The keys to getting it working are the following:

- Give the container access to the TUN interface of the host (or the VM guest in the case of Docker Machine or Docker for Mac)
- Enable IPv6
- Assign a dedicated MAC address

It took me a while to figure this stuff out, so hopefully this helps you.  When the requirements aren't met, Yggdrasil exits with a barely helpful "permission denied" message.

Here is an example Dockerfile:

```dockerfile
FROM alpine:3.18.3

RUN \
  apk add \
  --no-cache \
  --no-check-certificate \
  --allow-untrusted \
  --no-scripts \
  yggdrasil=0.4.7-r9

CMD yggdrasil -useconffile /etc/yggdrasil.conf
```

At this point, you will likely want to create a yggdrasil.conf file that configures the host machine as a peer.

Assuming that your host is running Yggdrasil and listening for peers on TCP port 8008, this is how you can generate a new yggdrasil.conf file for your container:

```sh
echo "{ \"Peers\": [ \"tcp://host.docker.internal:8008\" ] }" | yggdrasil -useconf -normaliseconf > yggdrasil.conf
```

The specific port number doesn't matter so long as it's the one that the peer running on your host is listening for other peers on.  The `host.docker.internal` domain is automatically set up by Docker.

Here is an example of how to run a container:

```sh
docker build -t yggdrasil .
docker run \
--rm \
--cap-add "NET_ADMIN" \
--device "/dev/net/tun" \
--volume "./yggdrasil.conf:/etc/yggdrasil.conf" \
--sysctl "net.ipv6.conf.all.disable_ipv6=0" \
--mac-address "1E:FC:B2:8D:DF:D4" \
yggdrasil 
```

Here is an example with Docker Compose:

```yaml
version: '3.9'
services:
  yggdrasil:
    build: .
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun
    volumes:
      - "./yggdrasil.conf:/etc/yggdrasil.conf"
    sysctls:
      - "net.ipv6.conf.all.disable_ipv6=0"
    mac_address: 1E:FC:B2:8D:DF:D4
```

In which case, you would run `docker-compose up`.  To spin it down, run `docker-compose down`.

Check the log of the running container to see whether Yggdrasil is successfully reaching the host peer.  If you are unsure, you can check on your host machine by running `yggdrasilctl getPeers`.

Ok, this is all fine, but now you want to make another containerized service available from your Yggdrasil peer container, right?

The following is an approach using Docker Compose, with a Redis service as an example.

We will need to change the Dockerfile so that it will support forwarding the Redis port.


```dockerfile
FROM alpine:3.18.3

RUN \
  apk add \
  --no-cache \
  --no-check-certificate \
  --allow-untrusted \
  --no-scripts \
  yggdrasil=0.4.7-r9 \
  socat=1.7.4.4-r1 \
  supervisor=4.2.5-r2

CMD supervisord -c /etc/supervisord.conf
```

We will also need a supervisord.conf file:

```ini
[supervisord]
logfile=/dev/null
nodaemon=true
user=root

[program:yggdrasil]
command=yggdrasil -useconffile /etc/yggdrasil/yggdrasil.conf
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

[program:redis-forwarder]
command=socat TCP6-LISTEN:6379,fork,forever,reuseaddr TCP4:redis:6379
```

And now let's add a volume for the supervisord configuration:

```yaml
version: '3.9'
services:
  yggdrasil:
    build: .
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun
    volumes:
      - "./yggdrasil.conf:/etc/yggdrasil.conf"
      - "./supervisord.conf:/etc/supervisord.conf"
    sysctls:
      - "net.ipv6.conf.all.disable_ipv6=0"
    mac_address: 1E:FC:B2:8D:DF:D4
 
  redis:
    image: docker.io/redis:7.2.1-alpine3.18
    ports:
      - "6379:6379"
```

Run `docker-compose up` and use `nc -v <yggdrasil peer ipv6 address> 6379` or `redis-cli -h <yggdrasil peer ipv6 address>` to confirm that your containerized peer has joined your network and that you can reach the Redis service through it.

## Alternative Setup (if all else fails)

Can't get IPv6 to work in Docker?  `/dev/net/tun` doesn't exist?

There's a less optimal alternative that can work, and that's to run a virtual machine within a Docker container.  The reason this works is that it emulates a full network stack, bypassing any limitations on the host.

Here is an example alternative Dockerfile that runs Yggdrasil within a VM:

```dockerfile
FROM alpine:3.18.3 AS initramfs

WORKDIR /

RUN \
  apk add --no-cache \
    coreutils \
    curl

RUN mkdir -p /root

ENV initramfsDir=/initramfs

## Make default directory structure
RUN \
  mkdir -p \
  "$initramfsDir" \
  "$initramfsDir/bin" \
  "$initramfsDir/etc" \
  "$initramfsDir/mnt" \
  "$initramfsDir/proc" \
  "$initramfsDir/root" \
  "$initramfsDir/sbin" \
  "$initramfsDir/sys" \
  "$initramfsDir/var/run"

# Add base system
RUN \
  apk add \
  --root "$initramfsDir" \
  --no-cache \
  --initdb \
  --no-check-certificate \
  --allow-untrusted \
  --repository "https://dl-cdn.alpinelinux.org/alpine/v3.18/main" \
  busybox=1.36.1-r2 \
  supervisor \
  socat

# Add yggdrasil
RUN \
  apk add \
  --root "$initramfsDir" \
  --initdb \
  --no-cache \
  --no-check-certificate \
  --allow-untrusted \
  --no-scripts \
  --repository "https://dl-cdn.alpinelinux.org/alpine/v3.18/main" \
  --repository "https://dl-cdn.alpinelinux.org/alpine/v3.18/community" \
  yggdrasil=0.4.7-r9

ENV linuxLtsDir=/linux-virt

RUN mkdir -p $linuxLtsDir

RUN \
  curl -o linux-virt.tar -sSL https://dl-cdn.alpinelinux.org/alpine/v3.18/main/aarch64/linux-virt-6.1.54-r0.apk && \
  tar xvf linux-virt.tar -C $linuxLtsDir && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.alias $initramfsDir/lib/modules/6.1.54-0-virt/modules.alias && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.alias.bin $initramfsDir/lib/modules/6.1.54-0-virt/modules.alias.bin && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.builtin $initramfsDir/lib/modules/6.1.54-0-virt/modules.builtin && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.builtin.alias.bin $initramfsDir/lib/modules/6.1.54-0-virt/modules.builtin.alias.bin && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.builtin.bin $initramfsDir/lib/modules/6.1.54-0-virt/modules.builtin.bin && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.builtin.modinfo $initramfsDir/lib/modules/6.1.54-0-virt/modules.builtin.modinfo && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.dep $initramfsDir/lib/modules/6.1.54-0-virt/modules.dep && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.dep.bin $initramfsDir/lib/modules/6.1.54-0-virt/modules.dep.bin && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.devname $initramfsDir/lib/modules/6.1.54-0-virt/modules.devname && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.order $initramfsDir/lib/modules/6.1.54-0-virt/modules.order && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.softdep $initramfsDir/lib/modules/6.1.54-0-virt/modules.softdep && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.symbols $initramfsDir/lib/modules/6.1.54-0-virt/modules.symbols && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/modules.symbols.bin $initramfsDir/lib/modules/6.1.54-0-virt/modules.symbols.bin && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/kernel/drivers/net/ethernet/intel/e1000/e1000.ko.gz $initramfsDir/lib/modules/6.1.54-0-virt/kernel/drivers/net/ethernet/intel/e1000/e1000.ko.gz && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/kernel/net/9p/9pnet.ko.gz $initramfsDir/lib/modules/6.1.54-0-virt/kernel/net/9p/9pnet.ko.gz && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/kernel/net/packet/af_packet.ko.gz $initramfsDir/lib/modules/6.1.54-0-virt/kernel/net/packet/af_packet.ko.gz && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/kernel/drivers/net/tun.ko.gz $initramfsDir/lib/modules/6.1.54-0-virt/kernel/drivers/net/tun.ko.gz && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/kernel/fs/9p/9p.ko.gz $initramfsDir/lib/modules/6.1.54-0-virt/kernel/fs/9p/9p.ko.gz && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/kernel/net/9p/9pnet_virtio.ko.gz $initramfsDir/lib/modules/6.1.54-0-virt/kernel/net/9p/9pnet_virtio.ko.gz && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/kernel/fs/fscache/fscache.ko.gz $initramfsDir/lib/modules/6.1.54-0-virt/kernel/fs/fscache/fscache.ko.gz && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/kernel/fs/netfs/netfs.ko.gz $initramfsDir/lib/modules/6.1.54-0-virt/kernel/fs/netfs/netfs.ko.gz && \
  install -Dm644 $linuxLtsDir/lib/modules/6.1.54-0-virt/kernel/net/ipv6/ipv6.ko.gz $initramfsDir/lib/modules/6.1.54-0-virt/kernel/net/ipv6/ipv6.ko.gz


## Add host nameserver
RUN cat > $initramfsDir/etc/resolv.conf <<EOF
nameserver 10.0.2.3
EOF


## Add init file for initramfs
RUN cat > $initramfsDir/init <<EOF
#!/bin/sh

mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs dev /dev

echo /sbin/mdev > /proc/sys/kernel/hotplug

/sbin/mdev -s

# Shared Directory
modprobe virtio_pci
modprobe 9pnet
modprobe 9pnet_virtio
modprobe 9p

mkdir -p /etc/yggdrasil

mount -t 9p -o trans=virtio host0 /etc/yggdrasil

# Networking
modprobe e1000
# modprobe virtio_net
modprobe tun
modprobe ipv6

echo "nameserver 10.0.2.3" > /etc/resolv.conf

ip link set lo up
ip link set eth0 up
udhcpc -i eth0
ifconfig eth0 10.0.2.15
route add default gw 10.0.2.2 eth0

mkdir -p /var/run

socat TCP6-LISTEN:6379,fork,forever,reuseaddr TCP4:redis:6379 &
/usr/bin/yggdrasil -useconffile /etc/yggdrasil/yggdrasil.conf

# exec /bin/busybox sh
EOF

RUN chmod +x "$initramfsDir/init"

## Bundle initramfs file and copy vmlinuz
RUN \
  cd $initramfsDir && \
  find . | sort | cpio --quiet --renumber-inodes -o -H newc | gzip -9 > /root/initramfs && \
  cp "$linuxLtsDir/boot/vmlinuz-virt" "/root/vmlinuz"




FROM alpine:3.18.3 AS primary

COPY --from=initramfs /root /root

WORKDIR /root

RUN apk add --no-cache \
  qemu-system-aarch64

CMD qemu-system-aarch64 \
  -m "128M" \
  -machine "virt" \
  -cpu "cortex-a76" \
  -smp "1" \
  -kernel "vmlinuz" \
  -initrd "initramfs" \
  -serial "mon:stdio" \
  -append "ip=dhcp" \
  -device "e1000,mac=1E:FC:B2:8D:DF:D4,netdev=net0" \
  -netdev "user,id=net0" \
  -virtfs "local,path=/etc/yggdrasil,mount_tag=host0,security_model=passthrough,id=host0" \
  -nographic
```

This is just an example.  It's not as efficient because it emulates a CPU, but I believe it's workable if there's something about the host's Docker or network setup that makes it impossible to run Yggdrasil the normal way.