Skip to content

Instantly share code, notes, and snippets.

@toricls
Last active November 2, 2025 18:35
Show Gist options
  • Select an option

  • Save toricls/d3dd0bec7d4c6ddbcf2d25f211e8cd7b to your computer and use it in GitHub Desktop.

Select an option

Save toricls/d3dd0bec7d4c6ddbcf2d25f211e8cd7b to your computer and use it in GitHub Desktop.
Using Lima to run containers with containerd and nerdctl (without Docker Desktop) on M1 Macs

Used M1 Mac mini 2020 with macOS Big Sur Version 11.5.2.

1. Install Patched QEMU

According to the GitHub issue comment in Lima's GitHub repository, Lima requires a patched QEMU on M1 Macs.

Note that the followings are customized steps by @toricls based on the original steps and the script. Be sure to check the original files when you try it on your own.

Install QEMU on Silicon based Apple Macs

# Install necessary packages for building
brew install libffi gettext glib pkg-config autoconf automake pixman ninja

# Clone qemu using ghq instead of git
ghq get https://github.com/qemu/qemu

# Change directory to qemu repository
# Note: In my case, git repositories are stored under $HOME/go/src directory
cd $HOME/go/src/$(ghq list | grep qemu)

# Checkout to commit dated June 03, 2021 v6.0.0
git checkout 3c93dfa42c394fdd55684f2fbf24cf2f39b97d47

# Apply patch series v8 by Alexander Graf
curl https://patchwork.kernel.org/series/485309/mbox/ | git am

# Building qemu installer
mkdir build && cd build
../configure --target-list=aarch64-softmmu
make -j8

# Install qemu
sudo make install

Create Ubuntu Server using QEMU on Silicon based Apple Macs

"First run, close manually after installation aka first reboot." steps

# Change directory to qemu repository
# Note: Git repositories are stored under $HOME/go/src directory in my case
cd $HOME/go/src/$(ghq list | grep qemu)

mkdir /tmp/test-patched-qemu && cd /tmp/test-patched-qemu

curl https://cdimage.ubuntu.com/releases/20.04/release/ubuntu-20.04.2-live-server-arm64.iso -o ubuntu-lts.iso

qemu-img create -f qcow2 virtual-disk.qcow2 8G

cp $(dirname $(which qemu-img))/../share/qemu/edk2-aarch64-code.fd .

dd if=/dev/zero conv=sync bs=1m count=64 of=ovmf_vars.fd

# you'll see four files in the /tmp/test-patched-qemu directory, 1) edk2-aarch64-code.fd, 2) ovmf_vars.fd, 3) ubuntu-lts.iso, and 4) virtual-disk.qcow2
ls

qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=off \
  -cpu cortex-a72 -smp 4 -m 4G \
  -device virtio-gpu-pci \
  -device virtio-keyboard-pci \
  -drive "format=raw,file=edk2-aarch64-code.fd,if=pflash,readonly=on" \
  -drive "format=raw,file=ovmf_vars.fd,if=pflash" \
  -drive "format=qcow2,file=virtual-disk.qcow2" \
  -cdrom ubuntu-lts.iso

# I just tested just starting up it without any issues and skipped the "Second run, enjoy your Ubuntu Server." step, but you can of course finish instalation steps youself for your newly created VM to interact with it

2. Play with Lima

Installation

# According to the [installation doc](https://github.com/lima-vm/lima#installation), Lima supports Homebrew only for Intel Macs.

# Installing Lima manually
mkdir /tmp/lima && cd /tmp/lima

curl -L https://github.com/lima-vm/lima/releases/download/v0.6.1/lima-0.6.1-Darwin-arm64.tar.gz -o ./lima-0.6.1-Darwin-arm64.tar.gz

# List content in the tar.gz file
tar tzf /tmp/lima/lima-0.6.1-Darwin-arm64.tar.gz

# Extract them
tar -xzvf /tmp/lima/lima-0.6.1-Darwin-arm64.tar.gz -C /tmp/lima

# Rename the command "nerdctl.lima" to "nerdctl"
mv /tmp/lima/bin/nerdctl.lima /tmp/lima/bin/nerdctl

# Move the commands
sudo mv ./bin/* /usr/local/bin/
sudo mv ./share/lima /usr/local/share

Start VM

Save the following as default.yaml. It's altered from the original file for 1) setting loadDotSSHPubKeys to false, 2) removing the "mounts" section, and 3) changing num of cpu and disk size.

# ===================================================================== #
# BASIC CONFIGURATION
# ===================================================================== #

# Arch: "default", "x86_64", "aarch64".
# "default" corresponds to the host architecture.
arch: "default"

# An image must support systemd and cloud-init.
# Ubuntu and Fedora are known to work.
# Default: none (must be specified)
images:
  # Try to use a local image first.
  - location: "~/Downloads/hirsute-server-cloudimg-amd64.img"
    arch: "x86_64"
  - location: "~/Downloads/hirsute-server-cloudimg-arm64.img"
    arch: "aarch64"

  # Download the file from the internet when the local file is missing.
  # Hint: run `limactl prune` to invalidate the "current" cache
  - location: "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-amd64.img"
    arch: "x86_64"
  - location: "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-arm64.img"
    arch: "aarch64"

# CPUs: if you see performance issues, try limiting cpus to 1.
# Default: 4
cpus: 2

# Memory size
# Default: "4GiB"
memory: "4GiB"

# Disk size
# Default: "100GiB"
disk: "30GiB"

ssh:
  # A localhost port of the host. Forwarded to port 22 of the guest.
  # Currently, this port number has to be specified manually.
  # Default: none
  localPort: 60022
  # Load ~/.ssh/*.pub in addition to $LIMA_HOME/_config/user.pub .
  # This option is useful when you want to use other SSH-based
  # applications such as rsync with the Lima instance.
  # If you have an insecure key under ~/.ssh, do not use this option.
  # Default: true
  loadDotSSHPubKeys: false

# ===================================================================== #
# ADVANCED CONFIGURATION
# ===================================================================== #

containerd:
  # Enable system-wide (aka rootful)  containerd and its dependencies (BuildKit, Stargz Snapshotter)
  # Default: false
  system: false
  # Enable user-scoped (aka rootless) containerd and its dependencies
  # Default: true
  user: true

# ===================================================================== #
# FURTHER ADVANCED CONFIGURATION
# ===================================================================== #

firmware:
  # Use legacy BIOS instead of UEFI.
  # Default: false
  legacyBIOS: false

video:
  # QEMU display, e.g., "none", "cocoa", "sdl".
  # As of QEMU v5.2, enabling this is known to have negative impact
  # on performance on macOS hosts: https://gitlab.com/qemu-project/qemu/-/issues/334
  # Default: "none"
  display: "none"

network:
  # The instance can get routable IP addresses from the vmnet framework using
  # https://github.com/lima-vm/vde_vmnet. Both vde_switch and vde_vmnet
  # daemons must be running before the instance is started. The interface type
  # (host, shared, or bridged) is configured in vde_vmnet and not lima.
  vde:
    # vnl (virtual network locator) points to the vde_switch socket directory,
    # optionally with vde:// prefix
    # - vnl: "vde:///var/run/vde.ctl"
    #   # VDE Switch port number (not TCP/UDP port number). Set to 65535 for PTP mode.
    #   # Default: 0
    #   switchPort: 0
    #   # MAC address of the instance; lima will pick one based on the instance name,
    #   # so DHCP assigned ip addresses should remain constant over instance restarts.
    #   macAddress: ""
    #   # Interface name, defaults to "vde0", "vde1", etc.
    #   name: ""

# Port forwarding rules. Forwarding between ports 22 and ssh.localPort cannot be overridden.
# Rules are checked sequentially until the first one matches.
# portForwards:
#   - guestPort: 443
#     hostIP: "0.0.0.0" # overrides the default value "127.0.0.1"; allows privileged port forwarding
#   # default: hostPort: 443 (same as guestPort)
#   # default: guestIP: "127.0.0.1" (also matches bind addresses "0.0.0.0", "::", and "::1")
#   # default: proto: "tcp" (only valid value right now)
#   - guestPortRange: [4000, 4999]
#     hostIP:  "0.0.0.0" # overrides the default value "127.0.0.1"
#   # default: hostPortRange: [4000, 4999] (must specify same number of ports as guestPortRange)
#   - guestPort: 80
#     hostPort: 8080 # overrides the default value 80
#   - guestIP: "127.0.0.2" # overrides the default value "127.0.0.1"
#     hostIP: "127.0.0.2" # overrides the default value "127.0.0.1"
#   # default: guestPortRange: [1024, 65535]
#   # default: hostPortRange: [1024, 65535]
#   - guestPort: 8888
#     ignore: true (don't forward this port)
#   # Lima internally appends this fallback rule at the end:
#   - guestIP: "127.0.0.1"
#     guestPortRange: [1024, 65535]
#     hostIP: "127.0.0.1"
#     hostPortRange: [1024, 65535]
#   # Any port still not matched by a rule will not be forwarded (ignored)

# ===================================================================== #
# END OF TEMPLATE
# ===================================================================== #
$ limactl start default.yaml
? Creating an instance "default" Proceed with the default configuration
INFO[0001] Downloading "https://github.com/containerd/nerdctl/releases/download/v0.11.1/nerdctl-full-0.11.1-linux-arm64.tar.gz" (sha256:e2c8d0417b2fb79919f22a818813c646ad7ce0e600a951b6bac98340650e4435)
INFO[0001] Using cache "/Users/yshr/Library/Caches/lima/download/by-url-sha256/cf356d64aa826ad177396feef432a14e0df5b2bb998191eff281541d329ff12c/data"
INFO[0001] Attempting to download the image from "~/Downloads/hirsute-server-cloudimg-arm64.img"
INFO[0001] Attempting to download the image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-arm64.img"
INFO[0002] Using cache "/Users/yshr/Library/Caches/lima/download/by-url-sha256/c40afd2acd5f2078759e01e119168213674615fff0701d53bc2bf69e5be3bf69/data"
INFO[0002] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/xxxx/.lima/default/serial.log")
INFO[0002] SSH Local Port: 60022
INFO[0002] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0012] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0019] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0019] [hostagent] Waiting for the essential requirement 2 of 2: "the guest agent to be running"
INFO[0022] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0022] [hostagent] Waiting for the optional requirement 1 of 2: "systemd must be available"
INFO[0022] [hostagent] Forwarding "/run/user/505/lima-guestagent.sock" (guest) to "/Users/xxxx/.lima/default/ga.sock" (host)
INFO[0022] [hostagent] The optional requirement 1 of 2 is satisfied
INFO[0022] [hostagent] Waiting for the optional requirement 2 of 2: "containerd binaries to be installed"
INFO[0022] [hostagent] Not forwarding TCP 127.0.0.53:53
INFO[0022] [hostagent] Not forwarding TCP 0.0.0.0:22
INFO[0022] [hostagent] Not forwarding TCP [::]:22
INFO[0037] [hostagent] The optional requirement 2 of 2 is satisfied
INFO[0037] READY. Run `lima` to open the shell.

Run nginx container inside Lima VM

$ lima nerdctl run -d --name nginx -p 127.0.0.1:8080:80 nginx:alpine

docker.io/library/nginx:alpine:                                                   resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:859ec6f2dc548cd2e5144b7856f2b5c37b23bd061c0c93cfa41fb5fb78307ead:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:4375f141ad62ca2763d3698e81ab6c61fa2cb8dba169212bf92c845158dbafb1: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:78754800625ea92fe7b32be0754194394e84a2ec0018b037f4279822bfcf5712:   done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:0a8cd3c047c9c865bf65fa310e119ccbd3264a4050c81d0d6b5c3fd153dcce30:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:552d1f2373af9bfe12033568ebbfb0ccbb0de11279f9a415a29207e264d7f4d9:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:50f60b67a614a5a3025f6d389391af19154139b35d314b2aa89dcd2791c4050b:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:e14bcbebf464b5696ca521435ef24348933f399f49bc4c3f4bce4a6c9b6fce5b:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:d8e1f9d1b87e92444b4e8e5dab10322aa2d66c1a226e1419048328a36fc0f716:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:4d5947aec77d85cbbdc91c6bf3435095c91d516cf90af8d1394d1538d0103dcd:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 5.1 s                                                                    total:  9.3 Mi (1.8 MiB/s)
ab3ff970790da2ca30a7007b80e1fb65bbe86a4e39809096f764ec8c4cdbdba7

$ curl -I http://localhost:8080
HTTP/1.1 200 OK
Server: nginx/1.21.1
Date: Wed, 01 Sep 2021 04:23:20 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 06 Jul 2021 15:21:34 GMT
Connection: keep-alive
ETag: "60e474fe-264"
Accept-Ranges: bytes

Use alias docker for lima nerdctl

$ lima nerdctl ps
CONTAINER ID    IMAGE                             COMMAND                   CREATED           STATUS    PORTS                     NAMES
ab3ff970790d    docker.io/library/nginx:alpine    "/docker-entrypoint.…"    39 seconds ago    Up        127.0.0.1:8080->80/tcp    nginx

$ alias docker="lima nerdctl"

$ docker ps
CONTAINER ID    IMAGE                             COMMAND                   CREATED               STATUS    PORTS                     NAMES
ab3ff970790d    docker.io/library/nginx:alpine    "/docker-entrypoint.…"    About a minute ago    Up        127.0.0.1:8080->80/tcp    nginx

Woot! Woot!

@selvakn
Copy link

selvakn commented Sep 8, 2021

Patched Qemu for m1: brew install simnalamburt/x/qemu-hvf

@Rinfore
Copy link

Rinfore commented Sep 12, 2021

Hi Toricls, could I ask why you turned off mounting, and set loadDotSSHPubKeys to false? Did you face some significant bugs and/or data loss without these changes?

@toricls
Copy link
Author

toricls commented Sep 13, 2021

Thanks @selvakn for the info, will update this gist to use that one.

@toricls
Copy link
Author

toricls commented Sep 13, 2021

@Rinfore Hi Rinfore, at the time of writing this gist I removed the mount options for the simplicity of my test and actually there were some issues noted nerdctl#338 and lima#203, but it's fixed today (yay!). For loadDotSSHPubKeys: false, I had no plan to use other SSH-based applications such as rsync with the Lima instance as described in the default configuration file, so I just turned off that feature :)

Copy link

ghost commented Sep 27, 2021

I don't have a M1 Mac yet, so not able to test this myself right now. But according to the homebrew-core github it looks like the Qemu patch is now handled automatically by the homebrew scripts for M1 Macs, so there is no need to patch manually: Homebrew/homebrew-core@5e8eb54

@toricls
Copy link
Author

toricls commented Sep 27, 2021

Thank you @selvakn and @ryanbrown-iress for the info! I just updated the Gist to follow the latest Homebrew and Lima versions :)

@dan-developer
Copy link

As of qemu version 6.1.0 it is no longer necessary to apply a patch.

@dhirajftcsolar
Copy link

I am new to Lima and nerdctl using in M1 MAC Book Pro.

  • I need to transfer /dev/tty.usbserial-0001 usb port to inside container. Kindly help.
  • Also VPN not working.

@jg123
Copy link

jg123 commented Mar 25, 2022

The url's to the ubuntu images are out of date in this gist. As of right now they should be https://cloud-images.ubuntu.com/releases/21.10/release-20220201/ubuntu-21.10-server-cloudimg-arm64.img
See here for hte latest url's: https://github.com/lima-vm/lima/blob/master/examples/default.yaml

@Aaron7noraA
Copy link

Hello, Can somebody give me some advice about some error I encounter.

I use M1 as my host computer but can not correctly install.

When I start the VM by default.yaml (BTW, the ubuntu img online links had been modified. The original one is outdated)
$ limactl start default.yaml

I get an error message below

FATA[0002] exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/{MY_PATH}/.lima/default/ha.stderr.log") 

I can not find any solution, and this issue has stocked me for a while.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment