Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save bglopez/6c7817cfe6b7acfba73e79b57b7fdad6 to your computer and use it in GitHub Desktop.
Save bglopez/6c7817cfe6b7acfba73e79b57b7fdad6 to your computer and use it in GitHub Desktop.

Revisions

  1. @chrisdone chrisdone renamed this gist Jan 1, 2018. 1 changed file with 0 additions and 0 deletions.
    File renamed without changes.
  2. @chrisdone chrisdone created this gist Jan 1, 2018.
    519 changes: 519 additions & 0 deletions gistfile1.txt
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,519 @@
    ## Common

    ````
    export OPT=/opt
    export BUILDS=/some/where/mini_linux
    mkdir -p $BUILDS
    ````

    ## Linux kernel

    ````
    export LINUX=$OPT/linux
    export LINUX_BUILD=$BUILDS/linux
    mkdir -p $LINUX_BUILD
    cd $LINUX
    make O=$LINUX_BUILD allnoconfig
    cd $LINUX_BUILD
    make menuconfig
    ````

    Configure the kernel according the following:

    ````
    64-bit kernel ---> yes
    General setup ---> Initial RAM filesystem and RAM disk (initramfs/initrd) support ---> yes
    General setup ---> Configure standard kernel features ---> Enable support for printk ---> yes
    Executable file formats / Emulations ---> Kernel support for ELF binaries ---> yes
    Executable file formats / Emulations ---> Kernel support for scripts starting with #! ---> yes
    Device Drivers ---> Generic Driver Options ---> Maintain a devtmpfs filesystem to mount at /dev ---> yes
    Device Drivers ---> Generic Driver Options ---> Automount devtmpfs at /dev, after the kernel mounted the rootfs ---> yes
    Device Drivers ---> Character devices ---> Enable TTY ---> yes
    Device Drivers ---> Character devices ---> Serial drivers ---> 8250/16550 and compatible serial support ---> yes
    Device Drivers ---> Character devices ---> Serial drivers ---> Console on 8250/16550 and compatible serial port ---> yes
    File systems ---> Pseudo filesystems ---> /proc file system support ---> yes
    File systems ---> Pseudo filesystems ---> sysfs file system support ---> yes
    ````

    Build the kernel:

    ````
    time make -j8
    ````

    ````
    ...
    Kernel: arch/x86/boot/bzImage is ready (#1)

    real 2m37.247s
    user 1m58.541s
    sys 0m25.542s
    ````

    ## Busybox

    ````
    export BUSYBOX=$OPT/busybox
    export BUSYBOX_BUILD=$BUILDS/busybox
    mkdir -p $BUSYBOX_BUILD
    cd $BUSYBOX
    make O=$BUSYBOX_BUILD defconfig
    cd $BUSYBOX_BUILD
    make menuconfig
    ````

    Configure Busybox according the following:

    ````
    Busybox Settings ---> Build Options ---> Build BusyBox as a static binary (no shared libs) ---> yes
    ````

    Build Busybox:

    ````
    time make -j8
    ````

    ````
    ...
    Final link with: m

    real 0m20.356s
    user 0m46.959s
    sys 0m10.628s
    ````

    Install Busybox:

    ````
    make install
    ````

    Create an initramfs:

    ````
    export INITRAMFS_BUILD=$BUILDS/initramfs
    mkdir -p $INITRAMFS_BUILD
    cd $INITRAMFS_BUILD
    mkdir -p bin sbin etc proc sys usr/bin usr/sbin
    cp -a $BUSYBOX_BUILD/_install/* .
    ````

    Add a `$INITRAMFS_BUILD/init` script to the initramfs with the following content:

    ````
    #!/bin/sh

    mount -t proc none /proc
    mount -t sysfs none /sys

    cat <<!


    Boot took $(cut -d' ' -f1 /proc/uptime) seconds

    _ _ __ _
    /\/\ (_)_ __ (_) / /(_)_ __ _ ___ __
    / \| | '_ \| | / / | | '_ \| | | \ \/ /
    / /\/\ \ | | | | | / /__| | | | | |_| |> <
    \/ \/_|_| |_|_| \____/_|_| |_|\__,_/_/\_\


    Welcome to mini_linux


    !
    exec /bin/sh
    ````

    Create the initramfs archive:

    ````
    chmod +x init
    find . -print0 | cpio --null -ov --format=newc \
    | gzip -9 > $BUILDS/initramfs.cpio.gz
    ````

    ## Run and see (`<CTRL>a x` to quit)

    ````
    qemu-system-x86_64 -kernel $LINUX_BUILD/arch/x86_64/boot/bzImage \
    -initrd $BUILDS/initramfs.cpio.gz -nographic \
    -append "console=ttyS0"
    ````

    Note: for better performance, add the `-enable-kvm` option if your host has KVM enabled:
    ````
    qemu-system-x86_64 -kernel $LINUX_BUILD/arch/x86_64/boot/bzImage \
    -initrd $BUILDS/initramfs.cpio.gz -nographic \
    -append "console=ttyS0" -enable-kvm
    ````

    ````
    ...


    Boot took 0.45 seconds

    _ _ __ _
    /\/\ (_)_ __ (_) / /(_)_ __ _ ___ __
    / \| | '_ \| | / / | | '_ \| | | \ \/ /
    / /\/\ \ | | | | | / /__| | | | | |_| |> <
    \/ \/_|_| |_|_| \____/_|_| |_|\__,_/_/\_\


    Welcome to mini_linux


    / # ls /
    bin etc linuxrc root sys
    dev init proc sbin usr
    / # QEMU: Terminated
    ````

    ## Buildroot

    We assume that a toolchain is available in `/opt/toolchains/x86_64-unknown-linux-gnu` with prefix `x86_64-unknown-linux-gnu`, gcc version 5.x, kernel headers series 4.3.x, glibc C library and C++ support. These are reasonable defaults if you are using a toolchain generated by crosstool-NG. Adapt to your own situation. Notes:
    * You cannot use the native toolchain of your host computer (see Buildroot documentation to understand why).
    * If you do not have a toolchain already, you can build one using crosstool-NG (or Buildroot itself) and reuse it for other projects.
    * crosstool-NG is the recommended tool to build your own toolchain but avoid using uClibc (no IPV6 support), prefer uClibc-ng or glibc.
    * You can also use the built-in toolchain of Buildroot but be aware that it will take way longer than using an existing toolchain. Be also aware that in many cases you will have to re-build the toolchain after modifying the Buildroot configuration.
    * No yet convinced? Please use crosstool-NG, build and use your own toolchain.

    ````
    export BUILDROOT=$OPT/buildroot
    export BUILDROOT_BUILD=$BUILDS/buildroot
    mkdir -p $BUILDROOT_BUILD
    cd $BUILDROOT_BUILD
    touch Config.in external.mk
    echo 'name: mini_linux' > external.desc
    echo 'desc: minimal linux system with buildroot' >> external.desc
    mkdir configs overlay
    cd $BUILDROOT
    make O=$BUILDROOT_BUILD BR2_EXTERNAL=$BUILDROOT_BUILD qemu_x86_64_defconfig
    cd $BUILDROOT_BUILD
    make menuconfig
    ````

    Configure Buildroot according the following:

    ````
    Build options ---> Location to save buildroot config ---> $(BR2_EXTERNAL)/configs/mini_linux_defconfig
    Build options ---> Download dir ---> /some/where/buildroot_dl
    Build options ---> Number of jobs to run simultaneously (0 for auto) ---> 8
    Build options ---> Enable compiler cache ---> yes
    Build options ---> Compiler cache location ---> /some/where/buildroot_ccache
    Toolchain ---> Toolchain type ---> External toolchain
    Toolchain ---> Toolchain ---> Custom toolchain
    Toolchain ---> Toolchain origin ---> Pre-installed toolchain
    Toolchain ---> Toolchain path ---> /opt/toolchains/x86_64-unknown-linux-gnu
    Toolchain ---> Toolchain prefix ---> x86_64-unknown-linux-gnu
    Toolchain ---> External toolchain gcc version ---> 5.x
    Toolchain ---> External toolchain kernel headers series ---> 4.3.x
    Toolchain ---> External toolchain C library ---> glibc/eglibc
    Toolchain ---> Toolchain has C++ support? ---> yes
    System configuration ---> System hostname ---> mini_linux
    System configuration ---> System banner ---> Welcome to mini_linux
    System configuration ---> Run a getty (login prompt) after boot ---> TTY port ---> ttyS0
    System configuration ---> Network interface to configure through DHCP --->
    System configuration ---> Root filesystem overlay directories ---> $(BR2_EXTERNAL)/overlay
    Kernel ---> Linux Kernel ---> no
    Filesystem images ---> cpio the root filesystem (for use as an initial RAM filesystem) ---> yes
    Filesystem images ---> Compression method ---> gzip
    ````

    Save the configuration and build:

    ````
    make savedefconfig
    ````

    Add a `$BUILDROOT_BUILD/overlay/init` script to the overlay with the following content:

    ````
    #!/bin/sh
    /bin/mount -t devtmpfs devtmpfs /dev
    /bin/mount -t proc none /proc
    /bin/mount -t sysfs none /sys
    exec 0</dev/console
    exec 1>/dev/console
    exec 2>/dev/console
    cat <<!


    Boot took $(cut -d' ' -f1 /proc/uptime) seconds

    _ _ __ _
    /\/\ (_)_ __ (_) / /(_)_ __ _ ___ __
    / \| | '_ \| | / / | | '_ \| | | \ \/ /
    / /\/\ \ | | | | | / /__| | | | | |_| |> <
    \/ \/_|_| |_|_| \____/_|_| |_|\__,_/_/\_\


    Welcome to mini_linux


    !
    exec /bin/sh
    ````

    Build the root filesystem:

    ````
    chmod +x overlay/init
    time make
    ````

    ````
    ...
    real 1m52.905s
    user 0m50.682s
    sys 0m36.928s
    ````

    ## Run and see (`<CTRL>a x` to quit)

    ````
    qemu-system-x86_64 -kernel $LINUX_BUILD/arch/x86_64/boot/bzImage \
    -initrd $BUILDROOT_BUILD/images/rootfs.cpio.gz -nographic \
    -append "console=ttyS0"
    ````

    Note: for better performance, add the `-enable-kvm` option if your host has KVM enabled.

    ````
    ...


    Boot took 0.57 seconds

    _ _ __ _
    /\/\ (_)_ __ (_) / /(_)_ __ _ ___ __
    / \| | '_ \| | / / | | '_ \| | | \ \/ /
    / /\/\ \ | | | | | / /__| | | | | |_| |> <
    \/ \/_|_| |_|_| \____/_|_| |_|\__,_/_/\_\


    Welcome to mini_linux


    / # ls /
    bin init linuxrc opt run tmp
    dev lib media proc sbin usr
    etc lib64 mnt root sys var
    / # QEMU: Terminated
    ````

    ## Add and run a custom user application

    Create a new directory for the custom user applications:

    ````
    export APPS=$BUILDS/apps
    mkdir -p $APPS
    cd $APPS
    ````

    Add an application source file `$APPS/hello_world.c` with the following content:

    ````
    #include <stdio.h>

    int main(int argc, char **argv) {
    printf("mini_linux says: Hello world!\n");
    return 0;
    }
    ````

    Add a `$APPS/Makefile` with the following content (replace the `CROSS_COMPILE` definition with whatever is appropriate in your specific case):

    ````
    CROSS_COMPILE := /opt/toolchains/x86_64-unknown-linux-gnu/bin/x86_64-unknown-linux-gnu-
    CC := $(CROSS_COMPILE)gcc

    hello_world: hello_world.o
    $(CC) -o $@ $<

    hello_world.o: hello_world.c
    $(CC) -c -o $@ $<

    clean:
    rm -f hello_world hello_world.o
    ````

    Compile the application, copy it in the Buildroot overlay directory and re-build the root filesystem:

    ````
    make
    ...
    cp hello_world $BUILDROOT_BUILD/overlay
    ...
    cd $BUILDROOT_BUILD
    make
    ````

    ## Run and see (`<CTRL>a x` to quit)

    ````
    qemu-system-x86_64 -kernel $LINUX_BUILD/arch/x86_64/boot/bzImage \
    -initrd $BUILDROOT_BUILD/images/rootfs.cpio.gz -nographic \
    -append "console=ttyS0"
    ````

    Note: for better performance, add the `-enable-kvm` option if your host has KVM enabled.

    ````
    ...


    Boot took 0.57 seconds

    _ _ __ _
    /\/\ (_)_ __ (_) / /(_)_ __ _ ___ __
    / \| | '_ \| | / / | | '_ \| | | \ \/ /
    / /\/\ \ | | | | | / /__| | | | | |_| |> <
    \/ \/_|_| |_|_| \____/_|_| |_|\__,_/_/\_\


    Welcome to mini_linux


    / # ls
    bin init media root tmp
    dev lib mnt run usr
    etc lib64 opt sbin var
    hello_world linuxrc proc sys
    / # ./hello_world
    mini_linux says: Hello world!
    / # QEMU: Terminated
    ````

    ## Add loadable module support to the Linux kernel

    ````
    cd $LINUX_BUILD
    make menuconfig
    ````

    Change the kernel configuration according the following:

    ````
    Enable loadable module support ---> yes
    ````

    Re-build the kernel and its modules (none, in our case) and install the modules in the Buildroot overlay directory:

    ````
    make -j8
    make -j8 modules
    make modules_install INSTALL_MOD_PATH=$BUILDROOT_BUILD/overlay
    ````

    ## Add a custom user module

    Create a new directory for the custom user modules:

    ````
    export MODULES=$BUILDS/modules
    mkdir -p $MODULES
    cd $MODULES
    ````

    Add a module source file `$MODULES/hello_world.c` with the following content:

    ````
    /* hello_world.c */
    #include <linux/init.h>
    #include <linux/module.h>
    #include <linux/kernel.h>

    static int __init first_init(void)
    {
    pr_info("mini_linux module says: Hello world!\n");
    return 0;
    }

    static void __exit first_exit(void)
    {
    pr_info("Bye\n");
    }

    module_init(first_init);
    module_exit(first_exit);

    MODULE_LICENSE("GPL");
    MODULE_DESCRIPTION("My first module");
    MODULE_AUTHOR("The Doctor");
    ````

    Add a `$MODULES/Makefile` with the following content:

    ````
    ifneq ($(KERNELRELEASE),)
    # kbuild part of makefile
    obj-m := hello_world.o

    else
    # normal makefile
    KDIR ?= $(LINUX_BUILD)

    default:
    $(MAKE) -C $(KDIR) M=$$PWD

    modules_install:
    $(MAKE) -C $(KDIR) M=$$PWD $@

    clean:
    rm -rf *.o .*.cmd *.ko hello_world.mod.c modules.order Module.symvers .tmp_versions
    endif
    ````

    Compile the module, install it in the Buildroot overlay directory and re-build the root filesystem:

    ````
    make
    ...
    make modules_install INSTALL_MOD_PATH=$BUILDROOT_BUILD/overlay
    ...
    cd $BUILDROOT_BUILD
    make
    ````

    ## Run and see (`<CTRL>a x` to quit)

    ````
    qemu-system-x86_64 -kernel $LINUX_BUILD/arch/x86_64/boot/bzImage \
    -initrd $BUILDROOT_BUILD/images/rootfs.cpio.gz -nographic \
    -append "console=ttyS0"
    ````

    Note: for better performance, add the `-enable-kvm` option if your host has KVM enabled.

    ````
    ...


    Boot took 0.57 seconds

    _ _ __ _
    /\/\ (_)_ __ (_) / /(_)_ __ _ ___ __
    / \| | '_ \| | / / | | '_ \| | | \ \/ /
    / /\/\ \ | | | | | / /__| | | | | |_| |> <
    \/ \/_|_| |_|_| \____/_|_| |_|\__,_/_/\_\


    Welcome to mini_linux


    / # ls lib/modules/4.8.0\+/extra
    hello_world.ko
    / # lsmod
    Module Size Used by Not tainted
    / # insmod lib/modules/4.8.0\+/extra/hello_world.ko
    hello_world: loading out-of-tree module taints kernel.
    mini_linux module says: Hello world!
    / # lsmod
    Module Size Used by Tainted: G
    hello_world 704 -
    / # QEMU: Terminated
    ````