Skip to content

Instantly share code, notes, and snippets.

@MaxXor
Last active October 14, 2025 08:52
Show Gist options
  • Save MaxXor/ba1665f47d56c24018a943bb114640d7 to your computer and use it in GitHub Desktop.
Save MaxXor/ba1665f47d56c24018a943bb114640d7 to your computer and use it in GitHub Desktop.

Revisions

  1. MaxXor revised this gist Dec 4, 2022. 1 changed file with 5 additions and 3 deletions.
    8 changes: 5 additions & 3 deletions btrfs-guide.md
    Original file line number Diff line number Diff line change
    @@ -47,8 +47,9 @@ mount -t btrfs -o defaults,noatime,compress=zstd /dev/mapper/data1 /mnt/data

    Automatically mount btrfs filesystem on boot by editing `/etc/fstab`:
    ```sh
    /dev/mapper/data1 /mnt/data btrfs defaults,noatime,compress=zstd 0 2
    UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /mnt/data btrfs defaults,noatime,compress=zstd 0 2
    # Add option 'autodefrag' to allow automatic defragmentation: useful for files with lot of random writes like databases or virtual machine images
    # Use 'blkid /dev/mapper/data1' to get the UUID of the Btrfs partition, it is common for all raid devices
    ```

    ## Maintenance
    @@ -134,7 +135,8 @@ btrfs replace start 2 /dev/mapper/data3 /mnt/data
    btrfs replace status /mnt/data
    ```

    Once the replace operation has finished, the fstab entry is left unmodified with one of the working devices:
    Once the replace operation has finished, the fstab entry is left unmodified:
    ```
    /dev/mapper/data1 /mnt/data btrfs defaults,noatime,compress=zstd 0 2
    UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /mnt/data btrfs defaults,noatime,compress=zstd 0 2
    # Use 'blkid /dev/mapper/data1' to get the UUID of the Btrfs partition, it is common for all raid devices
    ```
  2. MaxXor revised this gist Dec 4, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion btrfs-guide.md
    Original file line number Diff line number Diff line change
    @@ -134,7 +134,7 @@ btrfs replace start 2 /dev/mapper/data3 /mnt/data
    btrfs replace status /mnt/data
    ```

    The fstab entry is left unmodified with one of the working devices:
    Once the replace operation has finished, the fstab entry is left unmodified with one of the working devices:
    ```
    /dev/mapper/data1 /mnt/data btrfs defaults,noatime,compress=zstd 0 2
    ```
  3. MaxXor revised this gist Dec 4, 2022. 1 changed file with 27 additions and 7 deletions.
    34 changes: 27 additions & 7 deletions btrfs-guide.md
    Original file line number Diff line number Diff line change
    @@ -94,24 +94,44 @@ Example with one failed device:
    - `/dev/mapper/data3` new device
    - `/mnt/data` mountpoint

    In case of failing/failed device, mount in degraded mode with the working device:
    In case of failing/failed device, mount in degraded mode with one of the working devices:
    ```sh
    mount -t btrfs -o defaults,noatime,compress=zstd,degraded /dev/mapper/data1 /mnt/data
    ```

    NOTE: Encrypt the new device before using it in the btrfs raid by following the steps above. Add new device to btrfs raid:
    Find the the device ID of the missing disk by executing `btrfs device usage /mnt/data`:

    ```sh
    btrfs device add /dev/mapper/data3 /mnt/data
    # btrfs device usage /mnt/data

    /dev/mapper/data1, ID: 1
    Device size: 7.28TiB
    Device slack: 0.00B
    Data,RAID1: 5.46TiB
    Metadata,RAID1: 7.00GiB
    System,RAID1: 32.00MiB
    Unallocated: 1.81TiB

    missing, ID: 2
    Device size: 7.28TiB
    Device slack: 0.00B
    Data,RAID1: 5.46TiB
    Metadata,RAID1: 7.00GiB
    System,RAID1: 32.00MiB
    Unallocated: 1.81TiB

    ```

    Remove the missing device (NOTE: `missing` is a special device name and not a placeholder):
    NOTE: Encrypt the new device before using it in the btrfs raid by following the steps above.

    Start the replace operation in background by adding the new device to the btrfs raid using the device ID of the missing disk:
    ```sh
    btrfs device delete missing /mnt/data
    btrfs replace start 2 /dev/mapper/data3 /mnt/data
    ```

    Re-balance data of btrfs raid:
    (Optional) Check the replace progress:
    ```sh
    btrfs balance start /mnt/data
    btrfs replace status /mnt/data
    ```

    The fstab entry is left unmodified with one of the working devices:
  4. MaxXor revised this gist May 12, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion btrfs-guide.md
    Original file line number Diff line number Diff line change
    @@ -65,7 +65,7 @@ Documentation=man:btrfs-scrub
    [Timer]
    # first saturday each month
    OnCalendar=Sun *-*-1..7 3:00:00
    OnCalendar=Sat *-*-1..7 3:00:00
    RandomizedDelaySec=10min
    [Install]
  5. MaxXor revised this gist Sep 24, 2021. No changes.
  6. MaxXor revised this gist Sep 24, 2021. 1 changed file with 36 additions and 1 deletion.
    37 changes: 36 additions & 1 deletion btrfs-guide.md
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # Btrfs guide
    # Encrypted Btrfs storage setup and maintenance guide

    ## Initial setup with LUKS/dm-crypt

    @@ -51,6 +51,41 @@ Automatically mount btrfs filesystem on boot by editing `/etc/fstab`:
    # Add option 'autodefrag' to allow automatic defragmentation: useful for files with lot of random writes like databases or virtual machine images
    ```

    ## Maintenance

    In a btrfs raid setup it is necessary to frequently run a `btrfs scrub` to check for corrupted blocks/flipped bits and repair them using a healthy copy from one of the mirror disks.

    In the example below a systemd-timer is used to run an automatic `btrfs scrub` job each month.

    /etc/systemd/system/btrfs-scrub.timer:
    ```
    [Unit]
    Description=Monthly scrub btrfs filesystem, verify block checksums
    Documentation=man:btrfs-scrub
    [Timer]
    # first saturday each month
    OnCalendar=Sun *-*-1..7 3:00:00
    RandomizedDelaySec=10min
    [Install]
    WantedBy=timers.target
    ```

    /etc/systemd/system/btrfs-scrub.service:
    ```
    [Unit]
    Description=Scrub btrfs filesystem, verify block checksums
    Documentation=man:btrfs-scrub
    [Service]
    Type=simple
    ExecStart=/bin/btrfs scrub start -Bd /mnt/data
    KillSignal=SIGINT
    IOSchedulingClass=idle
    CPUSchedulingPolicy=idle
    ```

    ## Recovery from device failure

    Example with one failed device:
  7. MaxXor created this gist Oct 15, 2019.
    85 changes: 85 additions & 0 deletions btrfs-guide.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,85 @@
    # Btrfs guide

    ## Initial setup with LUKS/dm-crypt

    This exemplary initial setup uses two devices `/dev/sdb` and `/dev/sdc` but can be applied to any amount of devices by following the steps with additional devices.

    Create keyfile:
    ```sh
    dd bs=64 count=1 if=/dev/urandom of=/etc/cryptkey iflag=fullblock
    chmod 600 /etc/cryptkey
    ```

    Encrypt devices:
    ```sh
    cryptsetup -v -c aes-xts-plain64 -h sha512 -s 512 luksFormat /dev/sdb /etc/cryptkey
    cryptsetup -v -c aes-xts-plain64 -h sha512 -s 512 luksFormat /dev/sdc /etc/cryptkey
    ```

    Backup LUKS header:
    ```sh
    cryptsetup luksHeaderBackup --header-backup-file ~/sdb.header.bak /dev/sdb
    cryptsetup luksHeaderBackup --header-backup-file ~/sdc.header.bak /dev/sdc
    ```

    Automatically unlock LUKS devices on boot by editing `/etc/crypttab`:
    ```sh
    data1 UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /etc/cryptkey luks,noearly #,discard (for SSDs)
    data2 UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /etc/cryptkey luks,noearly #,discard (for SSDs)
    # Use 'blkid /dev/sdb' to get the UUID
    ```

    Unlock encrypted devices now to create the filesystem in next step:
    ```sh
    cryptsetup open --key-file=/etc/cryptkey --type luks /dev/sdb data1
    cryptsetup open --key-file=/etc/cryptkey --type luks /dev/sdc data2
    ```

    Create filesystem:
    ```sh
    mkfs.btrfs -m raid1 -d raid1 /dev/mapper/data1 /dev/mapper/data2
    ```

    Mount filesystem:
    ```sh
    mount -t btrfs -o defaults,noatime,compress=zstd /dev/mapper/data1 /mnt/data
    ```

    Automatically mount btrfs filesystem on boot by editing `/etc/fstab`:
    ```sh
    /dev/mapper/data1 /mnt/data btrfs defaults,noatime,compress=zstd 0 2
    # Add option 'autodefrag' to allow automatic defragmentation: useful for files with lot of random writes like databases or virtual machine images
    ```

    ## Recovery from device failure

    Example with one failed device:
    - `/dev/mapper/data1` working device
    - `/dev/mapper/data2` failed device
    - `/dev/mapper/data3` new device
    - `/mnt/data` mountpoint

    In case of failing/failed device, mount in degraded mode with the working device:
    ```sh
    mount -t btrfs -o defaults,noatime,compress=zstd,degraded /dev/mapper/data1 /mnt/data
    ```

    NOTE: Encrypt the new device before using it in the btrfs raid by following the steps above. Add new device to btrfs raid:
    ```sh
    btrfs device add /dev/mapper/data3 /mnt/data
    ```

    Remove the missing device (NOTE: `missing` is a special device name and not a placeholder):
    ```sh
    btrfs device delete missing /mnt/data
    ```

    Re-balance data of btrfs raid:
    ```sh
    btrfs balance start /mnt/data
    ```

    The fstab entry is left unmodified with one of the working devices:
    ```
    /dev/mapper/data1 /mnt/data btrfs defaults,noatime,compress=zstd 0 2
    ```