Skip to content

Instantly share code, notes, and snippets.

@rosswf
Last active September 11, 2025 01:47
Show Gist options
  • Select an option

  • Save rosswf/e5c4c85efb54a9f8d6e19a21cf09aa63 to your computer and use it in GitHub Desktop.

Select an option

Save rosswf/e5c4c85efb54a9f8d6e19a21cf09aa63 to your computer and use it in GitHub Desktop.

Revisions

  1. rosswf revised this gist Apr 24, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion k3s.md
    Original file line number Diff line number Diff line change
    @@ -99,7 +99,7 @@ alias kube-vip="ctr run --rm --net-host ghcr.io/kube-vip/kube-vip:v0.4.4 vip /ku
    ### Generate and deploy manifest
    ```bash
    kube-vip manifest daemonset \
    --interface eth0 \
    --interface eth0 \
    --address 192.168.0.160 \
    --inCluster \
    --taint \
  2. rosswf revised this gist Apr 24, 2022. 1 changed file with 4 additions and 5 deletions.
    9 changes: 4 additions & 5 deletions k3s.md
    Original file line number Diff line number Diff line change
    @@ -65,10 +65,6 @@ k3sup install \

    kube-config will be saved to $HOME/.kube/config for use with kubectl.

    ### Modify kubeconfig.yml

    Modify kubeconfig and change the server IP address to that used by kube-vip `192.168.0.160`.

    # Deploy kube-vip DaemonSet

    #### Installation [Docs](https://kube-vip.chipzoller.dev/docs/installation/daemonset/)
    @@ -108,13 +104,16 @@ kube-vip manifest daemonset \
    --inCluster \
    --taint \
    --controlplane \
    --services \
    --arp \
    --leaderElection | tee /var/lib/rancher/k3s/server/manifests/kube-vip.yaml
    ```

    This should autodeploy because the manifest is copied to the k3s/server folder.

    ### Modify kubeconfig.yml

    Modify kubeconfig and change the server IP address to that used by kube-vip `192.168.0.160`.

    # Deploy Addtional Master Nodes

    ### Requirements & Information needed:
  3. rosswf revised this gist Apr 22, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion k3s.md
    Original file line number Diff line number Diff line change
    @@ -58,7 +58,7 @@ k3sup install \
    --tls-san 192.168.0.160 \
    --cluster \
    --k3s-channel latest \
    --k3s-extra-args "--no-deploy servicelb" \
    --k3s-extra-args "--disable servicelb" \
    --local-path $HOME/.kube/config \
    --user ross
    ```
  4. rosswf revised this gist Apr 22, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion k3s.md
    Original file line number Diff line number Diff line change
    @@ -280,7 +280,7 @@ curl 192.168.0.XXX

    ### Teardown
    ```bash
    kubectl delete deployment,service
    kubectl delete deployment,service nginx
    ```

    # Resources and Thanks
  5. rosswf revised this gist Apr 22, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion k3s.md
    Original file line number Diff line number Diff line change
    @@ -55,7 +55,7 @@ Requirements:
    ```bash
    k3sup install \
    --ip 192.168.0.161 \
    --tls-san 192.168.0.160
    --tls-san 192.168.0.160 \
    --cluster \
    --k3s-channel latest \
    --k3s-extra-args "--no-deploy servicelb" \
  6. rosswf revised this gist Apr 22, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion k3s.md
    Original file line number Diff line number Diff line change
    @@ -40,7 +40,7 @@ Requirements:
    - SSH key copied over
    - Sudo setup for no password `%sudo ALL=(ALL:ALL) NOPASSWD: ALL`

    # First Master Node
    # Deploy First Master Node

    ### Requirements & Information needed:
    - IP Address of node: `--ip = 192.168.0.161` (dust-01)
  7. rosswf created this gist Apr 22, 2022.
    290 changes: 290 additions & 0 deletions k3s.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,290 @@
    # Prerequisites

    ### kubectl

    #### Install the required tools for deploying and controlling k3s.

    #### Installation [Docs](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/):
    ```bash
    # Download
    curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
    ```
    ```bash
    # Checksum
    curl -LO "https://dl.k8s.io/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"
    echo "$(cat kubectl.sha256) kubectl" | sha256sum --check
    ```
    ```bash
    # Install
    sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
    ```

    ### k3sup

    #### Installation [Docs](https://github.com/alexellis/k3sup#download-k3sup-tldr)

    ```bash
    # Download
    curl -sLS https://get.k3sup.dev | sh
    ```

    ```bash
    # Install
    sudo install k3sup /usr/local/bin/
    ```

    ### Nodes

    Requirements:
    - Static IP address
    - SSH key copied over
    - Sudo setup for no password `%sudo ALL=(ALL:ALL) NOPASSWD: ALL`

    # First Master Node

    ### Requirements & Information needed:
    - IP Address of node: `--ip = 192.168.0.161` (dust-01)
    - IP Address to be used by kube-vup: `--tls-san = 192.168.0.160`
    - User with sudo priviledges: `--user ross`


    ### Deploy k3s:

    ***Note: Ensure `$HOME/.kube/` folder exists***

    ```bash
    k3sup install \
    --ip 192.168.0.161 \
    --tls-san 192.168.0.160
    --cluster \
    --k3s-channel latest \
    --k3s-extra-args "--no-deploy servicelb" \
    --local-path $HOME/.kube/config \
    --user ross
    ```

    kube-config will be saved to $HOME/.kube/config for use with kubectl.

    ### Modify kubeconfig.yml

    Modify kubeconfig and change the server IP address to that used by kube-vip `192.168.0.160`.

    # Deploy kube-vip DaemonSet

    #### Installation [Docs](https://kube-vip.chipzoller.dev/docs/installation/daemonset/)

    ### Requirements & Information needed:
    - IP Address to be used by kube-vup: `--address 192.168.0.160`
    - Network interface (check with `ip a`): `--interface eth0`

    ### Apply RBAC

    ```bash
    kubectl apply -f https://kube-vip.io/manifests/rbac.yaml
    ```

    ### Pull image and create alias

    ***SSH into first master node and do these steps as root***
    ```bash
    ssh 192.168.0.161
    ```

    ```bash
    # Pull image - Check latest version on github or refer to docs.
    ctr image pull ghcr.io/kube-vip/kube-vip:v0.4.4
    ```

    ```bash
    # Create alias
    alias kube-vip="ctr run --rm --net-host ghcr.io/kube-vip/kube-vip:v0.4.4 vip /kube-vip"
    ```

    ### Generate and deploy manifest
    ```bash
    kube-vip manifest daemonset \
    --interface eth0 \
    --address 192.168.0.160 \
    --inCluster \
    --taint \
    --controlplane \
    --services \
    --arp \
    --leaderElection | tee /var/lib/rancher/k3s/server/manifests/kube-vip.yaml
    ```

    This should autodeploy because the manifest is copied to the k3s/server folder.

    # Deploy Addtional Master Nodes

    ### Requirements & Information needed:

    - IP Address of new master nodes:
    - `--ip = 192.168.0.162` (dust-02)
    - `--ip = 192.168.0.163` (dust-03)
    - IP address used by kube-vip: `--server-ip 192.168.0.160`
    - User with sudo priviledges: `--user ross`

    ### Deploy k3s to additional master nodes & join cluster:

    ***Note `--server` flag is used***

    ```bash
    k3sup join \
    --ip 192.168.0.162 \
    --server-ip 192.168.0.160 \
    --server \
    --k3s-channel latest \
    --user ross
    ```

    ### Check nodes are up
    ```bash
    kubectl get nodes
    ```

    # Add Worker Nodes

    ### Requirements & Information needed:

    - IP Address of new worker nodes:
    - `--ip = 192.168.0.164` (dust-04)
    - `--ip = 192.168.0.165` (dust-05)
    - IP address used by kube-vip: `--server-ip 192.168.0.160`
    - User with sudo priviledges: `--user ross`

    ### Deploy k3s to additional master nodes & join cluster:

    ```bash
    k3sup join \
    --ip 192.168.0.164 \
    --server-ip 192.168.0.160 \
    --k3s-channel latest \
    --user ross
    ```

    ### Check nodes are up
    ```bash
    kubectl get nodes
    ```

    # Deploy and Configure MetalLB
    #### Installation [Docs](https://metallb.universe.tf/installation/)

    ### Install manifests

    ```bash
    # Create namespace (metallb-system)
    kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/namespace.yaml
    ```

    ```bash
    # Deploy metallb
    kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml
    ```

    ### Create ConfigMap

    Create config.yml as below, modify `addresses` to be the range of IP addresses MetalLb can hand out.

    ```yml
    # config.yml
    apiVersion: v1
    kind: ConfigMap
    metadata:
    namespace: metallb-system
    name: config
    data:
    config: |
    address-pools:
    - name: default
    protocol: layer2
    addresses:
    - 192.168.0.170-192.168.0.180
    ```
    ### Deploy ConfigMap
    ```bash
    kubectl apply -f config.yml
    ```

    ### Check MetalLb is running
    ```bash
    kubectl get ds -n metallb-system
    ```

    # Test Deployment using Nginx

    ### Create manifests

    ```yml
    # deployment.yml
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: nginx
    spec:
    selector:
    matchLabels:
    app: nginx
    replicas: 3
    template:
    metadata:
    labels:
    app: nginx
    spec:
    containers:
    - name: nginx
    image: nginx:alpine
    ports:
    - containerPort: 80
    ```
    ```yml
    # service.yml
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: nginx
    spec:
    selector:
    app: nginx
    ports:
    - port: 80
    targetPort: 80
    type: LoadBalancer
    ```
    ### Deploy manifests
    ```bash
    kubectl apply -f deployment.yml
    ```

    ```bash
    kubectl apply -f service.yml
    ```

    ### Check IP address assigned by Metal-LB
    ```bash
    kubectl describe service nginx
    ```

    ### Check accessible
    ```bash
    # ip address handed out by MetalLB from describe service above
    curl 192.168.0.XXX
    ```

    ### Teardown
    ```bash
    kubectl delete deployment,service
    ```

    # Resources and Thanks
    Thanks to:
    - [adrian (dot) goins](https://www.youtube.com/watch?v=9PLw1xalcYA)
    - [Alex Ellis](https://blog.alexellis.io/bare-metal-kubernetes-with-k3s/)
    - [Techno Tim](https://www.youtube.com/watch?v=CbkEWcUZ7zM)