Skip to content

Instantly share code, notes, and snippets.

@busyboy77

busyboy77/rke2-commands.md

Forked from superseb/rke2-commands.md
Created September 30, 2024 13:10
Show Gist options
  • Save busyboy77/b8bef991dad4b9d90a8ca37d10cad14a to your computer and use it in GitHub Desktop.
Save busyboy77/b8bef991dad4b9d90a8ca37d10cad14a to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @superseb superseb revised this gist May 29, 2024. 1 changed file with 24 additions and 20 deletions.
    44 changes: 24 additions & 20 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,5 +1,7 @@
    # RKE2 commands

    * Updated on May 29 to accommodate etcd container not having `/bin/sh` available anymore.

    ## Install

    ```
    @@ -126,52 +128,54 @@ export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    * `etcdctl check perf`

    ```
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl check perf"; done
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt check perf; done
    ```

    * `etcdctl endpoint status`

    ```
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status"; done
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt endpoint status; done
    ```

    * `etcdctl endpoint health`

    ```
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint health"; done
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt endpoint health; done
    ```

    * `etcdctl alarm list`


    ```
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl alarm list"; done
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt alarm list; done
    ```

    * `etcdctl compact`

    ```
    rev=$(kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --write-out fields | grep Revision | cut -d: -f2")
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl compact \"$(echo $rev)\""
    rev=$(kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt endpoint status --write-out fields | grep Revision | cut -d: -f2)
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt compact \"$(echo $rev)\"
    ```

    * `etcdctl defrag`

    ```
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl defrag --cluster"
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt defrag --cluster
    ```

    * curl metrics

    Older versions of the etcd image have a version of curl in the image that does not work with EC certificate keys.

    Since k8s 1.28, `curl` is no longer present in the image

    ```
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- sh -c "curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key -s https://127.0.0.1:2379/metrics"
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key -s https://127.0.0.1:2379/metrics
    ```
    * curl health

    ```
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- sh -c "curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key -s https://127.0.0.1:2379/health"
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key -s https://127.0.0.1:2379/health
    ```

    ### on the etcd host itself
    @@ -186,48 +190,48 @@ etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.contai
    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl check perf"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt check perf
    ```

    * `etcdctl endpoint status`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --cluster --write-out=table"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt endpoint status --cluster --write-out=table
    ```

    * `etcdctl endpoint health`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint health --cluster --write-out=table"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt endpoint health --cluster --write-out=table
    ```

    * `etcdctl alarm list`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl alarm list"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt alarm list
    ```

    * `etcdctl compact`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    rev=$(/var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --write-out fields | grep Revision | cut -d: -f2")
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl compact \"$(echo $rev)\""
    rev=$(/var/lib/rancher/rke2/bin/crictl exec $etcdcontainer etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt endpoint status --write-out fields | grep Revision | cut -d: -f2)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt compact $rev
    ```

    * `etcdctl defrag`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl defrag --cluster"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt defrag --cluster
    ```

    * curl metrics
    @@ -245,29 +249,29 @@ curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var
    * Check etcd connectivity between members on HTTP/2379

    ```
    for endpoint in $(/var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl member list" |cut -d, -f5); do echo "Validating connection to ${endpoint}/health"; curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key "${endpoint}/health"; echo ""; done
    for endpoint in $(/var/lib/rancher/rke2/bin/crictl exec $etcdcontainer etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt member list |cut -d, -f5); do echo "Validating connection to ${endpoint}/health"; curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key "${endpoint}/health"; echo ""; done
    ```

    * Watching etcd changes

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl watch --prefix /registry"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt watch --prefix /registry
    ```

    * Query etcd directly

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl get /registry --prefix=true --keys-only"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt get /registry --prefix=true --keys-only
    ```

    * etcd list of count per key

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl get /registry --prefix=true --keys-only" | grep -v ^$ | awk -F'/' '{ if ($3 ~ /cattle.io/) {h[$3"/"$4]++} else { h[$3]++ }} END { for(k in h) print h[k], k }' | sort -nr
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer etcdctl --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt get /registry --prefix=true --keys-only | grep -v ^$ | awk -F'/' '{ if ($3 ~ /cattle.io/) {h[$3"/"$4]++} else { h[$3]++ }} END { for(k in h) print h[k], k }' | sort -nr
    ```
  2. @superseb superseb revised this gist Sep 7, 2023. 1 changed file with 22 additions and 0 deletions.
    22 changes: 22 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -8,6 +8,28 @@ systemctl daemon-reload
    systemctl start rke2-server
    ```

    More options to define RKE2 version

    ```
    # Stable channel
    # curl https://update.rke2.io/v1-release/channels/stable
    curl -sL https://get.rke2.io | INSTALL_RKE2_CHANNEL=stable sh
    # Latest channel
    # curl https://update.rke2.io/v1-release/channels/latest
    curl -sL https://get.rke2.io | INSTALL_RKE2_CHANNEL=latest sh
    # Testing channel
    # curl https://update.rke2.io/v1-release/channels/testing
    curl -sL https://get.rke2.io | INSTALL_RKE2_CHANNEL=testing sh
    # Minor channel for latest available patch version
    # curl https://update.rke2.io/v1-release/channels/v1.27
    curl -sL https://get.rke2.io | INSTALL_RKE2_CHANNEL=v1.27 sh
    # Minor channel for latest available patch version
    # curl https://update.rke2.io/v1-release/channels/v1.26
    curl -sL https://get.rke2.io | INSTALL_RKE2_CHANNEL=v1.26 sh
    # Configure specific version
    curl -sL https://get.rke2.io | INSTALL_RKE2_VERSION=v1.27.5+rke2r1 sh
    ```

    Various exploration/debug commmands for RKE2

    ## binaries
  3. @superseb superseb revised this gist Jul 27, 2023. No changes.
  4. @superseb superseb revised this gist Jul 27, 2023. 1 changed file with 24 additions and 0 deletions.
    24 changes: 24 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -224,4 +224,28 @@ curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var

    ```
    for endpoint in $(/var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl member list" |cut -d, -f5); do echo "Validating connection to ${endpoint}/health"; curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key "${endpoint}/health"; echo ""; done
    ```

    * Watching etcd changes

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl watch --prefix /registry"
    ```

    * Query etcd directly

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl get /registry --prefix=true --keys-only"
    ```

    * etcd list of count per key

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl get /registry --prefix=true --keys-only" | grep -v ^$ | awk -F'/' '{ if ($3 ~ /cattle.io/) {h[$3"/"$4]++} else { h[$3]++ }} END { for(k in h) print h[k], k }' | sort -nr
    ```
  5. @superseb superseb revised this gist Jul 27, 2023. 1 changed file with 6 additions and 0 deletions.
    6 changes: 6 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -218,4 +218,10 @@ curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var

    ```
    curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key https://127.0.0.1:2379/health
    ```

    * Check etcd connectivity between members on HTTP/2379

    ```
    for endpoint in $(/var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl member list" |cut -d, -f5); do echo "Validating connection to ${endpoint}/health"; curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key "${endpoint}/health"; echo ""; done
    ```
  6. @superseb superseb revised this gist Jul 27, 2023. 1 changed file with 9 additions and 1 deletion.
    10 changes: 9 additions & 1 deletion rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -141,8 +141,16 @@ kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd -

    * curl metrics

    This does not work with the embedded curl in the image and the ECDSA certificate (key)
    Older versions of the etcd image have a version of curl in the image that does not work with EC certificate keys.

    ```
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- sh -c "curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key -s https://127.0.0.1:2379/metrics"
    ```
    * curl health

    ```
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- sh -c "curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key -s https://127.0.0.1:2379/health"
    ```

    ### on the etcd host itself

  7. @superseb superseb revised this gist Jul 27, 2023. 1 changed file with 6 additions and 0 deletions.
    6 changes: 6 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -204,4 +204,10 @@ etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.contai

    ```
    curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key https://127.0.0.1:2379/metrics
    ```

    * curl health

    ```
    curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key https://127.0.0.1:2379/health
    ```
  8. @superseb superseb revised this gist Feb 22, 2022. 1 changed file with 28 additions and 32 deletions.
    60 changes: 28 additions & 32 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -13,43 +13,39 @@ Various exploration/debug commmands for RKE2
    ## binaries

    ```
    $ ls -la /var/lib/rancher/rke2/bin/
    total 263716
    drwxr-xr-x 2 root root 4096 Oct 9 15:53 .
    drwxr-xr-x 3 root root 4096 Oct 9 15:53 ..
    -rwxr-xr-x 1 root root 35422984 Oct 9 15:53 containerd
    -rwxr-xr-x 1 root root 7204400 Oct 9 15:53 containerd-shim
    -rwxr-xr-x 1 root root 10247488 Oct 9 15:53 containerd-shim-runc-v1
    -rwxr-xr-x 1 root root 10255744 Oct 9 15:53 containerd-shim-runc-v2
    -rwxr-xr-x 1 root root 21173056 Oct 9 15:53 crictl
    -rwxr-xr-x 1 root root 18724136 Oct 9 15:53 ctr
    -rwxr-xr-x 1 root root 44474208 Oct 9 15:52 kubectl
    -rwxr-xr-x 1 root root 111544592 Oct 9 15:53 kubelet
    -rwxr-xr-x 1 root root 10683624 Oct 9 15:53 runc
    -rwxr-xr-x 1 root root 285008 Oct 9 15:53 socat
    $ ls -1 /var/lib/rancher/rke2/bin/*
    /var/lib/rancher/rke2/bin/containerd
    /var/lib/rancher/rke2/bin/containerd-shim
    /var/lib/rancher/rke2/bin/containerd-shim-runc-v1
    /var/lib/rancher/rke2/bin/containerd-shim-runc-v2
    /var/lib/rancher/rke2/bin/crictl
    /var/lib/rancher/rke2/bin/ctr
    /var/lib/rancher/rke2/bin/kubectl
    /var/lib/rancher/rke2/bin/kubelet
    /var/lib/rancher/rke2/bin/runc
    ```

    ## tar.gz contents

    ```
    ./share
    ./share/rke2
    ./share/rke2/rke2-cis-sysctl.conf
    ./share/rke2/LICENSE.txt
    ./share/rke2-windows
    ./lib
    ./lib/systemd
    ./lib/systemd/system
    ./lib/systemd/system/rke2-server.env
    ./lib/systemd/system/rke2-agent.env
    ./lib/systemd/system/rke2-server.service
    ./lib/systemd/system/rke2-agent.service
    ./bin
    ./bin/rke2-killall.sh
    ./bin/rke2-uninstall.sh
    ./bin/rke2
    ./bin/rke2-uninstall.ps1
    ./bin/rke2.exe
    lib/
    lib/systemd/
    lib/systemd/system/
    lib/systemd/system/rke2-agent.service
    lib/systemd/system/rke2-agent.env
    lib/systemd/system/rke2-server.service
    lib/systemd/system/rke2-server.env
    share/
    share/rke2/
    share/rke2/rke2-cis-sysctl.conf
    share/rke2/LICENSE.txt
    share/rke2-windows/
    bin/
    bin/rke2.exe
    bin/rke2
    bin/rke2-uninstall.ps1
    bin/rke2-uninstall.sh
    bin/rke2-killall.sh
    ```

    ## systemd
  9. @superseb superseb revised this gist Nov 11, 2021. 1 changed file with 13 additions and 0 deletions.
    13 changes: 13 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -130,6 +130,19 @@ for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl alarm list"; done
    ```

    * `etcdctl compact`

    ```
    rev=$(kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --write-out fields | grep Revision | cut -d: -f2")
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl compact \"$(echo $rev)\""
    ```

    * `etcdctl defrag`

    ```
    kubectl -n kube-system exec $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name | head -1) -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl defrag --cluster"
    ```

    * curl metrics

    This does not work with the embedded curl in the image and the ECDSA certificate (key)
  10. @superseb superseb revised this gist Nov 11, 2021. 1 changed file with 17 additions and 0 deletions.
    17 changes: 17 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -174,6 +174,23 @@ etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.contai
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl alarm list"
    ```

    * `etcdctl compact`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    rev=$(/var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --write-out fields | grep Revision | cut -d: -f2")
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl compact \"$(echo $rev)\""
    ```

    * `etcdctl defrag`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl defrag --cluster"
    ```

    * curl metrics

    ```
  11. @superseb superseb revised this gist Sep 29, 2021. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -55,6 +55,7 @@ drwxr-xr-x 3 root root 4096 Oct 9 15:53 ..
    ## systemd

    * `/usr/local/lib/systemd/system/rke2-server.service`
    * `/usr/local/lib/systemd/system/rke2-agent.service`

    ## kubeconfig

  12. @superseb superseb revised this gist Sep 29, 2021. 1 changed file with 21 additions and 16 deletions.
    37 changes: 21 additions & 16 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -29,22 +29,27 @@ drwxr-xr-x 3 root root 4096 Oct 9 15:53 ..
    -rwxr-xr-x 1 root root 285008 Oct 9 15:53 socat
    ```

    ## tar.gz install contents

    ```
    /usr/local/share/
    /usr/local/share/rke2/
    /usr/local/share/rke2/rke2-cis-sysctl.conf
    /usr/local/share/rke2/LICENSE.txt
    /usr/local/bin/
    /usr/local/bin/rke2
    /usr/local/bin/rke2-uninstall.sh
    /usr/local/bin/rke2-killall.sh
    /usr/local/lib/
    /usr/local/lib/systemd/
    /usr/local/lib/systemd/system/
    /usr/local/lib/systemd/system/rke2-server.service
    /usr/local/lib/systemd/system/rke2-agent.service
    ## tar.gz contents

    ```
    ./share
    ./share/rke2
    ./share/rke2/rke2-cis-sysctl.conf
    ./share/rke2/LICENSE.txt
    ./share/rke2-windows
    ./lib
    ./lib/systemd
    ./lib/systemd/system
    ./lib/systemd/system/rke2-server.env
    ./lib/systemd/system/rke2-agent.env
    ./lib/systemd/system/rke2-server.service
    ./lib/systemd/system/rke2-agent.service
    ./bin
    ./bin/rke2-killall.sh
    ./bin/rke2-uninstall.sh
    ./bin/rke2
    ./bin/rke2-uninstall.ps1
    ./bin/rke2.exe
    ```

    ## systemd
  13. @superseb superseb revised this gist Sep 29, 2021. 1 changed file with 1 addition and 3 deletions.
    4 changes: 1 addition & 3 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -126,9 +126,7 @@ for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -

    * curl metrics

    ```
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- sh -c "curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key https://127.0.0.1:2379/metrics"; done
    ```
    This does not work with the embedded curl in the image and the ECDSA certificate (key)


    ### on the etcd host itself
  14. @superseb superseb revised this gist Sep 29, 2021. 1 changed file with 36 additions and 0 deletions.
    36 changes: 36 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -97,6 +97,42 @@ export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml

    ## etcd

    ### using kubectl

    * `etcdctl check perf`

    ```
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl check perf"; done
    ```

    * `etcdctl endpoint status`

    ```
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status"; done
    ```

    * `etcdctl endpoint health`

    ```
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint health"; done
    ```

    * `etcdctl alarm list`


    ```
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl alarm list"; done
    ```

    * curl metrics

    ```
    for etcdpod in $(kubectl -n kube-system get pod -l component=etcd --no-headers -o custom-columns=NAME:.metadata.name); do kubectl -n kube-system exec $etcdpod -- sh -c "curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key https://127.0.0.1:2379/metrics"; done
    ```


    ### on the etcd host itself

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
  15. @superseb superseb revised this gist Apr 20, 2021. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -115,15 +115,15 @@ etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.contai
    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --write-out=table"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --cluster --write-out=table"
    ```

    * `etcdctl endpoint health`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint health --write-out=table"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint health --cluster --write-out=table"
    ```

    * `etcdctl alarm list`
  16. @superseb superseb revised this gist Oct 28, 2020. 1 changed file with 8 additions and 0 deletions.
    8 changes: 8 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,5 +1,13 @@
    # RKE2 commands

    ## Install

    ```
    curl -sL https://get.rke2.io | sh
    systemctl daemon-reload
    systemctl start rke2-server
    ```

    Various exploration/debug commmands for RKE2

    ## binaries
  17. @superseb superseb revised this gist Oct 9, 2020. 1 changed file with 4 additions and 0 deletions.
    4 changes: 4 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -98,27 +98,31 @@ etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.contai

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl check perf"
    ```

    * `etcdctl endpoint status`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --write-out=table"
    ```

    * `etcdctl endpoint health`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint health --write-out=table"
    ```

    * `etcdctl alarm list`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl alarm list"
    ```

  18. @superseb superseb revised this gist Oct 9, 2020. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -83,7 +83,7 @@ export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml

    ## logging

    * `journalctl -u rke2`
    * `journalctl -f -u rke2-server`
    * `/var/lib/rancher/rke2/agent/containerd/containerd.log`
    * `/var/lib/rancher/rke2/agent/logs/kubelet.log`

  19. @superseb superseb revised this gist Oct 9, 2020. 1 changed file with 19 additions and 2 deletions.
    21 changes: 19 additions & 2 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -21,10 +21,27 @@ drwxr-xr-x 3 root root 4096 Oct 9 15:53 ..
    -rwxr-xr-x 1 root root 285008 Oct 9 15:53 socat
    ```

    ## tar.gz install contents

    ```
    /usr/local/share/
    /usr/local/share/rke2/
    /usr/local/share/rke2/rke2-cis-sysctl.conf
    /usr/local/share/rke2/LICENSE.txt
    /usr/local/bin/
    /usr/local/bin/rke2
    /usr/local/bin/rke2-uninstall.sh
    /usr/local/bin/rke2-killall.sh
    /usr/local/lib/
    /usr/local/lib/systemd/
    /usr/local/lib/systemd/system/
    /usr/local/lib/systemd/system/rke2-server.service
    /usr/local/lib/systemd/system/rke2-agent.service
    ```

    ## systemd

    * `/etc/systemd/system/rke2.service.env`
    * `/etc/systemd/system/rke2.service`
    * `/usr/local/lib/systemd/system/rke2-server.service`

    ## kubeconfig

  20. @superseb superseb revised this gist Oct 9, 2020. 1 changed file with 31 additions and 12 deletions.
    43 changes: 31 additions & 12 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -2,6 +2,25 @@

    Various exploration/debug commmands for RKE2

    ## binaries

    ```
    $ ls -la /var/lib/rancher/rke2/bin/
    total 263716
    drwxr-xr-x 2 root root 4096 Oct 9 15:53 .
    drwxr-xr-x 3 root root 4096 Oct 9 15:53 ..
    -rwxr-xr-x 1 root root 35422984 Oct 9 15:53 containerd
    -rwxr-xr-x 1 root root 7204400 Oct 9 15:53 containerd-shim
    -rwxr-xr-x 1 root root 10247488 Oct 9 15:53 containerd-shim-runc-v1
    -rwxr-xr-x 1 root root 10255744 Oct 9 15:53 containerd-shim-runc-v2
    -rwxr-xr-x 1 root root 21173056 Oct 9 15:53 crictl
    -rwxr-xr-x 1 root root 18724136 Oct 9 15:53 ctr
    -rwxr-xr-x 1 root root 44474208 Oct 9 15:52 kubectl
    -rwxr-xr-x 1 root root 111544592 Oct 9 15:53 kubelet
    -rwxr-xr-x 1 root root 10683624 Oct 9 15:53 runc
    -rwxr-xr-x 1 root root 285008 Oct 9 15:53 socat
    ```

    ## systemd

    * `/etc/systemd/system/rke2.service.env`
    @@ -11,11 +30,11 @@ Various exploration/debug commmands for RKE2

    ```
    export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
    kubectl get nodes
    /var/lib/rancher/rke2/bin/kubectl get nodes
    ```

    ```
    kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get nodes
    /var/lib/rancher/rke2/bin/kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get nodes
    ```

    ## containerd
    @@ -27,22 +46,22 @@ kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get nodes
    List containers using ctr

    ```
    ctr --address /run/k3s/containerd/containerd.sock --namespace k8s.io container ls
    /var/lib/rancher/rke2/bin/ctr --address /run/k3s/containerd/containerd.sock --namespace k8s.io container ls
    ```

    ### crictl (not installed/included by default)
    ### crictl

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    crictl ps
    /var/lib/rancher/rke2/bin/crictl ps
    ```

    ```
    crictl --config /var/lib/rancher/rke2/agent/etc/crictl.yaml ps
    /var/lib/rancher/rke2/bin/crictl --config /var/lib/rancher/rke2/agent/etc/crictl.yaml ps
    ```

    ```
    crictl --runtime-endpoint unix:///run/k3s/containerd/containerd.sock ps -a
    /var/lib/rancher/rke2/bin/crictl --runtime-endpoint unix:///run/k3s/containerd/containerd.sock ps -a
    ```

    ## logging
    @@ -55,35 +74,35 @@ crictl --runtime-endpoint unix:///run/k3s/containerd/containerd.sock ps -a

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(crictl ps --label io.kubernetes.container.name=etcd --quiet)
    etcdcontainer=$(/var/lib/rancher/rke2/bin/crictl ps --label io.kubernetes.container.name=etcd --quiet)
    ```

    * `etcdctl check perf`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl check perf"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl check perf"
    ```

    * `etcdctl endpoint status`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --write-out=table"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --write-out=table"
    ```

    * `etcdctl endpoint health`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint health --write-out=table"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint health --write-out=table"
    ```

    * `etcdctl alarm list`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl alarm list"
    /var/lib/rancher/rke2/bin/crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl alarm list"
    ```

    * curl metrics
  21. @superseb superseb revised this gist Oct 6, 2020. 1 changed file with 0 additions and 18 deletions.
    18 changes: 0 additions & 18 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -2,24 +2,6 @@

    Various exploration/debug commmands for RKE2

    ## binaries

    Necessary binaries unpacked from image and symlinked in `/usr/local/bin` (only when using `install.sh` script)

    ```
    -rwxr-xr-x 1 root root 151543800 Jul 22 06:22 rke2
    -rwxr-xr-x 1 root root 1014 Jul 22 06:22 rke2-uninstall.sh
    -rwxr-xr-x 1 root root 1716 Jul 22 06:22 rke2-killall.sh
    lrwxrwxrwx 1 root root 106 Jul 22 06:23 containerd -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/containerd
    lrwxrwxrwx 1 root root 119 Jul 22 06:23 containerd-shim-runc-v1 -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/containerd-shim-runc-v1
    lrwxrwxrwx 1 root root 111 Jul 22 06:23 containerd-shim -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/containerd-shim
    lrwxrwxrwx 1 root root 119 Jul 22 06:23 containerd-shim-runc-v2 -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/containerd-shim-runc-v2
    lrwxrwxrwx 1 root root 99 Jul 22 06:23 ctr -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/ctr
    lrwxrwxrwx 1 root root 103 Jul 22 06:23 kubelet -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/kubelet
    lrwxrwxrwx 1 root root 103 Jul 22 06:23 kubectl -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/kubectl
    lrwxrwxrwx 1 root root 100 Jul 22 06:23 runc -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/runc
    ```

    ## systemd

    * `/etc/systemd/system/rke2.service.env`
  22. @superseb superseb revised this gist Oct 5, 2020. 1 changed file with 6 additions and 0 deletions.
    6 changes: 6 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -102,4 +102,10 @@ crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETC
    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl alarm list"
    ```

    * curl metrics

    ```
    curl -L --cacert /var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --cert /var/lib/rancher/rke2/server/tls/etcd/server-client.crt --key /var/lib/rancher/rke2/server/tls/etcd/server-client.key https://127.0.0.1:2379/metrics
    ```
  23. @superseb superseb revised this gist Sep 29, 2020. No changes.
  24. @superseb superseb revised this gist Sep 29, 2020. 1 changed file with 5 additions and 0 deletions.
    5 changes: 5 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -72,29 +72,34 @@ crictl --runtime-endpoint unix:///run/k3s/containerd/containerd.sock ps -a
    ## etcd

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    etcdcontainer=$(crictl ps --label io.kubernetes.container.name=etcd --quiet)
    ```

    * `etcdctl check perf`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl check perf"
    ```

    * `etcdctl endpoint status`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --write-out=table"
    ```

    * `etcdctl endpoint health`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint health --write-out=table"
    ```

    * `etcdctl alarm list`

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl alarm list"
    ```
  25. @superseb superseb revised this gist Jul 30, 2020. 1 changed file with 4 additions and 0 deletions.
    4 changes: 4 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -71,6 +71,10 @@ crictl --runtime-endpoint unix:///run/k3s/containerd/containerd.sock ps -a

    ## etcd

    ```
    etcdcontainer=$(crictl ps --label io.kubernetes.container.name=etcd --quiet)
    ```

    * `etcdctl check perf`

    ```
  26. @superseb superseb revised this gist Jul 30, 2020. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -4,7 +4,7 @@ Various exploration/debug commmands for RKE2

    ## binaries

    Necessary binaries unpacked from image and symlinked in `/usr/local/bin`:
    Necessary binaries unpacked from image and symlinked in `/usr/local/bin` (only when using `install.sh` script)

    ```
    -rwxr-xr-x 1 root root 151543800 Jul 22 06:22 rke2
  27. @superseb superseb revised this gist Jul 22, 2020. 1 changed file with 26 additions and 0 deletions.
    26 changes: 26 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -68,3 +68,29 @@ crictl --runtime-endpoint unix:///run/k3s/containerd/containerd.sock ps -a
    * `journalctl -u rke2`
    * `/var/lib/rancher/rke2/agent/containerd/containerd.log`
    * `/var/lib/rancher/rke2/agent/logs/kubelet.log`

    ## etcd

    * `etcdctl check perf`

    ```
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl check perf"
    ```

    * `etcdctl endpoint status`

    ```
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint status --write-out=table"
    ```

    * `etcdctl endpoint health`

    ```
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl endpoint health --write-out=table"
    ```

    * `etcdctl alarm list`

    ```
    crictl exec $etcdcontainer sh -c "ETCDCTL_ENDPOINTS='https://127.0.0.1:2379' ETCDCTL_CACERT='/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt' ETCDCTL_CERT='/var/lib/rancher/rke2/server/tls/etcd/server-client.crt' ETCDCTL_KEY='/var/lib/rancher/rke2/server/tls/etcd/server-client.key' ETCDCTL_API=3 etcdctl alarm list"
    ```
  28. @superseb superseb revised this gist Jul 22, 2020. 1 changed file with 5 additions and 0 deletions.
    5 changes: 5 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -50,6 +50,11 @@ ctr --address /run/k3s/containerd/containerd.sock --namespace k8s.io container l

    ### crictl (not installed/included by default)

    ```
    export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
    crictl ps
    ```

    ```
    crictl --config /var/lib/rancher/rke2/agent/etc/crictl.yaml ps
    ```
  29. @superseb superseb created this gist Jul 22, 2020.
    65 changes: 65 additions & 0 deletions rke2-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,65 @@
    # RKE2 commands

    Various exploration/debug commmands for RKE2

    ## binaries

    Necessary binaries unpacked from image and symlinked in `/usr/local/bin`:

    ```
    -rwxr-xr-x 1 root root 151543800 Jul 22 06:22 rke2
    -rwxr-xr-x 1 root root 1014 Jul 22 06:22 rke2-uninstall.sh
    -rwxr-xr-x 1 root root 1716 Jul 22 06:22 rke2-killall.sh
    lrwxrwxrwx 1 root root 106 Jul 22 06:23 containerd -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/containerd
    lrwxrwxrwx 1 root root 119 Jul 22 06:23 containerd-shim-runc-v1 -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/containerd-shim-runc-v1
    lrwxrwxrwx 1 root root 111 Jul 22 06:23 containerd-shim -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/containerd-shim
    lrwxrwxrwx 1 root root 119 Jul 22 06:23 containerd-shim-runc-v2 -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/containerd-shim-runc-v2
    lrwxrwxrwx 1 root root 99 Jul 22 06:23 ctr -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/ctr
    lrwxrwxrwx 1 root root 103 Jul 22 06:23 kubelet -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/kubelet
    lrwxrwxrwx 1 root root 103 Jul 22 06:23 kubectl -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/kubectl
    lrwxrwxrwx 1 root root 100 Jul 22 06:23 runc -> /var/lib/rancher/rke2/data/e54e0eb8ba58f4f8e5e252bb58a07022700890934ec360ab816c01fe5d90ddef/bin/runc
    ```

    ## systemd

    * `/etc/systemd/system/rke2.service.env`
    * `/etc/systemd/system/rke2.service`

    ## kubeconfig

    ```
    export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
    kubectl get nodes
    ```

    ```
    kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get nodes
    ```

    ## containerd

    * socket located at `/run/k3s/containerd/containerd.sock`

    ### ctr

    List containers using ctr

    ```
    ctr --address /run/k3s/containerd/containerd.sock --namespace k8s.io container ls
    ```

    ### crictl (not installed/included by default)

    ```
    crictl --config /var/lib/rancher/rke2/agent/etc/crictl.yaml ps
    ```

    ```
    crictl --runtime-endpoint unix:///run/k3s/containerd/containerd.sock ps -a
    ```

    ## logging

    * `journalctl -u rke2`
    * `/var/lib/rancher/rke2/agent/containerd/containerd.log`
    * `/var/lib/rancher/rke2/agent/logs/kubelet.log`