tosin2013 · August 13, 2020 22:33 · Feb 6, 2020 · Feb 6, 2020 · Feb 6, 2020 · Feb 5, 2020
diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -1,14 +1,16 @@
 ## What is this about?
 
-This gist describes a possible IoT/Edge computing setup using Kubernetes-style declarative management. It utilizes standalone kubelet + CRI-O + CNI on Fedora Core 31. The goal is to place a Kubernetes Pod manifest on an single node and access the application from the network.
+This gist describes how to set up standalone kubelet + CRI-O + CNI on Fedora Core 31. The goal is to place a Kubernetes Pod manifest on an single node and access the application from the network. This guide has been tested on x86-64 and armv7 deployments.
 
 ### Prepare the system
 
 Make sure the system is up to date:
 
 `dnf -y update`
 
-On Fedora 31 [by default cgroup v2](https://www.redhat.com/sysadmin/fedora-31-control-group-v2) are used. However the kubelet does not seem to be compatible with this as of time of writing. Revert this by adding the `systemd.unified_cgroup_hierarchy=0` switch to the kernel command line in `/etc/default/grub` like below:
+On Fedora 31 [by default cgroup v2](https://www.redhat.com/sysadmin/fedora-31-control-group-v2) are used. However the kubelet does not seem to be compatible with this as of time of writing. Revert this by adding the `systemd.unified_cgroup_hierarchy=0` switch to the kernel command line in
+
+**For x86-64 systems**, modify `/etc/default/grub` like below:
 
 ```diff
  GRUB_DEFAULT=saved
@@ -20,12 +22,31 @@ On Fedora 31 [by default cgroup v2](https://www.redhat.com/sysadmin/fedora-31-co
  GRUB_ENABLE_BLSCFG=true
 ```
 
-Regenerate your bootloader configuration...
+Regenerate your bootloader configuration using Grub2...
 
 on a BIOS-based system install `grub2-mkconfig -o /boot/grub2/grub.cfg`
 
 or on UEFI-based installs with `grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg`
 
+**For armv7 based systems** which employ extlinux, modify `/boot/extlinux/extlinux.conf`:
+
+```diff
+default=Fedora-Minimal-armhfp-31-1.9 (5.3.7-301.fc31.armv7hl)
+ label Fedora (5.4.17-200.fc31.armv7hl) 31 (Thirty One)
+ 	kernel /vmlinuz-5.4.17-200.fc31.armv7hl
+-	append ro root=UUID=b0abeeb4-7c58-4b29-9c60-3377437cde8a rhgb quiet LANG=en_US.UTF-8
++	append ro root=UUID=b0abeeb4-7c58-4b29-9c60-3377437cde8a rhgb quiet LANG=en_US.UTF-8 systemd.unified_cgroup_hierarchy=0
+ 	fdtdir /dtb-5.4.17-200.fc31.armv7hl/
+ 	initrd /initramfs-5.4.17-200.fc31.armv7hl.img
+
+ label Fedora-Minimal-armhfp-31-1.9 (5.3.7-301.fc31.armv7hl)
+ 	kernel /vmlinuz-5.3.7-301.fc31.armv7hl
+-	append ro root=UUID=b0abeeb4-7c58-4b29-9c60-3377437cde8a rhgb quiet LANG=en_US.UTF-8
++	append ro root=UUID=b0abeeb4-7c58-4b29-9c60-3377437cde8a rhgb quiet LANG=en_US.UTF-8 systemd.unified_cgroup_hierarchy=0
+ 	fdtdir /dtb-5.3.7-301.fc31.armv7hl/
+ 	initrd /initramfs-5.3.7-301.fc31.armv7hl.img
+```
+
 Reboot the system.
 
 `systemctl reboot`
@@ -222,6 +243,8 @@ spec:
           memory: "50Mi"
 ```
 
+**Warning** Note that this container image is x86-64 only. Attempting to start it on a non-x86-64 platform will likely result in the container crashing (`exec format error`). If you are running on an ARM 32/64 bit platform, feel free to use `docker.io/dmesser/echoserver:1.10`. 
+
 Verify that the pod is running:
 
 `crictl ps -o table`

diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -11,7 +11,6 @@ Make sure the system is up to date:
 On Fedora 31 [by default cgroup v2](https://www.redhat.com/sysadmin/fedora-31-control-group-v2) are used. However the kubelet does not seem to be compatible with this as of time of writing. Revert this by adding the `systemd.unified_cgroup_hierarchy=0` switch to the kernel command line in `/etc/default/grub` like below:
 
 ```diff
-@@ -3,6 +3,6 @@
  GRUB_DEFAULT=saved
  GRUB_DISABLE_SUBMENU=true
  GRUB_TERMINAL_OUTPUT="console"
@@ -112,9 +111,6 @@ We need to reconfigure the Kubelet systemd unit to not require docker but crio.
 Modify the unit file `/etc/systemd/system/kubelet.service` and replace the references to the docker unit with crio:
 
 ```diff
---- /usr/lib/systemd/system/kubelet.service	2020-02-02 00:23:32.319835673 +0100
-+++ /etc/systemd/system/kubelet.service	2020-02-02 17:27:01.368731070 +0100
-@@ -1,8 +1,8 @@
  [Unit]
  Description=Kubernetes Kubelet Server
  Documentation=https://github.com/GoogleCloudPlatform/kubernetes
@@ -136,7 +132,6 @@ Notify systemd about these changes:
 Remove the now obsolete flag `--allow-privileged` from `/etc/kubernetes/config` that would prevent the service from starting:
 
 ```diff
-@@ -16,7 +16,7 @@
  KUBE_LOG_LEVEL="--v=0"
 
  # Should this cluster be allowed to run privileged docker containers
@@ -154,7 +149,6 @@ Apply the below modifications to `/etc/kubernetes/kubelet` in order to:
 
 
 ```diff
-@@ -11,4 +11,4 @@
  KUBELET_HOSTNAME="--hostname-override=127.0.0.1"
 
  # Add your own!

diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -8,7 +8,7 @@ Make sure the system is up to date:
 
 `dnf -y update`
 
-On Fedora 31 [by default cgroup v2](https://www.redhat.com/sysadmin/fedora-31-control-group-v2) are used. However the kubelet does not seem to be compatible with this as of time of writing. Revert this by adding the following entry to `/etc/default/grub`:
+On Fedora 31 [by default cgroup v2](https://www.redhat.com/sysadmin/fedora-31-control-group-v2) are used. However the kubelet does not seem to be compatible with this as of time of writing. Revert this by adding the `systemd.unified_cgroup_hierarchy=0` switch to the kernel command line in `/etc/default/grub` like below:
 
 ```diff
 @@ -3,6 +3,6 @@
@@ -27,7 +27,7 @@ on a BIOS-based system install `grub2-mkconfig -o /boot/grub2/grub.cfg`
 
 or on UEFI-based installs with `grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg`
 
-Reboot the system to boot the kernel with the added command line switch `systemd.unified_cgroup_hierarchy=0`
+Reboot the system.
 
 `systemctl reboot`
 

diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -2,6 +2,35 @@
 
 This gist describes a possible IoT/Edge computing setup using Kubernetes-style declarative management. It utilizes standalone kubelet + CRI-O + CNI on Fedora Core 31. The goal is to place a Kubernetes Pod manifest on an single node and access the application from the network.
 
+### Prepare the system
+
+Make sure the system is up to date:
+
+`dnf -y update`
+
+On Fedora 31 [by default cgroup v2](https://www.redhat.com/sysadmin/fedora-31-control-group-v2) are used. However the kubelet does not seem to be compatible with this as of time of writing. Revert this by adding the following entry to `/etc/default/grub`:
+
+```diff
+@@ -3,6 +3,6 @@
+ GRUB_DEFAULT=saved
+ GRUB_DISABLE_SUBMENU=true
+ GRUB_TERMINAL_OUTPUT="console"
+-GRUB_CMDLINE_LINUX="resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet"
++GRUB_CMDLINE_LINUX="resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet systemd.unified_cgroup_hierarchy=0"
+ GRUB_DISABLE_RECOVERY="true"
+ GRUB_ENABLE_BLSCFG=true
+```
+
+Regenerate your bootloader configuration...
+
+on a BIOS-based system install `grub2-mkconfig -o /boot/grub2/grub.cfg`
+
+or on UEFI-based installs with `grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg`
+
+Reboot the system to boot the kernel with the added command line switch `systemd.unified_cgroup_hierarchy=0`
+
+`systemctl reboot`
+
 ### Install CRI-O
 
 CRI-O is available as a DNF module stream. On FC31 be sure to install the latest version to circumvent this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1754170

diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -1,6 +1,6 @@
 ## What is this about?
 
-This gist describes a possible IoT/Edge computing setup compatible with Kubernetes. It utilizes standalone kubelet + CRI-O + CNI.
+This gist describes a possible IoT/Edge computing setup using Kubernetes-style declarative management. It utilizes standalone kubelet + CRI-O + CNI on Fedora Core 31. The goal is to place a Kubernetes Pod manifest on an single node and access the application from the network.
 
 ### Install CRI-O
 

diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -11,6 +11,58 @@ dnf -y module enable cri-o:1.16
 dnf -y install crio
 ```
 
+To interact with the CRI-O environment also install `critctl`
+
+`dnf -y install cri-tools`
+
+### Configure CNI
+
+CNI will be responsible for connection containers launched through CRI-O to the hosts network. Portmapping (to enable use of `hostPort` in pod manifest) and firewall (to configure firewalld to permit traffic) will be used.
+
+Delete the default CNI config:
+
+```
+rm /etc/cni/net.d/*.conf
+```
+
+Place the following file into `/etc/cni/net.d/100-crio-bridge.conflist`
+
+```json
+{
+  "cniVersion": "0.4.0",
+  "name": "bridge-firewalld",
+  "plugins": [
+    {
+      "type": "bridge",
+      "bridge": "cni0",
+      "isDefaultGateway": true,
+      "isGateway": true,
+      "ipMasq": true,
+      "ipam": {
+        "type": "host-local",
+        "subnet": "10.88.0.0/16",
+        "routes": [
+          {
+            "dst": "0.0.0.0/0"
+          }
+        ]
+      }
+    },
+    { 
+      "type": "portmap",
+      "capabilities": {
+	"portMappings": true
+     }
+    },
+    {
+      "type": "firewall"
+    }
+  ]
+}
+```
+
+The above configuration will cause CNI to create Linux bridge `cni0` and attach veth-pairs between the host and the container. The containers will receive IPs in the `10.88.0.0/16` in the process. The bridge will act as gateway and IP masquerading will be configured to allow containers to networks external to the host (e.g. internet). Portmapping and firewalld rule manipulation will be conducted.
+
 ### Install Standalone kubelet
 
 `dnf -y install kubernetes-node`
@@ -91,4 +143,99 @@ Finally, enable and start the Kubelet.
 
 Verify the kubelet status
 
-`systemctl status kubelet`
+`systemctl status kubelet`
+
+Verify the status of CRI-O
+
+`systemctl status crio`
+
+Verify that both the runtime and CNI are ready:
+
+`crictl info`
+
+You should see the following:
+
+```json
+{
+  "status": {
+    "conditions": [
+      {
+        "type": "RuntimeReady",
+        "status": true,
+        "reason": "",
+        "message": ""
+      },
+      {
+        "type": "NetworkReady",
+        "status": true,
+        "reason": "",
+        "message": ""
+      }
+    ]
+  }
+}
+```
+
+
+When succcessful, place the following example pod manifest in `/etc/kubernetes/manifests/echoserver.yaml`
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: echoserver
+spec:
+  containers:
+    - name: echoserver
+      image: gcr.io/google-containers/echoserver:1.10
+      ports:
+        - name: web
+          containerPort: 8080
+          hostPort: 9091
+          protocol: TCP
+      resources:
+        limits:
+          cpu: "100m"
+          memory: "50Mi"
+```
+
+Verify that the pod is running:
+
+`crictl ps -o table`
+
+You should the pod running:
+
+```
+CONTAINER           IMAGE                                                              CREATED             STATE               NAME                ATTEMPT             POD ID
+99cb17ca96800       365ec60129c5426b4cf160257c06f6ad062c709e0576c8b3d9a5dcc488f5252d   11 minutes ago      Running             echoserver          2                   adfbfd4a31754
+```
+
+Verify  that the firewall CNI plugin has created a rule to allow traffic to the container:
+
+`firewall-cmd --info-zone=trusted`
+
+You should see the IP address of the container/pod in the source list of the `trusted` zone:
+
+```
+trusted (active)
+  target: ACCEPT
+  icmp-block-inversion: no
+  interfaces: 
+  sources: 10.88.0.18/32
+  services: 
+  ports: 
+  protocols: 
+  masquerade: no
+  forward-ports: 
+  source-ports: 
+  icmp-blocks: 
+  rich rules: 
+```
+
+You should be able to `curl` the container on its container port:
+
+`curl http://10.88.0.18:8080`
+
+You should also be able to `curl` the container on its host port from another system:
+
+`curl http://<host-ip>:9091`
diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -1,37 +1,58 @@
-### enable cgroups v1
+## What is this about?
 
-(FC31 specific) tell the kernel to use cgroupsv1 as per https://bugzilla.redhat.com/show_bug.cgi?id=1746355 and 
-append `systemd.unified_cgroup_hierarchy=0` to GRUB_CMD_LINUX variable in /etc/sysconfig/grub
+This gist describes a possible IoT/Edge computing setup compatible with Kubernetes. It utilizes standalone kubelet + CRI-O + CNI.
+
+### Install CRI-O
+
+CRI-O is available as a DNF module stream. On FC31 be sure to install the latest version to circumvent this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1754170
 
-```diff
-@@ -16,7 +16,7 @@
- KUBE_LOG_LEVEL="--v=0"
-
- # Should this cluster be allowed to run privileged docker containers
--KUBE_ALLOW_PRIV="--allow-privileged=false"
-+KUBE_ALLOW_PRIV="--allow-privileged=true"
-
- # How the controller-manager, scheduler, and proxy find the apiserver
- KUBE_MASTER="--master=http://127.0.0.1:8080"
-```
 ```
-grub2-mkconfig -o /boot/grub2/grub.cfg
-systemctl reboot
+dnf -y module enable cri-o:1.16
+dnf -y install crio
 ```
 
-### install kubelet
+### Install Standalone kubelet
 
 `dnf -y install kubernetes-node`
 
-### enable and start docker
 
-`systemctl enable docker --now`
+### Configure Standalone Kubelet
+
+#### Disable Docker
+
+You will notice that containerd and docker have been pulled in as a dependency. This is because by default the kubelet depends on Docker as the runtime. We will disable it and replace with CRI-O.
 
-### verify docker service is started
+`systemctl mask docker`
 
-`systemctl status docker`
+We need to reconfigure the Kubelet systemd unit to not require docker but crio. Copy the unit file into the drop-in location:
 
-### remove the missing flag `--allow-privileged` from `/etc/kubernetes/config`
+`cp /usr/lib/systemd/system/kubelet.service /etc/systemd/system/kubelet.service`
+
+Modify the unit file `/etc/systemd/system/kubelet.service` and replace the references to the docker unit with crio:
+
+```diff
+--- /usr/lib/systemd/system/kubelet.service	2020-02-02 00:23:32.319835673 +0100
++++ /etc/systemd/system/kubelet.service	2020-02-02 17:27:01.368731070 +0100
+@@ -1,8 +1,8 @@
+ [Unit]
+ Description=Kubernetes Kubelet Server
+ Documentation=https://github.com/GoogleCloudPlatform/kubernetes
+-After=docker.service
+-Requires=docker.service
++After=crio.service
++Requires=crio.service
+
+ [Service]
+ WorkingDirectory=/var/lib/kubelet
+```
+
+Notify systemd about these changes:
+
+`systemctl daemon-reload`
+
+#### Configure Kubelet
+
+Remove the now obsolete flag `--allow-privileged` from `/etc/kubernetes/config` that would prevent the service from starting:
 
 ```diff
 @@ -16,7 +16,7 @@
@@ -45,26 +66,29 @@ systemctl reboot
  KUBE_MASTER="--master=http://127.0.0.1:8080"
 ```
 
-### set the pod manifest path in `/etc/kubernetes/kubelet`
+Apply the below modifications to `/etc/kubernetes/kubelet` in order to:
+
+* enable static pod manifests stored on disk
+* enable the use of runc through CRIO as the container runtime ([details](https://github.com/cri-o/cri-o/blob/master/tutorials/kubernetes.md#preparing-kubelet))
+
 
 ```diff
 @@ -11,4 +11,4 @@
  KUBELET_HOSTNAME="--hostname-override=127.0.0.1"
 
  # Add your own!
 -KUBELET_ARGS="--cgroup-driver=systemd --fail-swap-on=false"
-+KUBELET_ARGS="--cgroup-driver=systemd --fail-swap-on=false --pod-manifest-path=/etc/kubernetes/manifests"
++KUBELET_ARGS="--cgroup-driver=systemd --fail-swap-on=false --pod-manifest-path=/etc/kubernetes/manifests --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --runtime-request-timeout=10m"
 ```
 
-### create the manifest directory
+Create the manifest directory:
 
 `mkdir /etc/kubernetes/manifests`
 
-### enable and start kubelet
+Finally, enable and start the Kubelet.
 
 `systemctl enable kubelet --now`
 
-
-### verify kubelet status
+Verify the kubelet status
 
 `systemctl status kubelet`
diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -31,7 +31,7 @@ systemctl reboot
 
 `systemctl status docker`
 
-### remove the missing flag `--allow-privileged` from /etc/kubernetes/config
+### remove the missing flag `--allow-privileged` from `/etc/kubernetes/config`
 
 ```diff
 @@ -16,7 +16,7 @@
@@ -45,7 +45,7 @@ systemctl reboot
  KUBE_MASTER="--master=http://127.0.0.1:8080"
 ```
 
-### set the pod manifest path
+### set the pod manifest path in `/etc/kubernetes/kubelet`
 
 ```diff
 @@ -11,4 +11,4 @@
@@ -62,4 +62,9 @@ systemctl reboot
 
 ### enable and start kubelet
 
-`systemctl enable kubelet --now`
+`systemctl enable kubelet --now`
+
+
+### verify kubelet status
+
+`systemctl status kubelet`
diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -14,8 +14,10 @@ append `systemd.unified_cgroup_hierarchy=0` to GRUB_CMD_LINUX variable in /etc/s
  # How the controller-manager, scheduler, and proxy find the apiserver
  KUBE_MASTER="--master=http://127.0.0.1:8080"
 ```
-`grub2-mkconfig -o /boot/grub2/grub.cfg
-`systemctl reboot`
+```
+grub2-mkconfig -o /boot/grub2/grub.cfg
+systemctl reboot
+```
 
 ### install kubelet
 

diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -14,8 +14,7 @@ append `systemd.unified_cgroup_hierarchy=0` to GRUB_CMD_LINUX variable in /etc/s
  # How the controller-manager, scheduler, and proxy find the apiserver
  KUBE_MASTER="--master=http://127.0.0.1:8080"
 ```
-`grub2-mkconfig -o /boot/grub2/grub.cfg`
-
+`grub2-mkconfig -o /boot/grub2/grub.cfg
 `systemctl reboot`
 
 ### install kubelet

diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -15,6 +15,7 @@ append `systemd.unified_cgroup_hierarchy=0` to GRUB_CMD_LINUX variable in /etc/s
  KUBE_MASTER="--master=http://127.0.0.1:8080"
 ```
 `grub2-mkconfig -o /boot/grub2/grub.cfg`
+
 `systemctl reboot`
 
 ### install kubelet

diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -1,5 +1,8 @@
+### enable cgroups v1
+
 (FC31 specific) tell the kernel to use cgroupsv1 as per https://bugzilla.redhat.com/show_bug.cgi?id=1746355 and 
 append `systemd.unified_cgroup_hierarchy=0` to GRUB_CMD_LINUX variable in /etc/sysconfig/grub
+
 ```diff
 @@ -16,7 +16,7 @@
  KUBE_LOG_LEVEL="--v=0"
@@ -14,16 +17,20 @@ append `systemd.unified_cgroup_hierarchy=0` to GRUB_CMD_LINUX variable in /etc/s
 `grub2-mkconfig -o /boot/grub2/grub.cfg`
 `systemctl reboot`
 
-install kubelet
+### install kubelet
+
 `dnf -y install kubernetes-node`
 
-enable and start docker
+### enable and start docker
+
 `systemctl enable docker --now`
 
-verify docker service is tarted
+### verify docker service is started
+
 `systemctl status docker`
 
-remove the missing flag `--allow-privileged` from /etc/kubernetes/config
+### remove the missing flag `--allow-privileged` from /etc/kubernetes/config
+
 ```diff
 @@ -16,7 +16,7 @@
  KUBE_LOG_LEVEL="--v=0"
@@ -36,7 +43,8 @@ remove the missing flag `--allow-privileged` from /etc/kubernetes/config
  KUBE_MASTER="--master=http://127.0.0.1:8080"
 ```
 
-set the pod manifest path to /etc/kubernetes/manifests
+### set the pod manifest path
+
 ```diff
 @@ -11,4 +11,4 @@
  KUBELET_HOSTNAME="--hostname-override=127.0.0.1"
@@ -46,8 +54,10 @@ set the pod manifest path to /etc/kubernetes/manifests
 +KUBELET_ARGS="--cgroup-driver=systemd --fail-swap-on=false --pod-manifest-path=/etc/kubernetes/manifests"
 ```
 
-create the manifest directory
+### create the manifest directory
+
 `mkdir /etc/kubernetes/manifests`
 
-enable and start kubelet
+### enable and start kubelet
+
 `systemctl enable kubelet --now`
diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -25,8 +25,6 @@ verify docker service is tarted
 
 remove the missing flag `--allow-privileged` from /etc/kubernetes/config
 ```diff
---- a/config	2020-02-02 00:28:55.668625357 +0100
-+++ b/config	2020-02-02 00:29:16.048881141 +0100
 @@ -16,7 +16,7 @@
  KUBE_LOG_LEVEL="--v=0"
 
@@ -40,8 +38,6 @@ remove the missing flag `--allow-privileged` from /etc/kubernetes/config
 
 set the pod manifest path to /etc/kubernetes/manifests
 ```diff
---- a/kubelet	2020-02-02 00:16:14.130763383 +0100
-+++ b/kubelet	2020-02-02 00:16:05.185553220 +0100
 @@ -11,4 +11,4 @@
  KUBELET_HOSTNAME="--hostname-override=127.0.0.1"
 

diff --git a/standalone-kubelet-fc31.md b/standalone-kubelet-fc31.md
@@ -0,0 +1,57 @@
+(FC31 specific) tell the kernel to use cgroupsv1 as per https://bugzilla.redhat.com/show_bug.cgi?id=1746355 and 
+append `systemd.unified_cgroup_hierarchy=0` to GRUB_CMD_LINUX variable in /etc/sysconfig/grub
+```diff
+@@ -16,7 +16,7 @@
+ KUBE_LOG_LEVEL="--v=0"
+
+ # Should this cluster be allowed to run privileged docker containers
+-KUBE_ALLOW_PRIV="--allow-privileged=false"
++KUBE_ALLOW_PRIV="--allow-privileged=true"
+
+ # How the controller-manager, scheduler, and proxy find the apiserver
+ KUBE_MASTER="--master=http://127.0.0.1:8080"
+```
+`grub2-mkconfig -o /boot/grub2/grub.cfg`
+`systemctl reboot`
+
+install kubelet
+`dnf -y install kubernetes-node`
+
+enable and start docker
+`systemctl enable docker --now`
+
+verify docker service is tarted
+`systemctl status docker`
+
+remove the missing flag `--allow-privileged` from /etc/kubernetes/config
+```diff
+--- a/config	2020-02-02 00:28:55.668625357 +0100
++++ b/config	2020-02-02 00:29:16.048881141 +0100
+@@ -16,7 +16,7 @@
+ KUBE_LOG_LEVEL="--v=0"
+
+ # Should this cluster be allowed to run privileged docker containers
+-KUBE_ALLOW_PRIV="--allow-privileged=true"
++KUBE_ALLOW_PRIV=""
+
+ # How the controller-manager, scheduler, and proxy find the apiserver
+ KUBE_MASTER="--master=http://127.0.0.1:8080"
+```
+
+set the pod manifest path to /etc/kubernetes/manifests
+```diff
+--- a/kubelet	2020-02-02 00:16:14.130763383 +0100
++++ b/kubelet	2020-02-02 00:16:05.185553220 +0100
+@@ -11,4 +11,4 @@
+ KUBELET_HOSTNAME="--hostname-override=127.0.0.1"
+
+ # Add your own!
+-KUBELET_ARGS="--cgroup-driver=systemd --fail-swap-on=false"
++KUBELET_ARGS="--cgroup-driver=systemd --fail-swap-on=false --pod-manifest-path=/etc/kubernetes/manifests"
+```
+
+create the manifest directory
+`mkdir /etc/kubernetes/manifests`
+
+enable and start kubelet
+`systemctl enable kubelet --now`
No results found