-
-
Save developer-guy/c7e0b3e92cb38f681b7c9b9aa4ab6dae to your computer and use it in GitHub Desktop.
Revisions
-
jimangel revised this gist
Jan 13, 2020 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -6,7 +6,7 @@ If this is the first time you're hearing of these deprecations **STOP** and read ### Common misunderstandings 1) The exact `apiVersion:` value that I use in my manifest is stored on disk (etcd). When you create an object in Kubernetes, the `apiVersion:` value that persists on disk (etcd) will be Kubernetes' _preferred_ version, regardless of the `apiVersion:` value you submit. -
jimangel revised this gist
Jan 13, 2020 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -14,9 +14,9 @@ If this is the first time you're hearing of these deprecations **STOP** and read 2) `kubectl get $OBJECT -o yaml` will print my exact manifest object I created. When you output a manifest, the `apiVersion:` value will be Kubernetes' _preferred_ version. For that reason, we can't use `kubectl output` to search for running deprecated objects. 3) If I upgrade my cluster with running deprecated objects, they will fail. During an upgrade, Kubernetes will convert all `apiVersion:` values to _preferred_ versions. As a result, there is little to no impact to running applications or objects. -
jimangel revised this gist
Jan 13, 2020 . 1 changed file with 3 additions and 6 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -38,20 +38,17 @@ If this is the first time you're hearing of these deprecations **STOP** and read > **IMPORTANT:** Do not use `-o yaml` for checking, as the output it generates will use `kubectl's` preferred API. ``` # NOTE: THIS WILL ONLY FIND OBJECTS THAT WERE CREATED USING kubectl apply (NOT create)! # check all namespace scoped resources for namespace in $(kubectl get ns -o=jsonpath='{.items[*].metadata.name}'); do kubectl apply view-last-applied networkpolicy,ds,deployment,statefulset,rs,ing -n $namespace | conftest test -p deprek8.rego -; done # manually review your PSP source manifests # you can always grep some output for the lack of a preferred API # <some source yaml> | grep -E -v 'apps/v1|networking.k8s.io/v1|policy/v1beta1|networking.k8s.io/v1beta1' ``` ## Configure `kubeadm` to pre-deprecate the API's in earlier versions -
jimangel revised this gist
Jan 13, 2020 . 1 changed file with 36 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -4,6 +4,24 @@ Examples of how to test the impact of the v1.16 API deprecations and ways to deb If this is the first time you're hearing of these deprecations **STOP** and read this [blog post](https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/) (thanks [@vllry](https://github.com/vllry)!). ### Common misunderstandings 1) The exact `apiVersion:` value that I use in my manifest is stored in on disk (etcd). When you create an object in Kubernetes, the `apiVersion:` value that persists on disk (etcd) will be Kubernetes' _preferred_ version, regardless of the `apiVersion:` value you submit. There is one partial exception to this, when using `kubectl apply ...` a `last-applied:` label will persist on disk (etcd) with a value of the entire manifest as applied. 2) `kubectl get $OBJECT -o yaml` will print my exact manifest object I created. When you output a manifest, the `apiVersion:` value will be Kubernetes' _preferred_ version. For that reason, we can't use `kubectl output` to search for running deprecated deployments. 3) If I upgrade my cluster with running deprecated deployments, they will fail. During an upgrade, Kubernetes will convert all `apiVersion:` values to _preferred_ versions. As a result, there is little to no impact to running applications or objects. **TL;DR:** The best way to not get bit by deprecated API's is to scan and test the source manifests. ### Needed before you start: - This binary (conftest): https://github.com/instrumenta/conftest#installation - This file of rego rules (deprek8) @@ -182,6 +200,24 @@ spec: EOF ``` **But wait!** `convert` is being deprecated! Is there another way? ``` # Use a period to append a specific object spec kubectl get deployments test -o yaml ... apiVersion: extensions/v1beta1 kind: Deployment ... kubectl get deployments.apps test -o yaml ... apiVersion: apps/v1 kind: Deployment ... ``` ### Test with KinD If you haven't heard about KinD, start here: https://kind.sigs.k8s.io/ -
jimangel revised this gist
Jan 8, 2020 . 1 changed file with 2 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -17,6 +17,8 @@ If this is the first time you're hearing of these deprecations **STOP** and read ## Check a running cluster for deprecated APIs > **IMPORTANT:** Do not use `-o yaml` for checking, as the output it generates will use `kubectl's` preferred API. ``` # NOTE: THIS IS A WORK IN PROGRESS # PLEASE VALIDATE AT YOUR OWN RISK :) -
jimangel revised this gist
Jan 7, 2020 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -22,7 +22,7 @@ If this is the first time you're hearing of these deprecations **STOP** and read # PLEASE VALIDATE AT YOUR OWN RISK :) # check all namespace scoped resources for namespace in $(kubectl get ns -o=jsonpath='{.items[*].metadata.name}'); do kubectl apply view-last-applied networkpolicy,ds,deployment,statefulset,rs,ing -n $namespace | conftest test -p deprek8.rego -; done -
jimangel revised this gist
Jan 6, 2020 . 1 changed file with 9 additions and 6 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,22 +1,25 @@ ## Kubernetes v1.16 API deprecation testing Examples of how to test the impact of the v1.16 API deprecations and ways to debug early! If this is the first time you're hearing of these deprecations **STOP** and read this [blog post](https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/) (thanks [@vllry](https://github.com/vllry)!). ### Needed before you start: - This binary (conftest): https://github.com/instrumenta/conftest#installation - This file of rego rules (deprek8) ``` curl https://raw.githubusercontent.com/naquada/deprek8/master/policy/deprek8.rego > deprek8.rego # NOTE: If using conftest 0.16.0 (latest), you can provide the url WITHOUT creating a file # conftest test --update https://raw.githubusercontent.com/naquada/deprek8/master/policy/deprek8.rego - ``` ## Check a running cluster for deprecated APIs ``` # NOTE: THIS IS A WORK IN PROGRESS # PLEASE VALIDATE AT YOUR OWN RISK :) # check all namespace scoped resources for namespace in $(kubectl get ns -o=jsonpath='{.items[0].metadata.name}'); do -
jimangel revised this gist
Jan 6, 2020 . 1 changed file with 15 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -14,8 +14,21 @@ If this is the first time you're hearing of these deprecations, **STOP** and rea - This binary (conftest): https://github.com/instrumenta/conftest#installation ## Check a running cluster for using deprecated APIs ``` # NOTE: THIS IS A WORK IN PROGRESS - PLEASE VALIDATE AT YOUR OWN RISK THIS WAY :) # check all namespace scoped resources for namespace in $(kubectl get ns -o=jsonpath='{.items[0].metadata.name}'); do kubectl apply view-last-applied networkpolicy,ds,deployment,statefulset,rs,ing -n $namespace | conftest test -p deprek8.rego -; done # manually check your PSPs w/ # kubectl get psp # kubectl describe psp <name> (review) # you can always grep some output for the lack of a preferred API # <some yaml or kubectl output> | grep -E -v 'apps/v1|networking.k8s.io/v1|policy/v1beta1|networking.k8s.io/v1beta1' ``` ## Configure `kubeadm` to pre-deprecate the API's in earlier versions @@ -86,6 +99,7 @@ spec: name: devnull EOF ``` Bad (should fail) ``` cat <<EOF | kubectl apply -f - -
jimangel created this gist
Jan 6, 2020 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,187 @@ ## Kubernetes v1.16 API deprecation testing Examples of how to test the impact of the v1.16 API deprecations and ways to test and debug early! If this is the first time you're hearing of these deprecations, **STOP** and read this [blog post](https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/) (thanks [@vllry](https://github.com/vllry)!). ### Needed before you start: - This file of rego rules (deprek8) ``` curl https://raw.githubusercontent.com/naquada/deprek8/master/policy/deprek8.rego > deprek8.rego ``` - This binary (conftest): https://github.com/instrumenta/conftest#installation ## Check a running cluster for using deprecated APIs ``` kubectl get networkpolicy,psp,ds,deployment,statefulset,rs,ing -A -o yaml | conftest test -p deprek8.rego - ``` ## Configure `kubeadm` to pre-deprecate the API's in earlier versions ``` apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration metadata: name: config apiServer: extraArgs: runtime-config: "apps/v1beta1=false,apps/v1beta2=false,extensions/v1beta1/daemonsets=false,extensions/v1beta1/deployments=false,extensions/v1beta1/replicasets=false,extensions/v1beta1/networkpolicies=false,extensions/v1beta1/podsecuritypolicies=false" # kubeadm init <...> --config <ClusterConfig>.yaml # validate with: # kubectl exec -it <APISERVER-NAME> -n kube-system | ps -ef | grep runtime-config ``` ### Testing valid and invalid APIs Good (should pass) ``` # generic apps/v1 deployment cat <<EOF | kubectl apply -f - apiVersion: apps/v1 kind: Deployment metadata: creationTimestamp: null labels: run: nginx name: nginx spec: replicas: 1 selector: matchLabels: run: nginx template: metadata: labels: run: nginx spec: containers: - image: devnull name: devnull EOF # Now with more with rego! cat <<EOF | conftest test -p deprek8.rego - apiVersion: apps/v1 kind: Deployment metadata: creationTimestamp: null labels: run: nginx name: nginx spec: replicas: 1 selector: matchLabels: run: nginx template: metadata: labels: run: nginx spec: containers: - image: devnull name: devnull EOF ``` Bad (should fail) ``` cat <<EOF | kubectl apply -f - apiVersion: apps/v1beta2 kind: Deployment metadata: creationTimestamp: null labels: run: nginx name: nginx spec: replicas: 1 selector: matchLabels: run: nginx template: metadata: labels: run: nginx spec: containers: - image: devnull name: devnull EOF # Now with more with rego! cat <<EOF | conftest test -p deprek8.rego - apiVersion: apps/v1beta2 kind: Deployment metadata: creationTimestamp: null labels: run: nginx name: nginx spec: replicas: 1 selector: matchLabels: run: nginx template: metadata: labels: run: nginx spec: containers: - image: devnull name: devnull EOF ``` ### Using `kubectl` to fix a "bad" deployment with `convert` ``` cat <<EOF | kubectl convert -f - | kubectl apply -f - apiVersion: apps/v1beta2 kind: Deployment metadata: creationTimestamp: null labels: run: nginx name: nginx spec: replicas: 1 selector: matchLabels: run: nginx template: metadata: labels: run: nginx spec: containers: - image: devnull name: devnull EOF ``` ### Test with KinD If you haven't heard about KinD, start here: https://kind.sigs.k8s.io/ ``` cat <<EOF > kind-config.yaml kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 kubeadmConfigPatches: - | apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration metadata: name: config apiServer: extraArgs: runtime-config: "apps/v1beta1=false,apps/v1beta2=false,extensions/v1beta1/daemonsets=false,extensions/v1beta1/deployments=false,extensions/v1beta1/replicasets=false,extensions/v1beta1/networkpolicies=false,extensions/v1beta1/podsecuritypolicies=false" EOF # using v1.15.6 since the APIs are gone in v1.16.0 kind create cluster --image=kindest/node:v1.15.6@sha256:18c4ab6b61c991c249d29df778e651f443ac4bcd4e6bdd37e0c83c0d33eaae78 --config kind-config.yaml ```