Skip to content

Instantly share code, notes, and snippets.

@scottrigby
Last active September 3, 2025 21:05
Show Gist options
  • Save scottrigby/a1a42c3292ec7899837c578ffdaaf92a to your computer and use it in GitHub Desktop.
Save scottrigby/a1a42c3292ec7899837c578ffdaaf92a to your computer and use it in GitHub Desktop.
GitOps for Helm Users

GitOps for Helm Users

  1. Install Flux CLI and Kind:

    $ brew upgrade fluxcd/tap/flux kind
    $ brew reinstall fluxcd/tap/flux kind
    $ flux --version && kind --version
    flux version 0.27.0
    kind version 0.11.1
  2. Make Personal Access Token for creating repositories

    1. Generate new token in dev settings
    2. Check all permissions under repo & save
    3. Copy PAT to buffer
  3. Export env vars locally

    I've done this in advance for now.

    πŸ’‘ If you want to show during a demo, follow best security practices by making the PAT off camera - or copy from a secure password app on camera etc – then read to var silently read -s, then export var.

    $ export GITHUB_TOKEN=[paste PAT]
    $ echo $GITHUB_TOKEN | wc -c
      41
  4. Create local demo cluster

    $ kind create cluster
    (took 40s)
  5. Simple bootstrap:

    πŸ’‘ The more complex your org is, the more complex your directory structure and patterns usually are.

    There is no gold standard.

    Flux is not opinionated about how directories are structured, rather it tries to be as flexible as possible to accommodate different patterns.

    $ flux bootstrap github \
      --interval 10s \
      --owner scottrigby --personal \
      --repository flux-for-helm-users \
      --branch main \
      --path=clusters/dev
    β–Ί connecting to github.com
    βœ” repository "https://github.com/scottrigby/flux-for-helm-users" created
    β–Ί cloning branch "main" from Git repository "https://github.com/scottrigby/flux-for-helm-users.git"
    βœ” cloned repository
    β–Ί generating component manifests
    βœ” generated component manifests
    βœ” committed sync manifests to "main" ("42a5e71e792cf3ca0393fefea4c4375e72d9fc47")
    β–Ί pushing component manifests to "https://github.com/scottrigby/flux-for-helm-users.git"
    βœ” installed components
    βœ” reconciled components
    β–Ί determining if source secret "flux-system/flux-system" exists
    β–Ί generating source secret
    βœ” public key: ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMbDpSb+E912hnXZWX/x9RFWPscqsTJ/8bzgYLgYEywpkWwQNZVCjdvhLiNEexXMqk5IO3JxF9ScAa76IB6kYRFZ8WlGwoBNINU2HcXmtJF/9LZgUKzF53ioK9esCO+rYw==
    βœ” configured deploy key "flux-system-main-flux-system-./clusters/dev" for "https://github.com/scottrigby/flux-for-helm-users"
    β–Ί applying source secret "flux-system/flux-system"
    βœ” reconciled source secret
    β–Ί generating sync manifests
    βœ” generated sync manifests
    βœ” committed sync manifests to "main" ("055e5edfbace022504101c763b65b1f7c2134187")
    β–Ί pushing sync manifests to "https://github.com/scottrigby/flux-for-helm-users.git"
    β–Ί applying sync manifests
    βœ” reconciled sync configuration
    β—Ž waiting for Kustomization "flux-system/flux-system" to be reconciled
    βœ” Kustomization reconciled successfully
    β–Ί confirming components are healthy
    βœ” helm-controller: deployment ready
    βœ” kustomize-controller: deployment ready
    βœ” notification-controller: deployment ready
    βœ” source-controller: deployment ready
    βœ” all components are healthy
    (took 1m3s)
    
  6. Clone the newly created git repo to your local workspace

    $ cd ~/code/github.com/scottrigby \
      && git clone [email protected]:scottrigby/flux-for-helm-users.git \
      && cd flux-for-helm-users
    $ tree
    .
    └── clusters
        └── dev
            └── flux-system
                β”œβ”€β”€ gotk-components.yaml
                β”œβ”€β”€ gotk-sync.yaml
                └── kustomization.yaml
    
    3 directories, 3 files
  7. Lets create a Helm release the most common way, using the Helm CLI

    Remember that we set custom values. We will get back to this later.

    helm repo add podinfo https://stefanprodan.github.io/podinfo

    Lets set some values to make this fun.

    πŸ’‘ Helm CLI is great to show all the available options in a chart:

    $ helm show values podinfo --repo https://stefanprodan.github.io/podinfo
    # Default values for podinfo.
    
    replicaCount: 1
    logLevel: info
    
    ui:
      color: "#34577c"
      message: ""
      logo: ""
    
    etc…
    $ helm upgrade -i my-release podinfo/podinfo \
      --set replicaCount=2 \
      --set logLevel=debug \
      --set ui.color='red'
    Release "my-release" does not exist. Installing it now.
    …
  8. Now lets convert these to declarative CRs that Flux understands

    Create a Source Custom Resource locally

    πŸ’‘ The Helm CLI reads your locally defined Helm repo info (created in step 7). But the Flux Helm controller in your cluster will also need this same info.

    We'll tell Flux about the Helm repo info with a HelmRepository CR representing a Flux source. Instead of helm add repo you can use flux create source helm to export the CRD to a local file:

    $ flux create source helm podinfo \
      --url=https://stefanprodan.github.io/podinfo \
      --namespace=default \
      --export > clusters/dev/source-helmrepo-podinfo.yaml
    $ cat clusters/dev/source-helmrepo-podinfo.yaml
    ---
    apiVersion: source.toolkit.fluxcd.io/v1beta1
    kind: HelmRepository
    metadata:
      name: podinfo
      namespace: default
    spec:
      interval: 1m0s
      url: https://stefanprodan.github.io/podinfo

    Next we'll create a HelmRelease Custom Resource locally, using the same Helm values we earlier specified with the Helm CLI.

    πŸ’‘ Helm CLI makes it very easy to get the values we earlier set for the release. We'll first export these to a file then take a look at its contents:

    $ helm get values my-release -oyaml > my-values.yaml
    $ cat my-values.yaml
    logLevel: debug
    replicaCount: 2
    ui:
      color: red

    And again Flux CLI makes it easy to create the CR. You may also do this by hand, or with an IDE (for example with the VSCode Flux plugin), but the CLI command eases this:

    $ flux create helmrelease my-release \
      --release-name=my-release \
      --source=HelmRepository/podinfo \
      --chart=podinfo \
      --chart-version=">4.0.0" \
      --namespace=default \
      --values my-values.yaml \
      --export > ./clusters/dev/podinfo-helmrelease.yaml
    $ cat clusters/dev/podinfo-helmrelease.yaml
    ---
    apiVersion: helm.toolkit.fluxcd.io/v2beta1
    kind: HelmRelease
    metadata:
      name: my-release
      namespace: default
    spec:
      chart:
        spec:
          chart: podinfo
          sourceRef:
            kind: HelmRepository
            name: podinfo
          version: '>4.0.0'
      interval: 1m0s
      values:
        logLevel: debug
        replicaCount: 2
        ui:
          color: red

    We now no longer need the temporary values file. Lets be tidy:

    rm my-values.yaml
  9. Lets go ahead and push this to Git

    $ git add .
    $ git commit -m 'Configure podinfo Helm Repo source and Helm Release'
    $ git push
    …

    πŸ’‘ From this point on, you are now doing GitOps.

  10. Lets check out the magic

    We can verify that Flux is now managing this Helm release.

    πŸ’‘ If you want to immediately trigger reconciliation on a local demo cluster you can manually call flux reconcile. We shouldn't need to trigger that manually in this demo because we set the interval to 10s.

    In real word clusters there are important use cases for setting up webhook receivers to automate this immediacy, and there are equally important use cases for letting your defined sync interval run its course.

    flux reconcile helmrelease my-release

    Flux will add labels

    $ kubectl get deploy my-release-podinfo -oyaml | grep flux
    helm.toolkit.fluxcd.io/name: my-release
    helm.toolkit.fluxcd.io/namespace: default
  11. Change a new Helm release value through Git

    You believe me that we are now doing GitOps, but let's prove it.

    Change a value in your HelmRelease CR:

    $ yq -i '.spec.values.ui.color = "blue"' clusters/dev/podinfo-helmrelease.yaml
    $ git add clusters/dev
    $ git diff --staged
    diff --git a/clusters/dev/podinfo-helmrelease.yaml b/clusters/dev/podinfo-helmrelease.yaml
    index b58eed2..5e1dc10 100644
    --- a/clusters/dev/podinfo-helmrelease.yaml
    +++ b/clusters/dev/podinfo-helmrelease.yaml
    @@ -17,5 +17,4 @@ spec:
        logLevel: debug
        replicaCount: 2
        ui:
    -      color: red
    +      color: blue
    $ git commit -m "blue me"
    $ git push

    We can see our Helm release incremented the revision:

    $ helm list
    NAME      	NAMESPACE	REVISION	UPDATED                              	STATUS  	CHART        	APP VERSION
    my-release	default  	3       	2022-02-17 06:16:42.2293519 +0000 UTC	deployed	podinfo-6.0.3	6.0.3 

    And that the new release revision applied our change:

    $ helm diff revision my-release 2 3
              env:
              - name: PODINFO_UI_COLOR
    -           value: red
    +           value: blue
              image: ghcr.io/stefanprodan/podinfo:6.0.3
              imagePullPolicy: IfNotPresent

    Let's get visual:

    $ kubectl -n default port-forward deploy/my-release-podinfo 8080:9898
    Forwarding from 127.0.0.1:8080 -> 9898

    Browse to http://localhost:8080

    Er mah gerd, it's blue!

    Let's pretend we're in incident management and want to use Helm rollback

  12. Pause and resume

    πŸ’‘ It's worth noting that Flux HelmRelease retains Helm release metadata and Helm's ability to manage the releases directly.

    There are various benefits to this, including the ability to continue using your favorite development tools that integrate with Helm releases (such as helm list, helm diff plugin, etc).

    This is also helpful in production. For example, there are legitimate use cases for pausing GitOps operations and temporarily using the Helm CLI, such as incident management. Pausing and resuming GitOps reconciliation may be done on a per Custom Resource basis without affecting the others, for example a single HelmRelease:

    $ flux suspend helmrelease my-release --namespace default
    β–Ί suspending helmreleases my-release in default namespace
    βœ” helmreleases suspended

    Let's rollback to red using the Helm CLI, to show that it works.

    $ helm rollback my-release 2
    Rollback was a success! Happy Helming!

    We can port forward again see that it worked:

    $ kubectl -n default port-forward deploy/my-release-podinfo 8080:9898
    Forwarding from 127.0.0.1:8080 -> 9898

    OK yay, back to red! πŸ˜…

    Once we're finished with our incident management window and want to resume GitOps reconciliation on that resource, we just need to resume again:

    $ flux resume helmrelease my-release --namespace default
    β–Ί resuming helmreleases my-release in default namespace
    βœ” helmreleases resumed
    β—Ž waiting for HelmRelease reconciliation
    βœ” HelmRelease reconciliation completed
    βœ” applied revision 6.0.3

    Port forward again, and take a look.

    Back to blue as planned.

  13. Cleanup demo cluster 🧹

    kind delete cluster

    Barney clen up video still

    And if you wish, feel free to delete your demo GitHub repo.

    πŸ’‘ Or not! These commands are idempotent, so you can feel free to keep your repo. In fact…

  14. But wait, there's more!

    Want to see how Flux handles your Helm release in a disaster recovery scenario?

    Let's simulate total cluster failure by just deleting it 😡:

    kind delete cluster

    We can create a new one by repeating step 4 (kind create cluster).

    Then just need to install Flux components into the new cluster by repeating step 5 (flux bootstrap command).

    πŸ’‘ Because we still have our desired state defined in the Git repo we specify in flux bootstrap, reconciliation will happen automatically. Our Helm release should now match what we've defined in Git, as the source of truth!

    πŸ” You'll notice the Helm metadata revision is back to 1, because that is only useful as in-cluster storage. New cluster, revisions start anew.

    $ helm list
    NAME      	NAMESPACE	REVISION	UPDATED                              	STATUS  	CHART        	APP VERSION
    my-release	default  	1       	2022-02-17 06:56:15.5594991 +0000 UTC	deployed	podinfo-6.0.3	6.0.3 
  15. Wrap up

    And there we have it folks!

    • On a local kind cluster, we simulated an existing Helm release using the Helm CLI you're already familiar with (helm install)
    • We used Flux CLI to bootstrap Flux components into the cluster, and simultaneously define and create (if it didn't already exist) a properly formatted Git repo containing the bootstrap manifests
    • Used Flux CLI to help us easily export our Helm release values and create Custom Resources for the Helm repo and release, along with our custom values
    • Push the files to Git, and show Flux taking ownership of managing your existing Helm release
    • Prove this by making changes to Git only, and watch Flux magically update your Helm release
    • Simulate incident management, by showing how to pause and resume the automated continuous reconciliation on a single HelmRelease
    • Simulate disaster recovery of your Helm release by deleting your cluster. Bootstrapping Flux again is all that's required to get your system and apps running again
@ganadurai
Copy link

Step 10, the command needs the namespace identifier
flux reconcile helmrelease $FLUX_HELM_RELEASE_RESOURCE -n default

Thanks.

@desiby
Copy link

desiby commented Apr 8, 2023

Step 10, the command needs the namespace identifier flux reconcile helmrelease $FLUX_HELM_RELEASE_RESOURCE -n default

Thanks.

yes or put the helm resources in the "flux-system" namespace to make command in step 10 work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment