Skip to content

Instantly share code, notes, and snippets.

@wrkode

wrkode/README.md

Forked from dgiebert/README.md
Created July 5, 2024 13:06
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save wrkode/b207dabba65ee741ae709897a70c1107 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save wrkode/b207dabba65ee741ae709897a70c1107 to your computer and use it in GitHub Desktop.
Download ZIP
Cilium Cluster Mesh RKE2
Raw
README.md

HelmChartConfig

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-cilium
  namespace: kube-system
spec:
  valuesContent: |-
    kubeProxyReplacement: strict
    k8sServiceHost: 127.0.0.1
    k8sServicePort: 6443
    ipv4NativeRoutingCIDR: 10.0.0.0/8
    # Transparent Encryption
    encryption:
      enabled: true
      type: wireguard
    # Cluster-mesh
    # cluster:
    #   name: cilium01
    #   id: 1

Use the CLI to connect 2 clusters

export CLUSTER1=cilium01 CLUSTER2=cilium02
cilium --helm-release-name rke2-cilium clustermesh enable --context $CLUSTER1 --service-type LoadBalancer
# Fix LoadBalancer in CLUSTER1
kubectl annotate svc -n kube-system clustermesh-apiserver cloudprovider.harvesterhci.io/ipam='dhcp'
# Fix cilium-ca for Hubble
kubectl label secret -n kube-system cilium-ca app.kubernetes.io/managed-by="Helm"
kubectl annotate secret -n kube-system cilium-ca meta.helm.sh/release-name="rke2-cilium"
kubectl annotate secret -n kube-system meta.helm.sh/release-namespace="kube-system"
kubectl --context=$CLUSTER1 get secret -n kube-system cilium-ca -o yaml | kubectl --context $CLUSTER2 create -f -
cilium --helm-release-name rke2-cilium clustermesh enable --context $CLUSTER2 --service-type LoadBalancer
# Fix LoadBalancer in CLUSTER2
kubectl annotate svc -n kube-system clustermesh-apiserver cloudprovider.harvesterhci.io/ipam='dhcp'

# Check Status of the Cluster Mesh Components in each cluster
cilium clustermesh status --context $CLUSTER1 --wait
cilium clustermesh status --context $CLUSTER2 --wait
# Start connecting
cilium --helm-release-name rke2-cilium clustermesh connect --context $CLUSTER1 --destination-context $CLUSTER2
cilium clustermesh status --context $CLUSTER1 --wait
cilium connectivity test --context $CLUSTER1 --multi-cluster $CLUSTER2
helm get values -n kube-system rke2-cilium

Use the CLI to connect 2 clusters

export CLUSTER1=cilium01 CLUSTER2=cilium02 CLUSTER3=cilium03
cilium --helm-release-name rke2-cilium clustermesh enable --context $CLUSTER1 --service-type LoadBalancer
# Fix LoadBalancer in CLUSTER1
kubectl annotate svc -n kube-system clustermesh-apiserver cloudprovider.harvesterhci.io/ipam='dhcp'
# Fix cilium-ca for Hubble
kubectl label secret -n kube-system cilium-ca app.kubernetes.io/managed-by="Helm"
kubectl annotate secret -n kube-system cilium-ca meta.helm.sh/release-name="rke2-cilium"
kubectl annotate secret -n kube-system meta.helm.sh/release-namespace="kube-system"
kubectl --context=$CLUSTER1 get secret -n kube-system cilium-ca -o yaml | kubectl --context $CLUSTER2 create -f -
kubectl --context=$CLUSTER1 get secret -n kube-system cilium-ca -o yaml | kubectl --context $CLUSTER3 create -f -
cilium --helm-release-name rke2-cilium clustermesh enable --context $CLUSTER2 --service-type LoadBalancer
# Fix LoadBalancer in CLUSTER2
cilium --helm-release-name rke2-cilium clustermesh enable --context $CLUSTER3 --service-type LoadBalancer
# Fix LoadBalancer in CLUSTER3
kubectl annotate svc -n kube-system clustermesh-apiserver cloudprovider.harvesterhci.io/ipam='dhcp'

# Check Status of the Cluster Mesh Components in each cluster
cilium clustermesh status --context $CLUSTER1 --wait
cilium clustermesh status --context $CLUSTER2 --wait
cilium clustermesh status --context $CLUSTER3 --wait
# Start connecting
cilium --helm-release-name rke2-cilium clustermesh connect --context $CLUSTER1 --destination-context $CLUSTER2
cilium --helm-release-name rke2-cilium clustermesh connect --context $CLUSTER2 --destination-context $CLUSTER3
cilium --helm-release-name rke2-cilium clustermesh connect --context $CLUSTER3 --destination-context $CLUSTER1
cilium clustermesh status --context $CLUSTER1 --wait
cilium connectivity test --context $CLUSTER1 --multi-cluster $CLUSTER2
# Persist all settings in the HelmChartConfig
helm get values -n kube-system rke2-cilium

Based on: https://docs.cilium.io/en/stable/network/clustermesh/clustermesh/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment