Skip to content

Instantly share code, notes, and snippets.

@usrbinkat

usrbinkat/01-README.md

Last active February 17, 2025 02:19
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save usrbinkat/f5b712c79c890f9deab0a2dacd5ecd0a to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save usrbinkat/f5b712c79c890f9deab0a2dacd5ecd0a to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. usrbinkat revised this gist Oct 20, 2022. 1 changed file with 148 additions and 32 deletions.
    180 changes: 148 additions & 32 deletions 03-capi-cluster-talos.yaml
    View file
    Original file line number Diff line number Diff line change
    @@ -16,7 +16,7 @@ spec:
    clusterNetwork:
    pods:
    cidrBlocks:
    - 192.168.0.0/16
    - 192.168.3.0/16
    services:
    cidrBlocks:
    - 172.26.0.0/16
    @@ -136,30 +136,88 @@ spec:
    volumes:
    - name: vmdisk
    dataVolume:
    name: vmdisk-dv
    name: vmdisk-cp-dv
    domain:
    cpu:
    cores: 2
    cores: 4
    devices:
    disks:
    - name: vmdisk
    disk:
    bus: scsi
    bus: virtio
    memory:
    guest: "4Gi"
    guest: "6Gi"
    template:
    spec:
    nodeSelector:
    node-role.kubernetes.io/kubevirt: ""
    domain:
    clock:
    utc: {}
    timer: {}
    cpu:
    cores: 1
    sockets: 1
    threads: 4
    model: host-passthrough
    dedicatedCpuPlacement: false
    devices:
    rng: {}
    autoattachPodInterface: false
    autoattachSerialConsole: true
    autoattachGraphicsDevice: true
    networkInterfaceMultiqueue: false
    disks:
    - name: vmdisk
    bootOrder: 1
    disk:
    bus: virtio
    interfaces:
    - name: enp1s0
    model: virtio
    bridge: {}
    machine:
    type: q35
    resources:
    requests:
    memory: 12G
    devices.kubevirt.io/kvm: "1"
    networks:
    - name: enp1s0
    multus:
    networkName: nadbr0
    terminationGracePeriodSeconds: 0
    volumes:
    - name: vmdisk
    dataVolume:
    name: vmdisk-cp-dv
    dataVolumeTemplates:
    - metadata:
    name: vmdisk-dv
    spec:
    pvc:
    accessModes:
    - ReadWriteOnce
    resources:
    requests:
    storage: "15Gi"
    source:
    http:
    url: "https://github.com/siderolabs/talos/releases/download/v1.2.5/nocloud-amd64.raw.xz"
    - metadata:
    name: vmdisk-dv
    spec:
    accessModes:
    - ReadWriteMany
    capacity:
    storage: 64G
    config:
    featureGates:
    - HonorWaitForFirstConsumer
    pvc:
    accessModes:
    - ReadWriteMany
    resources:
    requests:
    storage: 64G
    limits:
    storage: 64G
    storageClassName: ceph-filesystem
    source:
    registry:
    url: "docker.io/katamo/talos:latest"
    imagePullPolicy: Always
    persistentVolumeReclaimPolicy: Delete
    storageClassName: ceph-filesystem
    volumeMode: Block
    ---
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: KubevirtMachineTemplate
    @@ -180,27 +238,85 @@ spec:
    name: vmdisk-dv
    domain:
    cpu:
    cores: 2
    cores: 4
    devices:
    disks:
    - name: vmdisk
    disk:
    bus: scsi
    bus: virtio
    memory:
    guest: "4Gi"
    guest: "6Gi"
    template:
    spec:
    nodeSelector:
    node-role.kubernetes.io/kubevirt: ""
    domain:
    clock:
    utc: {}
    timer: {}
    cpu:
    cores: 1
    sockets: 1
    threads: 4
    model: host-passthrough
    dedicatedCpuPlacement: false
    devices:
    rng: {}
    autoattachPodInterface: false
    autoattachSerialConsole: true
    autoattachGraphicsDevice: true
    networkInterfaceMultiqueue: false
    disks:
    - name: vmdisk
    bootOrder: 1
    disk:
    bus: virtio
    interfaces:
    - name: enp1s0
    model: virtio
    bridge: {}
    machine:
    type: q35
    resources:
    requests:
    memory: 12G
    devices.kubevirt.io/kvm: "1"
    networks:
    - name: enp1s0
    multus:
    networkName: nadbr0
    terminationGracePeriodSeconds: 0
    volumes:
    - name: vmdisk
    dataVolume:
    name: vmdisk-dv
    dataVolumeTemplates:
    - metadata:
    name: vmdisk-dv
    spec:
    pvc:
    accessModes:
    - ReadWriteOnce
    resources:
    requests:
    storage: "15Gi"
    source:
    http:
    url: "https://github.com/siderolabs/talos/releases/download/v1.2.5/nocloud-amd64.raw.xz"
    - metadata:
    name: vmdisk-dv
    spec:
    accessModes:
    - ReadWriteMany
    capacity:
    storage: 64G
    config:
    featureGates:
    - HonorWaitForFirstConsumer
    pvc:
    accessModes:
    - ReadWriteMany
    resources:
    requests:
    storage: 64G
    limits:
    storage: 64G
    storageClassName: ceph-filesystem
    source:
    registry:
    url: "docker.io/katamo/talos:latest"
    imagePullPolicy: Always
    persistentVolumeReclaimPolicy: Delete
    storageClassName: ceph-filesystem
    volumeMode: Block
    ---
    apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
    kind: TalosConfigTemplate
  2. usrbinkat renamed this gist Oct 20, 2022. 1 changed file with 0 additions and 0 deletions.
    0 README.md → 01-README.md
    View file
    File renamed without changes.
  3. usrbinkat revised this gist Oct 20, 2022. 3 changed files with 0 additions and 0 deletions.
    0 capi-cluster-kubeadm.yaml → 02-capi-cluster-kubeadm.yaml
    View file
    File renamed without changes.
    0 capi-cluster-talos.yaml → 03-capi-cluster-talos.yaml
    View file
    File renamed without changes.
    0 SCRATCH.md → README.md
    View file
    File renamed without changes.
  4. usrbinkat created this gist Oct 20, 2022.
    18 changes: 18 additions & 0 deletions SCRATCH.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,18 @@
    Bare Metal:
    - [x] [Talos](https://www.talos.dev/v1.2/)
    - [X] [Cert Manager](https://cert-manager.io/docs/)
    - [x] [Rook Ceph](https://rook.github.io/docs/rook/v1.10/Getting-Started/intro/)
    - [x] [Kubevirt](https://kubevirt.io/user-guide/)
    - [X] [Containerized Data Importer](https://kubevirt.io/user-guide/operations/containerized_data_importer/)
    - [X] [Cluster Network Addons Operator](https://github.com/kubevirt/cluster-network-addons-operator)
    - [ ] [Cluster API](https://cluster-api.sigs.k8s.io)
    - [ ] []()

    [CAPI Docs](https://cluster-api.sigs.k8s.io/)
    Initialise the providers
    ```
    clusterctl init -i kubevirt -c talos -b talos
    ```
    [Talos resources](https://github.com/siderolabs/sidero/releases/download/v0.5.5/cluster-template.yaml)
    [KubeVirt resources](https://github.com/kubernetes-sigs/cluster-api-provider-kubevirt/releases/download/v0.1.2/cluster-template.yaml)
    [Rough YAML that provisions kubevirt+talos+capi](https://github.com/sharingio/coder/blob/main/examples/templates/kubevirt-talos/manifest/kvtalos.yaml)
    162 changes: 162 additions & 0 deletions capi-cluster-kubeadm.yaml
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,162 @@
    ---
    apiVersion: cluster.x-k8s.io/v1beta1
    kind: Cluster
    metadata:
    name: "${CLUSTER_NAME}"
    namespace: "${NAMESPACE}"
    spec:
    clusterNetwork:
    pods:
    cidrBlocks:
    - 10.243.0.0/16
    services:
    cidrBlocks:
    - 10.95.0.0/16
    infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: KubevirtCluster
    name: '${CLUSTER_NAME}'
    namespace: "${NAMESPACE}"
    controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
    kind: KubeadmControlPlane
    name: '${CLUSTER_NAME}-control-plane'
    namespace: "${NAMESPACE}"
    ---
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: KubevirtCluster
    metadata:
    name: "${CLUSTER_NAME}"
    namespace: "${NAMESPACE}"
    spec:
    controlPlaneServiceTemplate:
    spec:
    type: ClusterIP
    ---
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: KubevirtMachineTemplate
    metadata:
    name: "${CLUSTER_NAME}-control-plane"
    namespace: "${NAMESPACE}"
    spec:
    template:
    spec:
    virtualMachineTemplate:
    metadata:
    namespace: "${NAMESPACE}"
    spec:
    runStrategy: Always
    template:
    spec:
    domain:
    cpu:
    cores: 2
    memory:
    guest: "4Gi"
    devices:
    disks:
    - disk:
    bus: virtio
    name: containervolume
    evictionStrategy: External
    volumes:
    - containerDisk:
    image: "${NODE_VM_IMAGE_TEMPLATE}"
    name: containervolume
    ---
    kind: KubeadmControlPlane
    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
    metadata:
    name: "${CLUSTER_NAME}-control-plane"
    namespace: "${NAMESPACE}"
    spec:
    replicas: ${CONTROL_PLANE_MACHINE_COUNT}
    machineTemplate:
    infrastructureRef:
    kind: KubevirtMachineTemplate
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    name: "${CLUSTER_NAME}-control-plane"
    namespace: "${NAMESPACE}"
    kubeadmConfigSpec:
    clusterConfiguration:
    imageRepository: ${IMAGE_REPO}
    networking:
    dnsDomain: "${CLUSTER_NAME}.${NAMESPACE}.local"
    podSubnet: 10.243.0.0/16
    serviceSubnet: 10.95.0.0/16
    initConfiguration:
    nodeRegistration:
    criSocket: "${CRI_PATH}"
    joinConfiguration:
    nodeRegistration:
    criSocket: "{CRI_PATH}"
    version: "${KUBERNETES_VERSION}"
    ---
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: KubevirtMachineTemplate
    metadata:
    name: "${CLUSTER_NAME}-md-0"
    namespace: "${NAMESPACE}"
    spec:
    template:
    spec:
    virtualMachineTemplate:
    metadata:
    namespace: "${NAMESPACE}"
    spec:
    runStrategy: Always
    template:
    spec:
    domain:
    cpu:
    cores: 2
    memory:
    guest: "4Gi"
    devices:
    disks:
    - disk:
    bus: virtio
    name: containervolume
    evictionStrategy: External
    volumes:
    - containerDisk:
    image: "${NODE_VM_IMAGE_TEMPLATE}"
    name: containervolume
    ---
    apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    kind: KubeadmConfigTemplate
    metadata:
    name: "${CLUSTER_NAME}-md-0"
    namespace: "${NAMESPACE}"
    spec:
    template:
    spec:
    joinConfiguration:
    nodeRegistration:
    kubeletExtraArgs: {}
    ---
    apiVersion: cluster.x-k8s.io/v1beta1
    kind: MachineDeployment
    metadata:
    name: "${CLUSTER_NAME}-md-0"
    namespace: "${NAMESPACE}"
    spec:
    clusterName: "${CLUSTER_NAME}"
    replicas: ${WORKER_MACHINE_COUNT}
    selector:
    matchLabels:
    template:
    spec:
    clusterName: "${CLUSTER_NAME}"
    version: "${KUBERNETES_VERSION}"
    bootstrap:
    configRef:
    name: "${CLUSTER_NAME}-md-0"
    namespace: "${NAMESPACE}"
    apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    kind: KubeadmConfigTemplate
    infrastructureRef:
    name: "${CLUSTER_NAME}-md-0"
    namespace: "${NAMESPACE}"
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: KubevirtMachineTemplate
    213 changes: 213 additions & 0 deletions capi-cluster-talos.yaml
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,213 @@
    apiVersion: cluster.x-k8s.io/v1beta1
    kind: Cluster
    metadata:
    labels:
    cluster-name: talos
    name: talos
    spec:
    controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
    kind: TalosControlPlane
    name: talos
    infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    kind: KubevirtCluster
    name: talos
    clusterNetwork:
    pods:
    cidrBlocks:
    - 192.168.0.0/16
    services:
    cidrBlocks:
    - 172.26.0.0/16
    ---
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: KubevirtCluster
    metadata:
    name: talos
    spec:
    controlPlaneServiceTemplate:
    spec:
    type: ClusterIP
    ---
    apiVersion: cluster.x-k8s.io/v1beta1
    kind: MachineDeployment
    metadata:
    name: talos
    spec:
    clusterName: talos
    replicas: 0
    selector:
    matchLabes: null
    template:
    spec:
    clusterName: talos
    version: v1.23.5
    bootstrap:
    configRef:
    apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    kind: TalosConfigTemplate
    name: talos
    infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: KubevirtMachineTemplate
    name: talos
    ---
    apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
    kind: TalosControlPlane
    metadata:
    name: talos
    spec:
    replicas: 1
    version: "v1.25.2"
    infrastructureTemplate:
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: KubevirtMachineTemplate
    name: talos-cp
    controlPlaneConfig:
    init:
    generateType: init
    configPatches:
    - path: /machine/install
    op: replace
    value:
    bootloader: true
    disk: /dev/vda
    image: ghcr.io/siderolabs/installer:v1.2.5
    wipe: false
    extraKernelArgs:
    - console=ttyS0
    - path: /machine/kubelet/extraArgs
    op: add
    value:
    cloud-provider: external
    - path: /machine/apiServer/extraArgs
    op: add
    value:
    cloud-provider: external
    - path: /machine/controllerManager/extraArgs
    op: add
    value:
    cloud-provider: external
    - path: /cluster/allowSchedulingOnMasters
    op: add
    value: true
    controlplane:
    generateType: controlplane
    configPatches:
    - path: /machine/install
    op: replace
    value:
    bootloader: true
    disk: /dev/vda
    image: ghcr.io/siderolabs/installer:v1.2.5
    wipe: false
    extraKernelArgs:
    - console=ttyS0
    - path: /machine/kubelet/extraArgs
    op: add
    value:
    cloud-provider: external
    - path: /machine/apiServer/extraArgs
    op: add
    value:
    cloud-provider: external
    - path: /machine/controllerManager/extraArgs
    op: add
    value:
    cloud-provider: external
    - path: /cluster/allowSchedulingOnMasters
    op: add
    value: true
    ---
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: KubevirtMachineTemplate
    metadata:
    name: talos-cp
    spec:
    template:
    spec:
    virtualMachineTemplate:
    spec:
    runStrategy: Always
    template:
    spec:
    evictionStrategy: External
    volumes:
    - name: vmdisk
    dataVolume:
    name: vmdisk-dv
    domain:
    cpu:
    cores: 2
    devices:
    disks:
    - name: vmdisk
    disk:
    bus: scsi
    memory:
    guest: "4Gi"
    dataVolumeTemplates:
    - metadata:
    name: vmdisk-dv
    spec:
    pvc:
    accessModes:
    - ReadWriteOnce
    resources:
    requests:
    storage: "15Gi"
    source:
    http:
    url: "https://github.com/siderolabs/talos/releases/download/v1.2.5/nocloud-amd64.raw.xz"
    ---
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: KubevirtMachineTemplate
    metadata:
    name: talos
    spec:
    template:
    spec:
    virtualMachineTemplate:
    spec:
    runStrategy: Always
    template:
    spec:
    evictionStrategy: External
    volumes:
    - name: vmdisk
    dataVolume:
    name: vmdisk-dv
    domain:
    cpu:
    cores: 2
    devices:
    disks:
    - name: vmdisk
    disk:
    bus: scsi
    memory:
    guest: "4Gi"
    dataVolumeTemplates:
    - metadata:
    name: vmdisk-dv
    spec:
    pvc:
    accessModes:
    - ReadWriteOnce
    resources:
    requests:
    storage: "15Gi"
    source:
    http:
    url: "https://github.com/siderolabs/talos/releases/download/v1.2.5/nocloud-amd64.raw.xz"
    ---
    apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
    kind: TalosConfigTemplate
    metadata:
    name: talos
    spec:
    template:
    spec:
    generateType: join
    talosVersion: v1.2.5