sigs.k8s.io/cluster-api@v1.7.1/docs/book/src/tasks/experimental-features/cluster-class/write-clusterclass.md (about)

     1  
     2  # Writing a ClusterClass
     3  
     4  A ClusterClass becomes more useful and valuable when it can be used to create many Cluster of a similar 
     5  shape. The goal of this document is to explain how ClusterClasses can be written in a way that they are 
     6  flexible enough to be used in as many Clusters as possible by supporting variants of the same base Cluster shape.
     7  
     8  **Table of Contents**
     9  
    10  * [Basic ClusterClass](#basic-clusterclass)
    11  * [ClusterClass with MachineHealthChecks](#clusterclass-with-machinehealthchecks)
    12  * [ClusterClass with patches](#clusterclass-with-patches)
    13  * [ClusterClass with custom naming strategies](#clusterclass-with-custom-naming-strategies)
    14      * [Defining a custom naming strategy for ControlPlane objects](#defining-a-custom-naming-strategy-for-controlplane-objects)
    15      * [Defining a custom naming strategy for MachineDeployment objects](#defining-a-custom-naming-strategy-for-machinedeployment-objects)
    16      * [Defining a custom naming strategy for MachinePool objects](#defining-a-custom-naming-strategy-for-machinepool-objects)
    17  * [Advanced features of ClusterClass with patches](#advanced-features-of-clusterclass-with-patches)
    18      * [MachineDeployment variable overrides](#machinedeployment-variable-overrides)
    19      * [Builtin variables](#builtin-variables)
    20      * [Complex variable types](#complex-variable-types)
    21      * [Using variable values in JSON patches](#using-variable-values-in-json-patches)
    22      * [Optional patches](#optional-patches)
    23      * [Version-aware patches](#version-aware-patches)
    24  * [JSON patches tips & tricks](#json-patches-tips--tricks)
    25  
    26  ## Basic ClusterClass
    27  
    28  The following example shows a basic ClusterClass. It contains templates to shape the control plane, 
    29  infrastructure and workers of a Cluster. When a Cluster is using this ClusterClass, the templates 
    30  are used to generate the objects of the managed topology of the Cluster.
    31  
    32  ```yaml
    33  apiVersion: cluster.x-k8s.io/v1beta1
    34  kind: ClusterClass
    35  metadata:
    36    name: docker-clusterclass-v0.1.0
    37  spec:
    38    controlPlane:
    39      ref:
    40        apiVersion: controlplane.cluster.x-k8s.io/v1beta1
    41        kind: KubeadmControlPlaneTemplate
    42        name: docker-clusterclass-v0.1.0
    43        namespace: default
    44      machineInfrastructure:
    45        ref:
    46          kind: DockerMachineTemplate
    47          apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    48          name: docker-clusterclass-v0.1.0
    49          namespace: default
    50    infrastructure:
    51      ref:
    52        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    53        kind: DockerClusterTemplate
    54        name: docker-clusterclass-v0.1.0-control-plane
    55        namespace: default
    56    workers:
    57      machineDeployments:
    58      - class: default-worker
    59        template:
    60          bootstrap:
    61            ref:
    62              apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    63              kind: KubeadmConfigTemplate
    64              name: docker-clusterclass-v0.1.0-default-worker
    65              namespace: default
    66          infrastructure:
    67            ref:
    68              apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    69              kind: DockerMachineTemplate
    70              name: docker-clusterclass-v0.1.0-default-worker
    71              namespace: default
    72  ```
    73  
    74  The following example shows a Cluster using this ClusterClass. In this case a `KubeadmControlPlane` 
    75  with the corresponding `DockerMachineTemplate`, a `DockerCluster` and a `MachineDeployment` with 
    76  the corresponding `KubeadmConfigTemplate` and `DockerMachineTemplate` will be created. This basic 
    77  ClusterClass is already very flexible. Via the topology on the Cluster the following can be configured:
    78  * `.spec.topology.version`: the Kubernetes version of the Cluster
    79  * `.spec.topology.controlPlane`: ControlPlane replicas and their metadata
    80  * `.spec.topology.workers`: MachineDeployments and their replicas, metadata and failure domain
    81  
    82  ```yaml
    83  apiVersion: cluster.x-k8s.io/v1beta1
    84  kind: Cluster
    85  metadata:
    86    name: my-docker-cluster
    87  spec:
    88    topology:
    89      class: docker-clusterclass-v0.1.0
    90      version: v1.22.4
    91      controlPlane:
    92        replicas: 3
    93        metadata:
    94          labels:
    95            cpLabel: cpLabelValue 
    96          annotations:
    97            cpAnnotation: cpAnnotationValue
    98      workers:
    99        machineDeployments:
   100        - class: default-worker
   101          name: md-0
   102          replicas: 4
   103          metadata:
   104            labels:
   105              mdLabel: mdLabelValue
   106            annotations:
   107              mdAnnotation: mdAnnotationValue
   108          failureDomain: region
   109  ```
   110  
   111  Best practices:
   112  * The ClusterClass name should be generic enough to make sense across multiple clusters, i.e. a 
   113    name which corresponds to a single Cluster, e.g. "my-cluster", is not recommended.
   114  * Try to keep the ClusterClass names short and consistent (if you publish multiple ClusterClasses).
   115  * As a ClusterClass usually evolves over time and you might want to rebase Clusters from one version 
   116    of a ClusterClass to another, consider including a version suffix in the ClusterClass name.
   117    For more information about changing a ClusterClass please see: [Changing a ClusterClass].
   118  * Prefix the templates used in a ClusterClass with the name of the ClusterClass.
   119  * Don't reuse the same template in multiple ClusterClasses. This is automatically taken care
   120    of by prefixing the templates with the name of the ClusterClass.
   121  
   122  <aside class="note">
   123  
   124  For a full example ClusterClass for CAPD you can take a look at
   125  [clusterclass-quickstart.yaml](https://github.com/kubernetes-sigs/cluster-api/blob/main/test/infrastructure/docker/templates/clusterclass-quick-start.yaml)
   126  (which is also used in the CAPD quickstart with ClusterClass).
   127  
   128  </aside>
   129  
   130  <aside class="note">
   131  
   132  <h1>Tip: clusterctl alpha topology plan</h1>
   133  
   134  The `clusterctl alpha topology plan` command can be used to test ClusterClasses; the output will show
   135  you how the resulting Cluster will look like, but without actually creating it.
   136  For more details please see: [clusterctl alpha topology plan].
   137  
   138  </aside>
   139  
   140  ## ClusterClass with MachinePools
   141  
   142  ClusterClass also supports MachinePool workers. They work very similar to MachineDeployments. MachinePools
   143  can be specified in the ClusterClass template under the workers section like so:
   144  
   145  ```yaml
   146  apiVersion: cluster.x-k8s.io/v1beta1
   147  kind: ClusterClass
   148  metadata:
   149    name: docker-clusterclass-v0.1.0
   150  spec:
   151    workers:
   152      machinePools:
   153      - class: default-worker
   154        template:
   155          bootstrap:
   156            ref:
   157              apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
   158              kind: KubeadmConfigTemplate
   159              name: quick-start-default-worker-bootstraptemplate
   160          infrastructure:
   161            ref:
   162              apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
   163              kind: DockerMachinePoolTemplate
   164              name: quick-start-default-worker-machinepooltemplate
   165  ```
   166  
   167  They can then be similarly defined as workers in the cluster template like so:
   168  
   169  ```yaml
   170  apiVersion: cluster.x-k8s.io/v1beta1
   171  kind: Cluster
   172  metadata:
   173    name: my-docker-cluster
   174  spec:
   175    topology:
   176      workers:
   177        machinePools:
   178        - class: default-worker
   179          name: mp-0
   180          replicas: 4
   181          metadata:
   182            labels:
   183              mpLabel: mpLabelValue
   184            annotations:
   185              mpAnnotation: mpAnnotationValue
   186          failureDomain: region
   187  ```
   188  
   189  ## ClusterClass with MachineHealthChecks
   190  
   191  `MachineHealthChecks` can be configured in the ClusterClass for the control plane and for a 
   192  MachineDeployment class. The following configuration makes sure a `MachineHealthCheck` is 
   193  created for the control plane and for every `MachineDeployment` using the `default-worker` class.
   194  
   195  ```yaml
   196  apiVersion: cluster.x-k8s.io/v1beta1
   197  kind: ClusterClass
   198  metadata:
   199    name: docker-clusterclass-v0.1.0
   200  spec:
   201    controlPlane:
   202      ...
   203      machineHealthCheck:
   204        maxUnhealthy: 33%
   205        nodeStartupTimeout: 15m
   206        unhealthyConditions:
   207        - type: Ready
   208          status: Unknown
   209          timeout: 300s
   210        - type: Ready
   211          status: "False"
   212          timeout: 300s
   213    workers:
   214      machineDeployments:
   215      - class: default-worker
   216        ...
   217        machineHealthCheck:
   218          unhealthyRange: "[0-2]"
   219          nodeStartupTimeout: 10m
   220          unhealthyConditions:
   221          - type: Ready
   222            status: Unknown
   223            timeout: 300s
   224          - type: Ready
   225            status: "False"
   226            timeout: 300s
   227  ```
   228  
   229  ## ClusterClass with patches
   230  
   231  As shown above, basic ClusterClasses are already very powerful. But there are cases where 
   232  more powerful mechanisms are required. Let's assume you want to manage multiple Clusters 
   233  with the same ClusterClass, but they require different values for a field in one of the 
   234  referenced templates of a ClusterClass.
   235  
   236  A concrete example would be to deploy Clusters with different registries. In this case, 
   237  every cluster needs a Cluster-specific value for `.spec.kubeadmConfigSpec.clusterConfiguration.imageRepository` 
   238  in `KubeadmControlPlane`. Use cases like this can be implemented with ClusterClass patches.
   239  
   240  **Defining variables in the ClusterClass**
   241  
   242  The following example shows how variables can be defined in the ClusterClass.
   243  A variable definition specifies the name and the schema of a variable and if it is 
   244  required. The schema defines how a variable is defaulted and validated. It supports 
   245  a subset of the schema of CRDs. For more information please see the [godoc](https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api/cluster.x-k8s.io/ClusterClass/v1beta1#spec-variables-schema-openAPIV3Schema).
   246  
   247  ```yaml
   248  apiVersion: cluster.x-k8s.io/v1beta1
   249  kind: ClusterClass
   250  metadata:
   251    name: docker-clusterclass-v0.1.0
   252  spec:
   253    ...
   254    variables:
   255    - name: imageRepository
   256      required: true
   257      schema:
   258        openAPIV3Schema:
   259          type: string
   260          description: ImageRepository is the container registry to pull images from.
   261          default: registry.k8s.io
   262          example: registry.k8s.io
   263  ```
   264  
   265  <aside class="note">
   266  
   267  <h1>Supported types</h1>
   268  
   269  The following basic types are supported: `string`, `integer`, `number` and `boolean`. We are also 
   270  supporting complex types, please see the [complex variable types](#complex-variable-types) section.
   271  
   272  </aside>
   273  
   274  **Defining patches in the ClusterClass**
   275  
   276  The variable can then be used in a patch to set a field on a template referenced in the ClusterClass.
   277  The `selector` specifies on which template the patch should be applied. `jsonPatches` specifies which JSON 
   278  patches should be applied to that template. In this case we set the `imageRepository` field of the 
   279  `KubeadmControlPlaneTemplate` to the value of the variable `imageRepository`. For more information 
   280  please see the [godoc](https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api/cluster.x-k8s.io/ClusterClass/v1beta1#spec-patches-definitions).
   281  
   282  ```yaml
   283  apiVersion: cluster.x-k8s.io/v1beta1
   284  kind: ClusterClass
   285  metadata:
   286    name: docker-clusterclass-v0.1.0
   287  spec:
   288    ...
   289    patches:
   290    - name: imageRepository
   291      definitions:
   292      - selector:
   293          apiVersion: controlplane.cluster.x-k8s.io/v1beta1
   294          kind: KubeadmControlPlaneTemplate
   295          matchResources:
   296            controlPlane: true
   297        jsonPatches:
   298        - op: add
   299          path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/imageRepository
   300          valueFrom:
   301            variable: imageRepository
   302  ```
   303  
   304  <aside class="note">
   305  
   306  <h1>Writing JSON patches</h1>
   307  
   308  * Only fields below `/spec` can be patched.
   309  * Only `add`, `remove` and `replace` operations are supported.
   310  * It's only possible to append and prepend to arrays. Insertions at a specific index are 
   311    not supported.
   312  * Be careful, appending or prepending an array variable to an array leads to a nested array
   313    (for more details please see this [issue](https://github.com/kubernetes-sigs/cluster-api/issues/5944)).
   314  
   315  </aside>
   316  
   317  **Setting variable values in the Cluster**
   318  
   319  After creating a ClusterClass with a variable definition, the user can now provide a value for 
   320  the variable in the Cluster as in the example below.
   321  
   322  ```yaml
   323  apiVersion: cluster.x-k8s.io/v1beta1
   324  kind: Cluster
   325  metadata:
   326    name: my-docker-cluster
   327  spec:
   328    topology:
   329      ...
   330      variables:
   331      - name: imageRepository
   332        value: my.custom.registry
   333  ```
   334  
   335  <aside class="note">
   336  
   337  <h1>Variable defaulting</h1>
   338  
   339  If the user does not set the value, but the corresponding variable definition in ClusterClass has
   340  a default value, the value is automatically added to the variables list.
   341  
   342  </aside>
   343  
   344  ## ClusterClass with custom naming strategies
   345  
   346  The controller needs to generate names for new objects when a Cluster is getting created
   347  from a ClusterClass. These names have to be unique for each namespace. The naming
   348  strategy enables this by concatenating the cluster name with a random suffix.
   349  
   350  It is possible to provide a custom template for the name generation of ControlPlane, MachineDeployment
   351  and MachinePool objects.
   352  
   353  The generated names must comply with the [RFC 1123](https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#dns-label-names) standard.
   354  
   355  ### Defining a custom naming strategy for ControlPlane objects
   356  
   357  The naming strategy for ControlPlane supports the following properties:
   358  
   359  - `template`: Custom template which is used when generating the name of the ControlPlane object.
   360  
   361  The following variables can be referenced in templates:
   362  
   363  - `.cluster.name`: The name of the cluster object.
   364  - `.random`: A random alphanumeric string, without vowels, of length 5.
   365  
   366  Example which would match the default behavior:
   367  
   368  ```yaml
   369  apiVersion: cluster.x-k8s.io/v1beta1
   370  kind: ClusterClass
   371  metadata:
   372    name: docker-clusterclass-v0.1.0
   373  spec:
   374    controlPlane:
   375      ...
   376      namingStrategy:
   377        template: "{{ .cluster.name }}-{{ .random }}"
   378    ...
   379  ```
   380  
   381  ### Defining a custom naming strategy for MachineDeployment objects
   382  
   383  The naming strategy for MachineDeployments supports the following properties:
   384  
   385  - `template`: Custom template which is used when generating the name of the MachineDeployment object.
   386  
   387  The following variables can be referenced in templates:
   388  
   389  - `.cluster.name`: The name of the cluster object.
   390  - `.random`: A random alphanumeric string, without vowels, of length 5.
   391  - `.machineDeployment.topologyName`: The name of the MachineDeployment topology (`Cluster.spec.topology.workers.machineDeployments[].name`)
   392  
   393  Example which would match the default behavior:
   394  
   395  ```yaml
   396  apiVersion: cluster.x-k8s.io/v1beta1
   397  kind: ClusterClass
   398  metadata:
   399    name: docker-clusterclass-v0.1.0
   400  spec:
   401    controlPlane:
   402      ...
   403    workers:
   404      machineDeployments:
   405      - class: default-worker
   406        ...
   407        namingStrategy:
   408          template: "{{ .cluster.name }}-{{ .machineDeployment.topologyName }}-{{ .random }}"
   409  ```
   410  
   411  ### Defining a custom naming strategy for MachinePool objects
   412  
   413  The naming strategy for MachinePools supports the following properties:
   414  
   415  - `template`: Custom template which is used when generating the name of the MachinePool object.
   416  
   417  The following variables can be referenced in templates:
   418  
   419  - `.cluster.name`: The name of the cluster object.
   420  - `.random`: A random alphanumeric string, without vowels, of length 5.
   421  - `.machinePool.topologyName`: The name of the MachinePool topology (`Cluster.spec.topology.workers.machinePools[].name`).
   422  
   423  Example which would match the default behavior:
   424  
   425  ```yaml
   426  apiVersion: cluster.x-k8s.io/v1beta1
   427  kind: ClusterClass
   428  metadata:
   429    name: docker-clusterclass-v0.1.0
   430  spec:
   431    controlPlane:
   432      ...
   433    workers:
   434      machinePools:
   435      - class: default-worker
   436        ...
   437        namingStrategy:
   438          template: "{{ .cluster.name }}-{{ .machinePool.topologyName }}-{{ .random }}"
   439  ```
   440  
   441  ## Advanced features of ClusterClass with patches
   442  
   443  This section will explain more advanced features of ClusterClass patches.
   444  
   445  ### MachineDeployment & MachinePool variable overrides
   446  
   447  If you want to use many variations of MachineDeployments in Clusters, you can either define
   448  a MachineDeployment class for every variation or you can define patches and variables to
   449  make a single MachineDeployment class more flexible. The same applies for MachinePools.
   450  
   451  In the following example we make the `instanceType` of a `AWSMachineTemplate` customizable.
   452  First we define the `workerMachineType` variable and the corresponding patch:
   453  
   454  ```yaml
   455  apiVersion: cluster.x-k8s.io/v1beta1
   456  kind: ClusterClass
   457  metadata:
   458    name: aws-clusterclass-v0.1.0
   459  spec:
   460    ...
   461    variables:
   462    - name: workerMachineType
   463      required: true
   464      schema:
   465        openAPIV3Schema:
   466          type: string
   467          default: t3.large
   468    patches:
   469    - name: workerMachineType
   470      definitions:
   471      - selector:
   472          apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
   473          kind: AWSMachineTemplate
   474          matchResources:
   475            machineDeploymentClass:
   476              names:
   477              - default-worker
   478        jsonPatches:
   479        - op: add
   480          path: /spec/template/spec/instanceType
   481          valueFrom:
   482            variable: workerMachineType
   483  ---
   484  apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
   485  kind: AWSMachineTemplate
   486  metadata:
   487    name: aws-clusterclass-v0.1.0-default-worker
   488  spec:
   489    template:
   490      spec:
   491        # instanceType: workerMachineType will be set by the patch.
   492        iamInstanceProfile: "nodes.cluster-api-provider-aws.sigs.k8s.io"
   493  ---
   494  ...
   495  ```
   496  
   497  In the Cluster resource the `workerMachineType` variable can then be set cluster-wide and
   498  it can also be overridden for an individual MachineDeployment or MachinePool.
   499  
   500  ```yaml
   501  apiVersion: cluster.x-k8s.io/v1beta1
   502  kind: Cluster
   503  metadata:
   504    name: my-aws-cluster
   505  spec:
   506    ...
   507    topology:
   508      class: aws-clusterclass-v0.1.0
   509      version: v1.22.0
   510      controlPlane:
   511        replicas: 3
   512      workers:
   513        machineDeployments:
   514        - class: "default-worker"
   515          name: "md-small-workers"
   516          replicas: 3
   517          variables:
   518            overrides:
   519            # Overrides the cluster-wide value with t3.small.
   520            - name: workerMachineType
   521              value: t3.small
   522        # Uses the cluster-wide value t3.large.
   523        - class: "default-worker"
   524          name: "md-large-workers"
   525          replicas: 3
   526      variables:
   527      - name: workerMachineType
   528        value: t3.large
   529  ```
   530  
   531  ### Builtin variables
   532  
   533  In addition to variables specified in the ClusterClass, the following builtin variables can be 
   534  referenced in patches:
   535  - `builtin.cluster.{name,namespace}`
   536  - `builtin.cluster.topology.{version,class}`
   537  - `builtin.cluster.network.{serviceDomain,services,pods,ipFamily}`
   538  - `builtin.controlPlane.{replicas,version,name}`
   539      - Please note, these variables are only available when patching control plane or control plane 
   540        machine templates.
   541  - `builtin.controlPlane.machineTemplate.infrastructureRef.name`
   542      - Please note, these variables are only available when using a control plane with machines and 
   543        when patching control plane or control plane machine templates.
   544  - `builtin.machineDeployment.{replicas,version,class,name,topologyName}`
   545      - Please note, these variables are only available when patching the templates of a MachineDeployment 
   546        and contain the values of the current `MachineDeployment` topology.
   547  - `builtin.machineDeployment.{infrastructureRef.name,bootstrap.configRef.name}`
   548      - Please note, these variables are only available when patching the templates of a MachineDeployment
   549        and contain the values of the current `MachineDeployment` topology.
   550  - `builtin.machinePool.{replicas,version,class,name,topologyName}`
   551      - Please note, these variables are only available when patching the templates of a MachinePool 
   552        and contain the values of the current `MachinePool` topology.
   553  - `builtin.machinePool.{infrastructureRef.name,bootstrap.configRef.name}`
   554      - Please note, these variables are only available when patching the templates of a MachinePool
   555        and contain the values of the current `MachinePool` topology.
   556  
   557  Builtin variables can be referenced just like regular variables, e.g.:
   558  ```yaml
   559  apiVersion: cluster.x-k8s.io/v1beta1
   560  kind: ClusterClass
   561  metadata:
   562    name: docker-clusterclass-v0.1.0
   563  spec:
   564    ...
   565    patches:
   566    - name: clusterName
   567      definitions:
   568      - selector:
   569        ...
   570        jsonPatches:
   571        - op: add
   572          path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/controllerManager/extraArgs/cluster-name
   573          valueFrom:
   574            variable: builtin.cluster.name
   575  ```
   576  
   577  **Tips & Tricks**
   578  
   579  Builtin variables can be used to dynamically calculate image names. The version used in the patch 
   580  will always be the same as the one we set in the corresponding MachineDeployment or MachinePool 
   581  (works the same way with `.builtin.controlPlane.version`).
   582  
   583  ```yaml
   584  apiVersion: cluster.x-k8s.io/v1beta1
   585  kind: ClusterClass
   586  metadata:
   587    name: docker-clusterclass-v0.1.0
   588  spec:
   589    ...
   590    patches:
   591    - name: customImage
   592      description: "Sets the container image that is used for running dockerMachines."
   593      definitions:
   594      - selector:
   595          apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
   596          kind: DockerMachineTemplate
   597          matchResources:
   598            machineDeploymentClass:
   599              names:
   600              - default-worker
   601        jsonPatches:
   602        - op: add
   603          path: /spec/template/spec/customImage
   604          valueFrom:
   605            template: |
   606              kindest/node:{{ .builtin.machineDeployment.version }}
   607  ```
   608  
   609  ### Complex variable types
   610  
   611  Variables can also be objects, maps and arrays. An object is specified with the type `object` and
   612  by the schemas of the fields of the object. A map is specified with the type `object` and the schema 
   613  of the map values. An array is specified via the type `array` and the schema of the array items.
   614  
   615  ```yaml
   616  apiVersion: cluster.x-k8s.io/v1beta1
   617  kind: ClusterClass
   618  metadata:
   619    name: docker-clusterclass-v0.1.0
   620  spec:
   621    ...
   622    variables:
   623    - name: httpProxy
   624      schema:
   625        openAPIV3Schema:
   626          type: object
   627          properties: 
   628            # Schema of the url field.
   629            url: 
   630              type: string
   631            # Schema of the noProxy field.
   632            noProxy:
   633              type: string
   634    - name: mdConfig
   635      schema:
   636        openAPIV3Schema:
   637          type: object
   638          additionalProperties:
   639            # Schema of the map values.
   640            type: object
   641            properties:
   642              osImage:
   643                type: string
   644    - name: dnsServers
   645      schema:
   646        openAPIV3Schema:
   647          type: array
   648          items:
   649            # Schema of the array items.
   650            type: string
   651  ```
   652  
   653  Objects, maps and arrays can be used in patches either directly by referencing the variable name,
   654  or by accessing individual fields. For example:
   655  ```yaml
   656  apiVersion: cluster.x-k8s.io/v1beta1
   657  kind: ClusterClass
   658  metadata:
   659    name: docker-clusterclass-v0.1.0
   660  spec:
   661    ...
   662    jsonPatches:
   663    - op: add
   664      path: /spec/template/spec/httpProxy/url
   665      valueFrom:
   666        # Use the url field of the httpProxy variable.
   667        variable: httpProxy.url
   668    - op: add
   669      path: /spec/template/spec/customImage
   670      valueFrom:
   671        # Use the osImage field of the mdConfig variable for the current MD class.
   672        template: "{{ (index .mdConfig .builtin.machineDeployment.class).osImage }}"
   673    - op: add
   674      path: /spec/template/spec/dnsServers
   675      valueFrom:
   676        # Use the entire dnsServers array.
   677        variable: dnsServers
   678    - op: add
   679      path: /spec/template/spec/dnsServer
   680      valueFrom:
   681        # Use the first item of the dnsServers array.
   682        variable: dnsServers[0]
   683  ```
   684  
   685  **Tips & Tricks**
   686  
   687  Complex variables can be used to make references in templates configurable, e.g. the `identityRef` used in `AzureCluster`.
   688  Of course it's also possible to only make the name of the reference configurable, including restricting the valid values 
   689  to a pre-defined enum.
   690  
   691  ```yaml
   692  apiVersion: cluster.x-k8s.io/v1beta1
   693  kind: ClusterClass
   694  metadata:
   695    name: azure-clusterclass-v0.1.0
   696  spec:
   697    ...
   698    variables:
   699    - name: clusterIdentityRef
   700      schema:
   701        openAPIV3Schema:
   702          type: object
   703          properties:
   704            kind:
   705              type: string
   706            name:
   707              type: string
   708  ```
   709  
   710  Even if OpenAPI schema allows defining free form objects, e.g.
   711  
   712  ```yaml
   713  variables:
   714    - name: freeFormObject
   715      schema:
   716        openAPIV3Schema:
   717          type: object
   718  ```
   719  
   720  User should be aware that the lack of the validation of users provided data could lead to problems
   721  when those values are used in patch or when the generated templates are created (see e.g.
   722   [6135](https://github.com/kubernetes-sigs/cluster-api/issues/6135)).
   723  
   724  As a consequence we recommend avoiding this practice while we are considering alternatives to make
   725  it explicit for the ClusterClass authors to opt-in in this feature, thus accepting the implied risks.
   726  
   727  ### Using variable values in JSON patches
   728  
   729  We already saw above that it's possible to use variable values in JSON patches. It's also 
   730  possible to calculate values via Go templating or to use hard-coded values.
   731  
   732  ```yaml
   733  apiVersion: cluster.x-k8s.io/v1beta1
   734  kind: ClusterClass
   735  metadata:
   736    name: docker-clusterclass-v0.1.0
   737  spec:
   738    ...
   739    patches:
   740    - name: etcdImageTag
   741      definitions:
   742      - selector:
   743        ...
   744        jsonPatches:
   745        - op: add
   746          path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/etcd
   747          valueFrom:
   748            # This template is first rendered with Go templating, then parsed by 
   749            # a YAML/JSON parser and then used as value of the JSON patch.
   750            # For example, if the variable etcdImageTag is set to `3.5.1-0` the 
   751            # .../clusterConfiguration/etcd field will be set to:
   752            # {"local": {"imageTag": "3.5.1-0"}}
   753            template: |
   754              local:
   755                imageTag: {{ .etcdImageTag }}
   756    - name: imageRepository
   757      definitions:
   758      - selector:
   759        ...
   760        jsonPatches:
   761        - op: add
   762          path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/imageRepository
   763          # This hard-coded value is used directly as value of the JSON patch.
   764          value: "my.custom.registry"
   765  ```
   766  
   767  <aside class="note">
   768  
   769  <h1>Variable paths</h1>
   770  
   771  * Paths can be used in `.valueFrom.template` and `.valueFrom.variable` to access nested fields of arrays and objects.
   772  * `.` is used to access a field of an object, e.g. `httpProxy.url`.
   773  * `[i]` is used to access an array element, e.g. `dnsServers[0]`.
   774  * Because of the way Go templates work, the paths in templates have to start with a dot.
   775  
   776  </aside>
   777  
   778  **Tips & Tricks**
   779  
   780  Templates can be used to implement defaulting behavior during JSON patch value calculation. This can be used if the simple
   781  constant default value which can be specified in the schema is not enough.
   782  ```yaml
   783          valueFrom:
   784            # If .vnetName is set, it is used. Otherwise, we will use `{{.builtin.cluster.name}}-vnet`.  
   785            template: "{{ if .vnetName }}{{.vnetName}}{{else}}{{.builtin.cluster.name}}-vnet{{end}}"
   786  ```
   787  When writing templates, a subset of functions from [the Sprig library](https://masterminds.github.io/sprig/) can be used to
   788  write expressions, e.g., `{{ .name | upper }}`. Only functions that are guaranteed to evaluate to the same result
   789  for a given input are allowed (e.g. `upper` or `max` can be used, while `now` or `randAlpha` cannot be used).
   790  
   791  ### Optional patches
   792  
   793  Patches can also be conditionally enabled. This can be done by configuring a Go template via `enabledIf`. 
   794  The patch is then only applied if the Go template evaluates to `true`. In the following example the `httpProxy` 
   795  patch is only applied if the `httpProxy` variable is set (and not empty).
   796  
   797  ```yaml
   798  apiVersion: cluster.x-k8s.io/v1beta1
   799  kind: ClusterClass
   800  metadata:
   801    name: docker-clusterclass-v0.1.0
   802  spec:
   803    ...
   804    variables:
   805    - name: httpProxy
   806      schema:
   807        openAPIV3Schema:
   808          type: string
   809    patches:
   810    - name: httpProxy
   811      enabledIf: "{{ if .httpProxy }}true{{end}}"
   812      definitions:
   813      ...  
   814  ```
   815  
   816  **Tips & Tricks**:
   817  
   818  Hard-coded values can be used to test the impact of a patch during development, gradually roll out patches, etc. .
   819  ```yaml
   820      enabledIf: false
   821  ```
   822  
   823  A boolean variable can be used to enable/disable a patch (or "feature"). This can have opt-in or opt-out behavior
   824  depending on the default value of the variable.
   825  ```yaml
   826      enabledIf: "{{ .httpProxyEnabled }}"
   827  ```
   828  
   829  Of course the same is possible by adding a boolean variable to a configuration object.
   830  ```yaml
   831      enabledIf: "{{ .httpProxy.enabled }}"
   832  ```
   833  
   834  Builtin variables can be leveraged to apply a patch only for a specific Kubernetes version.
   835  ```yaml
   836      enabledIf: '{{ semverCompare "1.21.1" .builtin.controlPlane.version }}'
   837  ```
   838  
   839  With `semverCompare` and `coalesce` a feature can be enabled in newer versions of Kubernetes for both KubeadmConfigTemplate and KubeadmControlPlane.
   840  ```yaml
   841      enabledIf: '{{ semverCompare "^1.22.0" (coalesce .builtin.controlPlane.version .builtin.machineDeployment.version )}}'
   842  ```
   843  
   844  <aside class="note">
   845  
   846  <h1>Builtin Variables</h1>
   847  
   848  Please be aware that while you can use builtin variables, if you use for example a MachineDeployment-specific variable this
   849  can mean that patches are only applied to some MachineDeployments. `enabledIf` is evaluated for each template that should be patched
   850  individually.
   851  
   852  </aside>
   853  
   854  ### Version-aware patches
   855  
   856  In some cases the ClusterClass authors want a patch to be computed according to the Kubernetes version in use.
   857  
   858  While this is not a problem "per se" and it does not differ from writing any other patch, it is important 
   859  to keep in mind that there could be different Kubernetes version in a Cluster at any time, all of them
   860  accessible via built in variables:
   861   
   862  - `builtin.cluster.topology.version` defines the Kubernetes version from `cluster.topology`, and it acts
   863    as the desired Kubernetes version for the entire cluster. However, during an upgrade workflow it could happen that
   864    some objects in the Cluster are still at the older version.
   865  - `builtin.controlPlane.version`, represent the desired version for the control plane object; usually this
   866    version changes immediately after `cluster.topology.version` is updated (unless there are other operations
   867    in progress preventing the upgrade to start).
   868  - `builtin.machineDeployment.version`, represent the desired version for each specific MachineDeployment object;
   869    this version changes only after the upgrade for the control plane is completed, and in case of many
   870    MachineDeployments in the same cluster, they are upgraded sequentially.
   871  - `builtin.machinePool.version`, represent the desired version for each specific MachinePool object;
   872    this version changes only after the upgrade for the control plane is completed, and in case of many
   873    MachinePools in the same cluster, they are upgraded sequentially.
   874  
   875  This info should provide the bases for developing version-aware patches, allowing the patch author to determine when a
   876  patch should adapt to the new Kubernetes version by choosing one of the above variables. In practice the
   877  following rules applies to the most common use cases:
   878  
   879  - When developing a version-aware patch for the control plane, `builtin.controlPlane.version` must be used.
   880  - When developing a version-aware patch for MachineDeployments, `builtin.machineDeployment.version` must be used.
   881  - When developing a version-aware patch for MachinePools, `builtin.machinePool.version` must be used.
   882  
   883  **Tips & Tricks**:
   884  
   885  Sometimes users need to define variables to be used by version-aware patches, and in this case it is important
   886  to keep in mind that there could be different Kubernetes versions in a Cluster at any time.
   887  
   888  A simple approach to solve this problem is to define a map of version-aware variables, with the key of each item
   889  being the Kubernetes version. Patch could then use the proper builtin variables as a lookup entry to fetch 
   890  the corresponding values for the Kubernetes version in use by each object.
   891  
   892  ## JSON patches tips & tricks
   893  
   894  JSON patches specification [RFC6902] requires that the target of
   895  add operation must exist.
   896  
   897  As a consequence ClusterClass authors should pay special attention when the following
   898  conditions apply in order to prevent errors when a patch is applied:
   899  
   900  * the patch tries to `add` a value to an **array** (which is a **slice** in the corresponding go struct)
   901  * the slice was defined with `omitempty`
   902  * the slice currently does not exist
   903  
   904  A workaround in this particular case is to create the array in the patch instead of adding to the non-existing one.
   905  When creating the slice, existing values would be overwritten so this should only be used when it does not exist.
   906  
   907  The following example shows both cases to consider while writing a patch for adding a value to a slice.
   908  This patch targets to add a file to the `files` slice of a `KubeadmConfigTemplate` which has [omitempty](https://github.com/kubernetes-sigs/cluster-api/blob/main/bootstrap/kubeadm/api/v1beta1/kubeadmconfig_types.go#L54) set.
   909  
   910  {{#tabs name:"tab-configuration-patches" tabs:"Add to existing slice,Create slice"}}
   911  {{#tab Add to existing slice}}
   912  
   913  This patch **requires** the key `.spec.template.spec.files` to exist to succeed.
   914  
   915  ```yaml
   916  apiVersion: cluster.x-k8s.io/v1beta1
   917  kind: ClusterClass
   918  metadata:
   919    name: my-clusterclass
   920  spec:
   921    ...
   922    patches:
   923    - name: add file
   924      definitions:
   925      - selector:
   926          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
   927          kind: KubeadmConfigTemplate
   928        jsonPatches:
   929        - op: add
   930          path: /spec/template/spec/files/-
   931          value:
   932            content: Some content.
   933            path: /some/file
   934  ---
   935  apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
   936  kind: KubeadmConfigTemplate
   937  metadata:
   938    name: "quick-start-default-worker-bootstraptemplate"
   939  spec:
   940    template:
   941      spec:
   942        ...
   943        files:
   944        - content: Some other content
   945          path: /some/other/file
   946  ```
   947  
   948  {{#/tab }}
   949  {{#tab Create slice}}
   950  
   951  This patch would **overwrite** an existing slice at `.spec.template.spec.files`.
   952  
   953  ```yaml
   954  apiVersion: cluster.x-k8s.io/v1beta1
   955  kind: ClusterClass
   956  metadata:
   957    name: my-clusterclass
   958  spec:
   959    ...
   960    patches:
   961    - name: add file
   962      definitions:
   963      - selector:
   964          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
   965          kind: KubeadmConfigTemplate
   966        jsonPatches:
   967        - op: add
   968          path: /spec/template/spec/files
   969          value:
   970          - content: Some content.
   971            path: /some/file
   972  ---
   973  apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
   974  kind: KubeadmConfigTemplate
   975  metadata:
   976    name: "quick-start-default-worker-bootstraptemplate"
   977  spec:
   978    template:
   979      spec:
   980        ...
   981  ```
   982  
   983  {{#/tab }}
   984  {{#/tabs }}
   985  
   986  <!-- links -->
   987  [Changing a ClusterClass]: ./change-clusterclass.md
   988  [clusterctl alpha topology plan]: ../../../clusterctl/commands/alpha-topology-plan.md
   989  [RFC6902]: https://datatracker.ietf.org/doc/html/rfc6902#appendix-A.12