sigs.k8s.io/cluster-api@v1.7.1/docs/book/src/tasks/experimental-features/cluster-class/write-clusterclass.md (about) 1 2 # Writing a ClusterClass 3 4 A ClusterClass becomes more useful and valuable when it can be used to create many Cluster of a similar 5 shape. The goal of this document is to explain how ClusterClasses can be written in a way that they are 6 flexible enough to be used in as many Clusters as possible by supporting variants of the same base Cluster shape. 7 8 **Table of Contents** 9 10 * [Basic ClusterClass](#basic-clusterclass) 11 * [ClusterClass with MachineHealthChecks](#clusterclass-with-machinehealthchecks) 12 * [ClusterClass with patches](#clusterclass-with-patches) 13 * [ClusterClass with custom naming strategies](#clusterclass-with-custom-naming-strategies) 14 * [Defining a custom naming strategy for ControlPlane objects](#defining-a-custom-naming-strategy-for-controlplane-objects) 15 * [Defining a custom naming strategy for MachineDeployment objects](#defining-a-custom-naming-strategy-for-machinedeployment-objects) 16 * [Defining a custom naming strategy for MachinePool objects](#defining-a-custom-naming-strategy-for-machinepool-objects) 17 * [Advanced features of ClusterClass with patches](#advanced-features-of-clusterclass-with-patches) 18 * [MachineDeployment variable overrides](#machinedeployment-variable-overrides) 19 * [Builtin variables](#builtin-variables) 20 * [Complex variable types](#complex-variable-types) 21 * [Using variable values in JSON patches](#using-variable-values-in-json-patches) 22 * [Optional patches](#optional-patches) 23 * [Version-aware patches](#version-aware-patches) 24 * [JSON patches tips & tricks](#json-patches-tips--tricks) 25 26 ## Basic ClusterClass 27 28 The following example shows a basic ClusterClass. It contains templates to shape the control plane, 29 infrastructure and workers of a Cluster. When a Cluster is using this ClusterClass, the templates 30 are used to generate the objects of the managed topology of the Cluster. 31 32 ```yaml 33 apiVersion: cluster.x-k8s.io/v1beta1 34 kind: ClusterClass 35 metadata: 36 name: docker-clusterclass-v0.1.0 37 spec: 38 controlPlane: 39 ref: 40 apiVersion: controlplane.cluster.x-k8s.io/v1beta1 41 kind: KubeadmControlPlaneTemplate 42 name: docker-clusterclass-v0.1.0 43 namespace: default 44 machineInfrastructure: 45 ref: 46 kind: DockerMachineTemplate 47 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 48 name: docker-clusterclass-v0.1.0 49 namespace: default 50 infrastructure: 51 ref: 52 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 53 kind: DockerClusterTemplate 54 name: docker-clusterclass-v0.1.0-control-plane 55 namespace: default 56 workers: 57 machineDeployments: 58 - class: default-worker 59 template: 60 bootstrap: 61 ref: 62 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 63 kind: KubeadmConfigTemplate 64 name: docker-clusterclass-v0.1.0-default-worker 65 namespace: default 66 infrastructure: 67 ref: 68 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 69 kind: DockerMachineTemplate 70 name: docker-clusterclass-v0.1.0-default-worker 71 namespace: default 72 ``` 73 74 The following example shows a Cluster using this ClusterClass. In this case a `KubeadmControlPlane` 75 with the corresponding `DockerMachineTemplate`, a `DockerCluster` and a `MachineDeployment` with 76 the corresponding `KubeadmConfigTemplate` and `DockerMachineTemplate` will be created. This basic 77 ClusterClass is already very flexible. Via the topology on the Cluster the following can be configured: 78 * `.spec.topology.version`: the Kubernetes version of the Cluster 79 * `.spec.topology.controlPlane`: ControlPlane replicas and their metadata 80 * `.spec.topology.workers`: MachineDeployments and their replicas, metadata and failure domain 81 82 ```yaml 83 apiVersion: cluster.x-k8s.io/v1beta1 84 kind: Cluster 85 metadata: 86 name: my-docker-cluster 87 spec: 88 topology: 89 class: docker-clusterclass-v0.1.0 90 version: v1.22.4 91 controlPlane: 92 replicas: 3 93 metadata: 94 labels: 95 cpLabel: cpLabelValue 96 annotations: 97 cpAnnotation: cpAnnotationValue 98 workers: 99 machineDeployments: 100 - class: default-worker 101 name: md-0 102 replicas: 4 103 metadata: 104 labels: 105 mdLabel: mdLabelValue 106 annotations: 107 mdAnnotation: mdAnnotationValue 108 failureDomain: region 109 ``` 110 111 Best practices: 112 * The ClusterClass name should be generic enough to make sense across multiple clusters, i.e. a 113 name which corresponds to a single Cluster, e.g. "my-cluster", is not recommended. 114 * Try to keep the ClusterClass names short and consistent (if you publish multiple ClusterClasses). 115 * As a ClusterClass usually evolves over time and you might want to rebase Clusters from one version 116 of a ClusterClass to another, consider including a version suffix in the ClusterClass name. 117 For more information about changing a ClusterClass please see: [Changing a ClusterClass]. 118 * Prefix the templates used in a ClusterClass with the name of the ClusterClass. 119 * Don't reuse the same template in multiple ClusterClasses. This is automatically taken care 120 of by prefixing the templates with the name of the ClusterClass. 121 122 <aside class="note"> 123 124 For a full example ClusterClass for CAPD you can take a look at 125 [clusterclass-quickstart.yaml](https://github.com/kubernetes-sigs/cluster-api/blob/main/test/infrastructure/docker/templates/clusterclass-quick-start.yaml) 126 (which is also used in the CAPD quickstart with ClusterClass). 127 128 </aside> 129 130 <aside class="note"> 131 132 <h1>Tip: clusterctl alpha topology plan</h1> 133 134 The `clusterctl alpha topology plan` command can be used to test ClusterClasses; the output will show 135 you how the resulting Cluster will look like, but without actually creating it. 136 For more details please see: [clusterctl alpha topology plan]. 137 138 </aside> 139 140 ## ClusterClass with MachinePools 141 142 ClusterClass also supports MachinePool workers. They work very similar to MachineDeployments. MachinePools 143 can be specified in the ClusterClass template under the workers section like so: 144 145 ```yaml 146 apiVersion: cluster.x-k8s.io/v1beta1 147 kind: ClusterClass 148 metadata: 149 name: docker-clusterclass-v0.1.0 150 spec: 151 workers: 152 machinePools: 153 - class: default-worker 154 template: 155 bootstrap: 156 ref: 157 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 158 kind: KubeadmConfigTemplate 159 name: quick-start-default-worker-bootstraptemplate 160 infrastructure: 161 ref: 162 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 163 kind: DockerMachinePoolTemplate 164 name: quick-start-default-worker-machinepooltemplate 165 ``` 166 167 They can then be similarly defined as workers in the cluster template like so: 168 169 ```yaml 170 apiVersion: cluster.x-k8s.io/v1beta1 171 kind: Cluster 172 metadata: 173 name: my-docker-cluster 174 spec: 175 topology: 176 workers: 177 machinePools: 178 - class: default-worker 179 name: mp-0 180 replicas: 4 181 metadata: 182 labels: 183 mpLabel: mpLabelValue 184 annotations: 185 mpAnnotation: mpAnnotationValue 186 failureDomain: region 187 ``` 188 189 ## ClusterClass with MachineHealthChecks 190 191 `MachineHealthChecks` can be configured in the ClusterClass for the control plane and for a 192 MachineDeployment class. The following configuration makes sure a `MachineHealthCheck` is 193 created for the control plane and for every `MachineDeployment` using the `default-worker` class. 194 195 ```yaml 196 apiVersion: cluster.x-k8s.io/v1beta1 197 kind: ClusterClass 198 metadata: 199 name: docker-clusterclass-v0.1.0 200 spec: 201 controlPlane: 202 ... 203 machineHealthCheck: 204 maxUnhealthy: 33% 205 nodeStartupTimeout: 15m 206 unhealthyConditions: 207 - type: Ready 208 status: Unknown 209 timeout: 300s 210 - type: Ready 211 status: "False" 212 timeout: 300s 213 workers: 214 machineDeployments: 215 - class: default-worker 216 ... 217 machineHealthCheck: 218 unhealthyRange: "[0-2]" 219 nodeStartupTimeout: 10m 220 unhealthyConditions: 221 - type: Ready 222 status: Unknown 223 timeout: 300s 224 - type: Ready 225 status: "False" 226 timeout: 300s 227 ``` 228 229 ## ClusterClass with patches 230 231 As shown above, basic ClusterClasses are already very powerful. But there are cases where 232 more powerful mechanisms are required. Let's assume you want to manage multiple Clusters 233 with the same ClusterClass, but they require different values for a field in one of the 234 referenced templates of a ClusterClass. 235 236 A concrete example would be to deploy Clusters with different registries. In this case, 237 every cluster needs a Cluster-specific value for `.spec.kubeadmConfigSpec.clusterConfiguration.imageRepository` 238 in `KubeadmControlPlane`. Use cases like this can be implemented with ClusterClass patches. 239 240 **Defining variables in the ClusterClass** 241 242 The following example shows how variables can be defined in the ClusterClass. 243 A variable definition specifies the name and the schema of a variable and if it is 244 required. The schema defines how a variable is defaulted and validated. It supports 245 a subset of the schema of CRDs. For more information please see the [godoc](https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api/cluster.x-k8s.io/ClusterClass/v1beta1#spec-variables-schema-openAPIV3Schema). 246 247 ```yaml 248 apiVersion: cluster.x-k8s.io/v1beta1 249 kind: ClusterClass 250 metadata: 251 name: docker-clusterclass-v0.1.0 252 spec: 253 ... 254 variables: 255 - name: imageRepository 256 required: true 257 schema: 258 openAPIV3Schema: 259 type: string 260 description: ImageRepository is the container registry to pull images from. 261 default: registry.k8s.io 262 example: registry.k8s.io 263 ``` 264 265 <aside class="note"> 266 267 <h1>Supported types</h1> 268 269 The following basic types are supported: `string`, `integer`, `number` and `boolean`. We are also 270 supporting complex types, please see the [complex variable types](#complex-variable-types) section. 271 272 </aside> 273 274 **Defining patches in the ClusterClass** 275 276 The variable can then be used in a patch to set a field on a template referenced in the ClusterClass. 277 The `selector` specifies on which template the patch should be applied. `jsonPatches` specifies which JSON 278 patches should be applied to that template. In this case we set the `imageRepository` field of the 279 `KubeadmControlPlaneTemplate` to the value of the variable `imageRepository`. For more information 280 please see the [godoc](https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api/cluster.x-k8s.io/ClusterClass/v1beta1#spec-patches-definitions). 281 282 ```yaml 283 apiVersion: cluster.x-k8s.io/v1beta1 284 kind: ClusterClass 285 metadata: 286 name: docker-clusterclass-v0.1.0 287 spec: 288 ... 289 patches: 290 - name: imageRepository 291 definitions: 292 - selector: 293 apiVersion: controlplane.cluster.x-k8s.io/v1beta1 294 kind: KubeadmControlPlaneTemplate 295 matchResources: 296 controlPlane: true 297 jsonPatches: 298 - op: add 299 path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/imageRepository 300 valueFrom: 301 variable: imageRepository 302 ``` 303 304 <aside class="note"> 305 306 <h1>Writing JSON patches</h1> 307 308 * Only fields below `/spec` can be patched. 309 * Only `add`, `remove` and `replace` operations are supported. 310 * It's only possible to append and prepend to arrays. Insertions at a specific index are 311 not supported. 312 * Be careful, appending or prepending an array variable to an array leads to a nested array 313 (for more details please see this [issue](https://github.com/kubernetes-sigs/cluster-api/issues/5944)). 314 315 </aside> 316 317 **Setting variable values in the Cluster** 318 319 After creating a ClusterClass with a variable definition, the user can now provide a value for 320 the variable in the Cluster as in the example below. 321 322 ```yaml 323 apiVersion: cluster.x-k8s.io/v1beta1 324 kind: Cluster 325 metadata: 326 name: my-docker-cluster 327 spec: 328 topology: 329 ... 330 variables: 331 - name: imageRepository 332 value: my.custom.registry 333 ``` 334 335 <aside class="note"> 336 337 <h1>Variable defaulting</h1> 338 339 If the user does not set the value, but the corresponding variable definition in ClusterClass has 340 a default value, the value is automatically added to the variables list. 341 342 </aside> 343 344 ## ClusterClass with custom naming strategies 345 346 The controller needs to generate names for new objects when a Cluster is getting created 347 from a ClusterClass. These names have to be unique for each namespace. The naming 348 strategy enables this by concatenating the cluster name with a random suffix. 349 350 It is possible to provide a custom template for the name generation of ControlPlane, MachineDeployment 351 and MachinePool objects. 352 353 The generated names must comply with the [RFC 1123](https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#dns-label-names) standard. 354 355 ### Defining a custom naming strategy for ControlPlane objects 356 357 The naming strategy for ControlPlane supports the following properties: 358 359 - `template`: Custom template which is used when generating the name of the ControlPlane object. 360 361 The following variables can be referenced in templates: 362 363 - `.cluster.name`: The name of the cluster object. 364 - `.random`: A random alphanumeric string, without vowels, of length 5. 365 366 Example which would match the default behavior: 367 368 ```yaml 369 apiVersion: cluster.x-k8s.io/v1beta1 370 kind: ClusterClass 371 metadata: 372 name: docker-clusterclass-v0.1.0 373 spec: 374 controlPlane: 375 ... 376 namingStrategy: 377 template: "{{ .cluster.name }}-{{ .random }}" 378 ... 379 ``` 380 381 ### Defining a custom naming strategy for MachineDeployment objects 382 383 The naming strategy for MachineDeployments supports the following properties: 384 385 - `template`: Custom template which is used when generating the name of the MachineDeployment object. 386 387 The following variables can be referenced in templates: 388 389 - `.cluster.name`: The name of the cluster object. 390 - `.random`: A random alphanumeric string, without vowels, of length 5. 391 - `.machineDeployment.topologyName`: The name of the MachineDeployment topology (`Cluster.spec.topology.workers.machineDeployments[].name`) 392 393 Example which would match the default behavior: 394 395 ```yaml 396 apiVersion: cluster.x-k8s.io/v1beta1 397 kind: ClusterClass 398 metadata: 399 name: docker-clusterclass-v0.1.0 400 spec: 401 controlPlane: 402 ... 403 workers: 404 machineDeployments: 405 - class: default-worker 406 ... 407 namingStrategy: 408 template: "{{ .cluster.name }}-{{ .machineDeployment.topologyName }}-{{ .random }}" 409 ``` 410 411 ### Defining a custom naming strategy for MachinePool objects 412 413 The naming strategy for MachinePools supports the following properties: 414 415 - `template`: Custom template which is used when generating the name of the MachinePool object. 416 417 The following variables can be referenced in templates: 418 419 - `.cluster.name`: The name of the cluster object. 420 - `.random`: A random alphanumeric string, without vowels, of length 5. 421 - `.machinePool.topologyName`: The name of the MachinePool topology (`Cluster.spec.topology.workers.machinePools[].name`). 422 423 Example which would match the default behavior: 424 425 ```yaml 426 apiVersion: cluster.x-k8s.io/v1beta1 427 kind: ClusterClass 428 metadata: 429 name: docker-clusterclass-v0.1.0 430 spec: 431 controlPlane: 432 ... 433 workers: 434 machinePools: 435 - class: default-worker 436 ... 437 namingStrategy: 438 template: "{{ .cluster.name }}-{{ .machinePool.topologyName }}-{{ .random }}" 439 ``` 440 441 ## Advanced features of ClusterClass with patches 442 443 This section will explain more advanced features of ClusterClass patches. 444 445 ### MachineDeployment & MachinePool variable overrides 446 447 If you want to use many variations of MachineDeployments in Clusters, you can either define 448 a MachineDeployment class for every variation or you can define patches and variables to 449 make a single MachineDeployment class more flexible. The same applies for MachinePools. 450 451 In the following example we make the `instanceType` of a `AWSMachineTemplate` customizable. 452 First we define the `workerMachineType` variable and the corresponding patch: 453 454 ```yaml 455 apiVersion: cluster.x-k8s.io/v1beta1 456 kind: ClusterClass 457 metadata: 458 name: aws-clusterclass-v0.1.0 459 spec: 460 ... 461 variables: 462 - name: workerMachineType 463 required: true 464 schema: 465 openAPIV3Schema: 466 type: string 467 default: t3.large 468 patches: 469 - name: workerMachineType 470 definitions: 471 - selector: 472 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 473 kind: AWSMachineTemplate 474 matchResources: 475 machineDeploymentClass: 476 names: 477 - default-worker 478 jsonPatches: 479 - op: add 480 path: /spec/template/spec/instanceType 481 valueFrom: 482 variable: workerMachineType 483 --- 484 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 485 kind: AWSMachineTemplate 486 metadata: 487 name: aws-clusterclass-v0.1.0-default-worker 488 spec: 489 template: 490 spec: 491 # instanceType: workerMachineType will be set by the patch. 492 iamInstanceProfile: "nodes.cluster-api-provider-aws.sigs.k8s.io" 493 --- 494 ... 495 ``` 496 497 In the Cluster resource the `workerMachineType` variable can then be set cluster-wide and 498 it can also be overridden for an individual MachineDeployment or MachinePool. 499 500 ```yaml 501 apiVersion: cluster.x-k8s.io/v1beta1 502 kind: Cluster 503 metadata: 504 name: my-aws-cluster 505 spec: 506 ... 507 topology: 508 class: aws-clusterclass-v0.1.0 509 version: v1.22.0 510 controlPlane: 511 replicas: 3 512 workers: 513 machineDeployments: 514 - class: "default-worker" 515 name: "md-small-workers" 516 replicas: 3 517 variables: 518 overrides: 519 # Overrides the cluster-wide value with t3.small. 520 - name: workerMachineType 521 value: t3.small 522 # Uses the cluster-wide value t3.large. 523 - class: "default-worker" 524 name: "md-large-workers" 525 replicas: 3 526 variables: 527 - name: workerMachineType 528 value: t3.large 529 ``` 530 531 ### Builtin variables 532 533 In addition to variables specified in the ClusterClass, the following builtin variables can be 534 referenced in patches: 535 - `builtin.cluster.{name,namespace}` 536 - `builtin.cluster.topology.{version,class}` 537 - `builtin.cluster.network.{serviceDomain,services,pods,ipFamily}` 538 - `builtin.controlPlane.{replicas,version,name}` 539 - Please note, these variables are only available when patching control plane or control plane 540 machine templates. 541 - `builtin.controlPlane.machineTemplate.infrastructureRef.name` 542 - Please note, these variables are only available when using a control plane with machines and 543 when patching control plane or control plane machine templates. 544 - `builtin.machineDeployment.{replicas,version,class,name,topologyName}` 545 - Please note, these variables are only available when patching the templates of a MachineDeployment 546 and contain the values of the current `MachineDeployment` topology. 547 - `builtin.machineDeployment.{infrastructureRef.name,bootstrap.configRef.name}` 548 - Please note, these variables are only available when patching the templates of a MachineDeployment 549 and contain the values of the current `MachineDeployment` topology. 550 - `builtin.machinePool.{replicas,version,class,name,topologyName}` 551 - Please note, these variables are only available when patching the templates of a MachinePool 552 and contain the values of the current `MachinePool` topology. 553 - `builtin.machinePool.{infrastructureRef.name,bootstrap.configRef.name}` 554 - Please note, these variables are only available when patching the templates of a MachinePool 555 and contain the values of the current `MachinePool` topology. 556 557 Builtin variables can be referenced just like regular variables, e.g.: 558 ```yaml 559 apiVersion: cluster.x-k8s.io/v1beta1 560 kind: ClusterClass 561 metadata: 562 name: docker-clusterclass-v0.1.0 563 spec: 564 ... 565 patches: 566 - name: clusterName 567 definitions: 568 - selector: 569 ... 570 jsonPatches: 571 - op: add 572 path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/controllerManager/extraArgs/cluster-name 573 valueFrom: 574 variable: builtin.cluster.name 575 ``` 576 577 **Tips & Tricks** 578 579 Builtin variables can be used to dynamically calculate image names. The version used in the patch 580 will always be the same as the one we set in the corresponding MachineDeployment or MachinePool 581 (works the same way with `.builtin.controlPlane.version`). 582 583 ```yaml 584 apiVersion: cluster.x-k8s.io/v1beta1 585 kind: ClusterClass 586 metadata: 587 name: docker-clusterclass-v0.1.0 588 spec: 589 ... 590 patches: 591 - name: customImage 592 description: "Sets the container image that is used for running dockerMachines." 593 definitions: 594 - selector: 595 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 596 kind: DockerMachineTemplate 597 matchResources: 598 machineDeploymentClass: 599 names: 600 - default-worker 601 jsonPatches: 602 - op: add 603 path: /spec/template/spec/customImage 604 valueFrom: 605 template: | 606 kindest/node:{{ .builtin.machineDeployment.version }} 607 ``` 608 609 ### Complex variable types 610 611 Variables can also be objects, maps and arrays. An object is specified with the type `object` and 612 by the schemas of the fields of the object. A map is specified with the type `object` and the schema 613 of the map values. An array is specified via the type `array` and the schema of the array items. 614 615 ```yaml 616 apiVersion: cluster.x-k8s.io/v1beta1 617 kind: ClusterClass 618 metadata: 619 name: docker-clusterclass-v0.1.0 620 spec: 621 ... 622 variables: 623 - name: httpProxy 624 schema: 625 openAPIV3Schema: 626 type: object 627 properties: 628 # Schema of the url field. 629 url: 630 type: string 631 # Schema of the noProxy field. 632 noProxy: 633 type: string 634 - name: mdConfig 635 schema: 636 openAPIV3Schema: 637 type: object 638 additionalProperties: 639 # Schema of the map values. 640 type: object 641 properties: 642 osImage: 643 type: string 644 - name: dnsServers 645 schema: 646 openAPIV3Schema: 647 type: array 648 items: 649 # Schema of the array items. 650 type: string 651 ``` 652 653 Objects, maps and arrays can be used in patches either directly by referencing the variable name, 654 or by accessing individual fields. For example: 655 ```yaml 656 apiVersion: cluster.x-k8s.io/v1beta1 657 kind: ClusterClass 658 metadata: 659 name: docker-clusterclass-v0.1.0 660 spec: 661 ... 662 jsonPatches: 663 - op: add 664 path: /spec/template/spec/httpProxy/url 665 valueFrom: 666 # Use the url field of the httpProxy variable. 667 variable: httpProxy.url 668 - op: add 669 path: /spec/template/spec/customImage 670 valueFrom: 671 # Use the osImage field of the mdConfig variable for the current MD class. 672 template: "{{ (index .mdConfig .builtin.machineDeployment.class).osImage }}" 673 - op: add 674 path: /spec/template/spec/dnsServers 675 valueFrom: 676 # Use the entire dnsServers array. 677 variable: dnsServers 678 - op: add 679 path: /spec/template/spec/dnsServer 680 valueFrom: 681 # Use the first item of the dnsServers array. 682 variable: dnsServers[0] 683 ``` 684 685 **Tips & Tricks** 686 687 Complex variables can be used to make references in templates configurable, e.g. the `identityRef` used in `AzureCluster`. 688 Of course it's also possible to only make the name of the reference configurable, including restricting the valid values 689 to a pre-defined enum. 690 691 ```yaml 692 apiVersion: cluster.x-k8s.io/v1beta1 693 kind: ClusterClass 694 metadata: 695 name: azure-clusterclass-v0.1.0 696 spec: 697 ... 698 variables: 699 - name: clusterIdentityRef 700 schema: 701 openAPIV3Schema: 702 type: object 703 properties: 704 kind: 705 type: string 706 name: 707 type: string 708 ``` 709 710 Even if OpenAPI schema allows defining free form objects, e.g. 711 712 ```yaml 713 variables: 714 - name: freeFormObject 715 schema: 716 openAPIV3Schema: 717 type: object 718 ``` 719 720 User should be aware that the lack of the validation of users provided data could lead to problems 721 when those values are used in patch or when the generated templates are created (see e.g. 722 [6135](https://github.com/kubernetes-sigs/cluster-api/issues/6135)). 723 724 As a consequence we recommend avoiding this practice while we are considering alternatives to make 725 it explicit for the ClusterClass authors to opt-in in this feature, thus accepting the implied risks. 726 727 ### Using variable values in JSON patches 728 729 We already saw above that it's possible to use variable values in JSON patches. It's also 730 possible to calculate values via Go templating or to use hard-coded values. 731 732 ```yaml 733 apiVersion: cluster.x-k8s.io/v1beta1 734 kind: ClusterClass 735 metadata: 736 name: docker-clusterclass-v0.1.0 737 spec: 738 ... 739 patches: 740 - name: etcdImageTag 741 definitions: 742 - selector: 743 ... 744 jsonPatches: 745 - op: add 746 path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/etcd 747 valueFrom: 748 # This template is first rendered with Go templating, then parsed by 749 # a YAML/JSON parser and then used as value of the JSON patch. 750 # For example, if the variable etcdImageTag is set to `3.5.1-0` the 751 # .../clusterConfiguration/etcd field will be set to: 752 # {"local": {"imageTag": "3.5.1-0"}} 753 template: | 754 local: 755 imageTag: {{ .etcdImageTag }} 756 - name: imageRepository 757 definitions: 758 - selector: 759 ... 760 jsonPatches: 761 - op: add 762 path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/imageRepository 763 # This hard-coded value is used directly as value of the JSON patch. 764 value: "my.custom.registry" 765 ``` 766 767 <aside class="note"> 768 769 <h1>Variable paths</h1> 770 771 * Paths can be used in `.valueFrom.template` and `.valueFrom.variable` to access nested fields of arrays and objects. 772 * `.` is used to access a field of an object, e.g. `httpProxy.url`. 773 * `[i]` is used to access an array element, e.g. `dnsServers[0]`. 774 * Because of the way Go templates work, the paths in templates have to start with a dot. 775 776 </aside> 777 778 **Tips & Tricks** 779 780 Templates can be used to implement defaulting behavior during JSON patch value calculation. This can be used if the simple 781 constant default value which can be specified in the schema is not enough. 782 ```yaml 783 valueFrom: 784 # If .vnetName is set, it is used. Otherwise, we will use `{{.builtin.cluster.name}}-vnet`. 785 template: "{{ if .vnetName }}{{.vnetName}}{{else}}{{.builtin.cluster.name}}-vnet{{end}}" 786 ``` 787 When writing templates, a subset of functions from [the Sprig library](https://masterminds.github.io/sprig/) can be used to 788 write expressions, e.g., `{{ .name | upper }}`. Only functions that are guaranteed to evaluate to the same result 789 for a given input are allowed (e.g. `upper` or `max` can be used, while `now` or `randAlpha` cannot be used). 790 791 ### Optional patches 792 793 Patches can also be conditionally enabled. This can be done by configuring a Go template via `enabledIf`. 794 The patch is then only applied if the Go template evaluates to `true`. In the following example the `httpProxy` 795 patch is only applied if the `httpProxy` variable is set (and not empty). 796 797 ```yaml 798 apiVersion: cluster.x-k8s.io/v1beta1 799 kind: ClusterClass 800 metadata: 801 name: docker-clusterclass-v0.1.0 802 spec: 803 ... 804 variables: 805 - name: httpProxy 806 schema: 807 openAPIV3Schema: 808 type: string 809 patches: 810 - name: httpProxy 811 enabledIf: "{{ if .httpProxy }}true{{end}}" 812 definitions: 813 ... 814 ``` 815 816 **Tips & Tricks**: 817 818 Hard-coded values can be used to test the impact of a patch during development, gradually roll out patches, etc. . 819 ```yaml 820 enabledIf: false 821 ``` 822 823 A boolean variable can be used to enable/disable a patch (or "feature"). This can have opt-in or opt-out behavior 824 depending on the default value of the variable. 825 ```yaml 826 enabledIf: "{{ .httpProxyEnabled }}" 827 ``` 828 829 Of course the same is possible by adding a boolean variable to a configuration object. 830 ```yaml 831 enabledIf: "{{ .httpProxy.enabled }}" 832 ``` 833 834 Builtin variables can be leveraged to apply a patch only for a specific Kubernetes version. 835 ```yaml 836 enabledIf: '{{ semverCompare "1.21.1" .builtin.controlPlane.version }}' 837 ``` 838 839 With `semverCompare` and `coalesce` a feature can be enabled in newer versions of Kubernetes for both KubeadmConfigTemplate and KubeadmControlPlane. 840 ```yaml 841 enabledIf: '{{ semverCompare "^1.22.0" (coalesce .builtin.controlPlane.version .builtin.machineDeployment.version )}}' 842 ``` 843 844 <aside class="note"> 845 846 <h1>Builtin Variables</h1> 847 848 Please be aware that while you can use builtin variables, if you use for example a MachineDeployment-specific variable this 849 can mean that patches are only applied to some MachineDeployments. `enabledIf` is evaluated for each template that should be patched 850 individually. 851 852 </aside> 853 854 ### Version-aware patches 855 856 In some cases the ClusterClass authors want a patch to be computed according to the Kubernetes version in use. 857 858 While this is not a problem "per se" and it does not differ from writing any other patch, it is important 859 to keep in mind that there could be different Kubernetes version in a Cluster at any time, all of them 860 accessible via built in variables: 861 862 - `builtin.cluster.topology.version` defines the Kubernetes version from `cluster.topology`, and it acts 863 as the desired Kubernetes version for the entire cluster. However, during an upgrade workflow it could happen that 864 some objects in the Cluster are still at the older version. 865 - `builtin.controlPlane.version`, represent the desired version for the control plane object; usually this 866 version changes immediately after `cluster.topology.version` is updated (unless there are other operations 867 in progress preventing the upgrade to start). 868 - `builtin.machineDeployment.version`, represent the desired version for each specific MachineDeployment object; 869 this version changes only after the upgrade for the control plane is completed, and in case of many 870 MachineDeployments in the same cluster, they are upgraded sequentially. 871 - `builtin.machinePool.version`, represent the desired version for each specific MachinePool object; 872 this version changes only after the upgrade for the control plane is completed, and in case of many 873 MachinePools in the same cluster, they are upgraded sequentially. 874 875 This info should provide the bases for developing version-aware patches, allowing the patch author to determine when a 876 patch should adapt to the new Kubernetes version by choosing one of the above variables. In practice the 877 following rules applies to the most common use cases: 878 879 - When developing a version-aware patch for the control plane, `builtin.controlPlane.version` must be used. 880 - When developing a version-aware patch for MachineDeployments, `builtin.machineDeployment.version` must be used. 881 - When developing a version-aware patch for MachinePools, `builtin.machinePool.version` must be used. 882 883 **Tips & Tricks**: 884 885 Sometimes users need to define variables to be used by version-aware patches, and in this case it is important 886 to keep in mind that there could be different Kubernetes versions in a Cluster at any time. 887 888 A simple approach to solve this problem is to define a map of version-aware variables, with the key of each item 889 being the Kubernetes version. Patch could then use the proper builtin variables as a lookup entry to fetch 890 the corresponding values for the Kubernetes version in use by each object. 891 892 ## JSON patches tips & tricks 893 894 JSON patches specification [RFC6902] requires that the target of 895 add operation must exist. 896 897 As a consequence ClusterClass authors should pay special attention when the following 898 conditions apply in order to prevent errors when a patch is applied: 899 900 * the patch tries to `add` a value to an **array** (which is a **slice** in the corresponding go struct) 901 * the slice was defined with `omitempty` 902 * the slice currently does not exist 903 904 A workaround in this particular case is to create the array in the patch instead of adding to the non-existing one. 905 When creating the slice, existing values would be overwritten so this should only be used when it does not exist. 906 907 The following example shows both cases to consider while writing a patch for adding a value to a slice. 908 This patch targets to add a file to the `files` slice of a `KubeadmConfigTemplate` which has [omitempty](https://github.com/kubernetes-sigs/cluster-api/blob/main/bootstrap/kubeadm/api/v1beta1/kubeadmconfig_types.go#L54) set. 909 910 {{#tabs name:"tab-configuration-patches" tabs:"Add to existing slice,Create slice"}} 911 {{#tab Add to existing slice}} 912 913 This patch **requires** the key `.spec.template.spec.files` to exist to succeed. 914 915 ```yaml 916 apiVersion: cluster.x-k8s.io/v1beta1 917 kind: ClusterClass 918 metadata: 919 name: my-clusterclass 920 spec: 921 ... 922 patches: 923 - name: add file 924 definitions: 925 - selector: 926 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 927 kind: KubeadmConfigTemplate 928 jsonPatches: 929 - op: add 930 path: /spec/template/spec/files/- 931 value: 932 content: Some content. 933 path: /some/file 934 --- 935 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 936 kind: KubeadmConfigTemplate 937 metadata: 938 name: "quick-start-default-worker-bootstraptemplate" 939 spec: 940 template: 941 spec: 942 ... 943 files: 944 - content: Some other content 945 path: /some/other/file 946 ``` 947 948 {{#/tab }} 949 {{#tab Create slice}} 950 951 This patch would **overwrite** an existing slice at `.spec.template.spec.files`. 952 953 ```yaml 954 apiVersion: cluster.x-k8s.io/v1beta1 955 kind: ClusterClass 956 metadata: 957 name: my-clusterclass 958 spec: 959 ... 960 patches: 961 - name: add file 962 definitions: 963 - selector: 964 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 965 kind: KubeadmConfigTemplate 966 jsonPatches: 967 - op: add 968 path: /spec/template/spec/files 969 value: 970 - content: Some content. 971 path: /some/file 972 --- 973 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 974 kind: KubeadmConfigTemplate 975 metadata: 976 name: "quick-start-default-worker-bootstraptemplate" 977 spec: 978 template: 979 spec: 980 ... 981 ``` 982 983 {{#/tab }} 984 {{#/tabs }} 985 986 <!-- links --> 987 [Changing a ClusterClass]: ./change-clusterclass.md 988 [clusterctl alpha topology plan]: ../../../clusterctl/commands/alpha-topology-plan.md 989 [RFC6902]: https://datatracker.ietf.org/doc/html/rfc6902#appendix-A.12