sigs.k8s.io/cluster-api@v1.6.3/docs/book/src/tasks/experimental-features/cluster-class/write-clusterclass.md (about) 1 2 # Writing a ClusterClass 3 4 A ClusterClass becomes more useful and valuable when it can be used to create many Cluster of a similar 5 shape. The goal of this document is to explain how ClusterClasses can be written in a way that they are 6 flexible enough to be used in as many Clusters as possible by supporting variants of the same base Cluster shape. 7 8 **Table of Contents** 9 10 * [Basic ClusterClass](#basic-clusterclass) 11 * [ClusterClass with MachineHealthChecks](#clusterclass-with-machinehealthchecks) 12 * [ClusterClass with patches](#clusterclass-with-patches) 13 * [ClusterClass with custom naming strategies](#clusterclass-with-custom-naming-strategies) 14 * [Defining a custom naming strategy for ControlPlane objects](#defining-a-custom-naming-strategy-for-controlplane-objects) 15 * [Defining a custom naming strategy for MachineDeployment objects](#defining-a-custom-naming-strategy-for-machinedeployment-objects) 16 * [Defining a custom naming strategy for MachinePool objects](#defining-a-custom-naming-strategy-for-machinepool-objects) 17 * [Advanced features of ClusterClass with patches](#advanced-features-of-clusterclass-with-patches) 18 * [MachineDeployment variable overrides](#machinedeployment-variable-overrides) 19 * [Builtin variables](#builtin-variables) 20 * [Complex variable types](#complex-variable-types) 21 * [Using variable values in JSON patches](#using-variable-values-in-json-patches) 22 * [Optional patches](#optional-patches) 23 * [Version-aware patches](#version-aware-patches) 24 * [JSON patches tips & tricks](#json-patches-tips--tricks) 25 26 ## Basic ClusterClass 27 28 The following example shows a basic ClusterClass. It contains templates to shape the control plane, 29 infrastructure and workers of a Cluster. When a Cluster is using this ClusterClass, the templates 30 are used to generate the objects of the managed topology of the Cluster. 31 32 ```yaml 33 apiVersion: cluster.x-k8s.io/v1beta1 34 kind: ClusterClass 35 metadata: 36 name: docker-clusterclass-v0.1.0 37 spec: 38 controlPlane: 39 ref: 40 apiVersion: controlplane.cluster.x-k8s.io/v1beta1 41 kind: KubeadmControlPlaneTemplate 42 name: docker-clusterclass-v0.1.0 43 namespace: default 44 machineInfrastructure: 45 ref: 46 kind: DockerMachineTemplate 47 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 48 name: docker-clusterclass-v0.1.0 49 namespace: default 50 infrastructure: 51 ref: 52 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 53 kind: DockerClusterTemplate 54 name: docker-clusterclass-v0.1.0-control-plane 55 namespace: default 56 workers: 57 machineDeployments: 58 - class: default-worker 59 template: 60 bootstrap: 61 ref: 62 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 63 kind: KubeadmConfigTemplate 64 name: docker-clusterclass-v0.1.0-default-worker 65 namespace: default 66 infrastructure: 67 ref: 68 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 69 kind: DockerMachineTemplate 70 name: docker-clusterclass-v0.1.0-default-worker 71 namespace: default 72 ``` 73 74 The following example shows a Cluster using this ClusterClass. In this case a `KubeadmControlPlane` 75 with the corresponding `DockerMachineTemplate`, a `DockerCluster` and a `MachineDeployment` with 76 the corresponding `KubeadmConfigTemplate` and `DockerMachineTemplate` will be created. This basic 77 ClusterClass is already very flexible. Via the topology on the Cluster the following can be configured: 78 * `.spec.topology.version`: the Kubernetes version of the Cluster 79 * `.spec.topology.controlPlane`: ControlPlane replicas and their metadata 80 * `.spec.topology.workers`: MachineDeployments and their replicas, metadata and failure domain 81 82 ```yaml 83 apiVersion: cluster.x-k8s.io/v1beta1 84 kind: Cluster 85 metadata: 86 name: my-docker-cluster 87 spec: 88 topology: 89 class: docker-clusterclass-v0.1.0 90 version: v1.22.4 91 controlPlane: 92 replicas: 3 93 metadata: 94 labels: 95 cpLabel: cpLabelValue 96 annotations: 97 cpAnnotation: cpAnnotationValue 98 workers: 99 machineDeployments: 100 - class: default-worker 101 name: md-0 102 replicas: 4 103 metadata: 104 labels: 105 mdLabel: mdLabelValue 106 annotations: 107 mdAnnotation: mdAnnotationValue 108 failureDomain: region 109 ``` 110 111 Best practices: 112 * The ClusterClass name should be generic enough to make sense across multiple clusters, i.e. a 113 name which corresponds to a single Cluster, e.g. "my-cluster", is not recommended. 114 * Try to keep the ClusterClass names short and consistent (if you publish multiple ClusterClasses). 115 * As a ClusterClass usually evolves over time and you might want to rebase Clusters from one version 116 of a ClusterClass to another, consider including a version suffix in the ClusterClass name. 117 For more information about changing a ClusterClass please see: [Changing a ClusterClass]. 118 * Prefix the templates used in a ClusterClass with the name of the ClusterClass. 119 * Don't reuse the same template in multiple ClusterClasses. This is automatically taken care 120 of by prefixing the templates with the name of the ClusterClass. 121 122 <aside class="note"> 123 124 For a full example ClusterClass for CAPD you can take a look at 125 [clusterclass-quickstart.yaml](https://github.com/kubernetes-sigs/cluster-api/blob/main/test/infrastructure/docker/templates/clusterclass-quick-start.yaml) 126 (which is also used in the CAPD quickstart with ClusterClass). 127 128 </aside> 129 130 <aside class="note"> 131 132 <h1>Tip: clusterctl alpha topology plan</h1> 133 134 The `clusterctl alpha topology plan` command can be used to test ClusterClasses; the output will show 135 you how the resulting Cluster will look like, but without actually creating it. 136 For more details please see: [clusterctl alpha topology plan]. 137 138 </aside> 139 140 ## ClusterClass with MachineHealthChecks 141 142 `MachineHealthChecks` can be configured in the ClusterClass for the control plane and for a 143 MachineDeployment class. The following configuration makes sure a `MachineHealthCheck` is 144 created for the control plane and for every `MachineDeployment` using the `default-worker` class. 145 146 ```yaml 147 apiVersion: cluster.x-k8s.io/v1beta1 148 kind: ClusterClass 149 metadata: 150 name: docker-clusterclass-v0.1.0 151 spec: 152 controlPlane: 153 ... 154 machineHealthCheck: 155 maxUnhealthy: 33% 156 nodeStartupTimeout: 15m 157 unhealthyConditions: 158 - type: Ready 159 status: Unknown 160 timeout: 300s 161 - type: Ready 162 status: "False" 163 timeout: 300s 164 workers: 165 machineDeployments: 166 - class: default-worker 167 ... 168 machineHealthCheck: 169 unhealthyRange: "[0-2]" 170 nodeStartupTimeout: 10m 171 unhealthyConditions: 172 - type: Ready 173 status: Unknown 174 timeout: 300s 175 - type: Ready 176 status: "False" 177 timeout: 300s 178 ``` 179 180 ## ClusterClass with patches 181 182 As shown above, basic ClusterClasses are already very powerful. But there are cases where 183 more powerful mechanisms are required. Let's assume you want to manage multiple Clusters 184 with the same ClusterClass, but they require different values for a field in one of the 185 referenced templates of a ClusterClass. 186 187 A concrete example would be to deploy Clusters with different registries. In this case, 188 every cluster needs a Cluster-specific value for `.spec.kubeadmConfigSpec.clusterConfiguration.imageRepository` 189 in `KubeadmControlPlane`. Use cases like this can be implemented with ClusterClass patches. 190 191 **Defining variables in the ClusterClass** 192 193 The following example shows how variables can be defined in the ClusterClass. 194 A variable definition specifies the name and the schema of a variable and if it is 195 required. The schema defines how a variable is defaulted and validated. It supports 196 a subset of the schema of CRDs. For more information please see the [godoc](https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api/cluster.x-k8s.io/ClusterClass/v1beta1#spec-variables-schema-openAPIV3Schema). 197 198 ```yaml 199 apiVersion: cluster.x-k8s.io/v1beta1 200 kind: ClusterClass 201 metadata: 202 name: docker-clusterclass-v0.1.0 203 spec: 204 ... 205 variables: 206 - name: imageRepository 207 required: true 208 schema: 209 openAPIV3Schema: 210 type: string 211 description: ImageRepository is the container registry to pull images from. 212 default: registry.k8s.io 213 example: registry.k8s.io 214 ``` 215 216 <aside class="note"> 217 218 <h1>Supported types</h1> 219 220 The following basic types are supported: `string`, `integer`, `number` and `boolean`. We are also 221 supporting complex types, please see the [complex variable types](#complex-variable-types) section. 222 223 </aside> 224 225 **Defining patches in the ClusterClass** 226 227 The variable can then be used in a patch to set a field on a template referenced in the ClusterClass. 228 The `selector` specifies on which template the patch should be applied. `jsonPatches` specifies which JSON 229 patches should be applied to that template. In this case we set the `imageRepository` field of the 230 `KubeadmControlPlaneTemplate` to the value of the variable `imageRepository`. For more information 231 please see the [godoc](https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api/cluster.x-k8s.io/ClusterClass/v1beta1#spec-patches-definitions). 232 233 ```yaml 234 apiVersion: cluster.x-k8s.io/v1beta1 235 kind: ClusterClass 236 metadata: 237 name: docker-clusterclass-v0.1.0 238 spec: 239 ... 240 patches: 241 - name: imageRepository 242 definitions: 243 - selector: 244 apiVersion: controlplane.cluster.x-k8s.io/v1beta1 245 kind: KubeadmControlPlaneTemplate 246 matchResources: 247 controlPlane: true 248 jsonPatches: 249 - op: add 250 path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/imageRepository 251 valueFrom: 252 variable: imageRepository 253 ``` 254 255 <aside class="note"> 256 257 <h1>Writing JSON patches</h1> 258 259 * Only fields below `/spec` can be patched. 260 * Only `add`, `remove` and `replace` operations are supported. 261 * It's only possible to append and prepend to arrays. Insertions at a specific index are 262 not supported. 263 * Be careful, appending or prepending an array variable to an array leads to a nested array 264 (for more details please see this [issue](https://github.com/kubernetes-sigs/cluster-api/issues/5944)). 265 266 </aside> 267 268 **Setting variable values in the Cluster** 269 270 After creating a ClusterClass with a variable definition, the user can now provide a value for 271 the variable in the Cluster as in the example below. 272 273 ```yaml 274 apiVersion: cluster.x-k8s.io/v1beta1 275 kind: Cluster 276 metadata: 277 name: my-docker-cluster 278 spec: 279 topology: 280 ... 281 variables: 282 - name: imageRepository 283 value: my.custom.registry 284 ``` 285 286 <aside class="note"> 287 288 <h1>Variable defaulting</h1> 289 290 If the user does not set the value, but the corresponding variable definition in ClusterClass has 291 a default value, the value is automatically added to the variables list. 292 293 </aside> 294 295 ## ClusterClass with custom naming strategies 296 297 The controller needs to generate names for new objects when a Cluster is getting created 298 from a ClusterClass. These names have to be unique for each namespace. The naming 299 strategy enables this by concatenating the cluster name with a random suffix. 300 301 It is possible to provide a custom template for the name generation of ControlPlane, MachineDeployment 302 and MachinePool objects. 303 304 The generated names must comply with the [RFC 1123](https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#dns-label-names) standard. 305 306 ### Defining a custom naming strategy for ControlPlane objects 307 308 The naming strategy for ControlPlane supports the following properties: 309 310 - `template`: Custom template which is used when generating the name of the ControlPlane object. 311 312 The following variables can be referenced in templates: 313 314 - `.cluster.name`: The name of the cluster object. 315 - `.random`: A random alphanumeric string, without vowels, of length 5. 316 317 Example which would match the default behavior: 318 319 ```yaml 320 apiVersion: cluster.x-k8s.io/v1beta1 321 kind: ClusterClass 322 metadata: 323 name: docker-clusterclass-v0.1.0 324 spec: 325 controlPlane: 326 ... 327 namingStrategy: 328 template: "{{ .cluster.name }}-{{ .random }}" 329 ... 330 ``` 331 332 ### Defining a custom naming strategy for MachineDeployment objects 333 334 The naming strategy for MachineDeployments supports the following properties: 335 336 - `template`: Custom template which is used when generating the name of the MachineDeployment object. 337 338 The following variables can be referenced in templates: 339 340 - `.cluster.name`: The name of the cluster object. 341 - `.random`: A random alphanumeric string, without vowels, of length 5. 342 - `.machineDeployment.topologyName`: The name of the MachineDeployment topology (`Cluster.spec.topology.workers.machineDeployments[].name`) 343 344 Example which would match the default behavior: 345 346 ```yaml 347 apiVersion: cluster.x-k8s.io/v1beta1 348 kind: ClusterClass 349 metadata: 350 name: docker-clusterclass-v0.1.0 351 spec: 352 controlPlane: 353 ... 354 workers: 355 machineDeployments: 356 - class: default-worker 357 ... 358 namingStrategy: 359 template: "{{ .cluster.name }}-{{ .machineDeployment.topologyName }}-{{ .random }}" 360 ``` 361 362 ### Defining a custom naming strategy for MachinePool objects 363 364 The naming strategy for MachinePools supports the following properties: 365 366 - `template`: Custom template which is used when generating the name of the MachinePool object. 367 368 The following variables can be referenced in templates: 369 370 - `.cluster.name`: The name of the cluster object. 371 - `.random`: A random alphanumeric string, without vowels, of length 5. 372 - `.machinePool.topologyName`: The name of the MachinePool topology (`Cluster.spec.topology.workers.machinePools[].name`). 373 374 Example which would match the default behavior: 375 376 ```yaml 377 apiVersion: cluster.x-k8s.io/v1beta1 378 kind: ClusterClass 379 metadata: 380 name: docker-clusterclass-v0.1.0 381 spec: 382 controlPlane: 383 ... 384 workers: 385 machinePools: 386 - class: default-worker 387 ... 388 namingStrategy: 389 template: "{{ .cluster.name }}-{{ .machinePool.topologyName }}-{{ .random }}" 390 ``` 391 392 ## Advanced features of ClusterClass with patches 393 394 This section will explain more advanced features of ClusterClass patches. 395 396 ### MachineDeployment variable overrides 397 398 If you want to use many variations of MachineDeployments in Clusters, you can either define 399 a MachineDeployment class for every variation or you can define patches and variables to 400 make a single MachineDeployment class more flexible. 401 402 In the following example we make the `instanceType` of a `AWSMachineTemplate` customizable. 403 First we define the `workerMachineType` variable and the corresponding patch: 404 405 ```yaml 406 apiVersion: cluster.x-k8s.io/v1beta1 407 kind: ClusterClass 408 metadata: 409 name: aws-clusterclass-v0.1.0 410 spec: 411 ... 412 variables: 413 - name: workerMachineType 414 required: true 415 schema: 416 openAPIV3Schema: 417 type: string 418 default: t3.large 419 patches: 420 - name: workerMachineType 421 definitions: 422 - selector: 423 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 424 kind: AWSMachineTemplate 425 matchResources: 426 machineDeploymentClass: 427 names: 428 - default-worker 429 jsonPatches: 430 - op: add 431 path: /spec/template/spec/instanceType 432 valueFrom: 433 variable: workerMachineType 434 --- 435 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 436 kind: AWSMachineTemplate 437 metadata: 438 name: aws-clusterclass-v0.1.0-default-worker 439 spec: 440 template: 441 spec: 442 # instanceType: workerMachineType will be set by the patch. 443 iamInstanceProfile: "nodes.cluster-api-provider-aws.sigs.k8s.io" 444 --- 445 ... 446 ``` 447 448 In the Cluster resource the `workerMachineType` variable can then be set cluster-wide and 449 it can also be overridden for an individual MachineDeployment. 450 451 ```yaml 452 apiVersion: cluster.x-k8s.io/v1beta1 453 kind: Cluster 454 metadata: 455 name: my-aws-cluster 456 spec: 457 ... 458 topology: 459 class: aws-clusterclass-v0.1.0 460 version: v1.22.0 461 controlPlane: 462 replicas: 3 463 workers: 464 machineDeployments: 465 - class: "default-worker" 466 name: "md-small-workers" 467 replicas: 3 468 variables: 469 overrides: 470 # Overrides the cluster-wide value with t3.small. 471 - name: workerMachineType 472 value: t3.small 473 # Uses the cluster-wide value t3.large. 474 - class: "default-worker" 475 name: "md-large-workers" 476 replicas: 3 477 variables: 478 - name: workerMachineType 479 value: t3.large 480 ``` 481 482 ### Builtin variables 483 484 In addition to variables specified in the ClusterClass, the following builtin variables can be 485 referenced in patches: 486 - `builtin.cluster.{name,namespace}` 487 - `builtin.cluster.topology.{version,class}` 488 - `builtin.cluster.network.{serviceDomain,services,pods,ipFamily}` 489 - `builtin.controlPlane.{replicas,version,name}` 490 - Please note, these variables are only available when patching control plane or control plane 491 machine templates. 492 - `builtin.controlPlane.machineTemplate.infrastructureRef.name` 493 - Please note, these variables are only available when using a control plane with machines and 494 when patching control plane or control plane machine templates. 495 - `builtin.machineDeployment.{replicas,version,class,name,topologyName}` 496 - Please note, these variables are only available when patching the templates of a MachineDeployment 497 and contain the values of the current `MachineDeployment` topology. 498 - `builtin.machineDeployment.{infrastructureRef.name,bootstrap.configRef.name}` 499 - Please note, these variables are only available when patching the templates of a MachineDeployment 500 and contain the values of the current `MachineDeployment` topology. 501 502 Builtin variables can be referenced just like regular variables, e.g.: 503 ```yaml 504 apiVersion: cluster.x-k8s.io/v1beta1 505 kind: ClusterClass 506 metadata: 507 name: docker-clusterclass-v0.1.0 508 spec: 509 ... 510 patches: 511 - name: clusterName 512 definitions: 513 - selector: 514 ... 515 jsonPatches: 516 - op: add 517 path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/controllerManager/extraArgs/cluster-name 518 valueFrom: 519 variable: builtin.cluster.name 520 ``` 521 522 **Tips & Tricks** 523 524 Builtin variables can be used to dynamically calculate image names. The version used in the patch 525 will always be the same as the one we set in the corresponding MachineDeployment (works the same way 526 with `.builtin.controlPlane.version`). 527 528 ```yaml 529 apiVersion: cluster.x-k8s.io/v1beta1 530 kind: ClusterClass 531 metadata: 532 name: docker-clusterclass-v0.1.0 533 spec: 534 ... 535 patches: 536 - name: customImage 537 description: "Sets the container image that is used for running dockerMachines." 538 definitions: 539 - selector: 540 apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 541 kind: DockerMachineTemplate 542 matchResources: 543 machineDeploymentClass: 544 names: 545 - default-worker 546 jsonPatches: 547 - op: add 548 path: /spec/template/spec/customImage 549 valueFrom: 550 template: | 551 kindest/node:{{ .builtin.machineDeployment.version }} 552 ``` 553 554 ### Complex variable types 555 556 Variables can also be objects, maps and arrays. An object is specified with the type `object` and 557 by the schemas of the fields of the object. A map is specified with the type `object` and the schema 558 of the map values. An array is specified via the type `array` and the schema of the array items. 559 560 ```yaml 561 apiVersion: cluster.x-k8s.io/v1beta1 562 kind: ClusterClass 563 metadata: 564 name: docker-clusterclass-v0.1.0 565 spec: 566 ... 567 variables: 568 - name: httpProxy 569 schema: 570 openAPIV3Schema: 571 type: object 572 properties: 573 # Schema of the url field. 574 url: 575 type: string 576 # Schema of the noProxy field. 577 noProxy: 578 type: string 579 - name: mdConfig 580 schema: 581 openAPIV3Schema: 582 type: object 583 additionalProperties: 584 # Schema of the map values. 585 type: object 586 properties: 587 osImage: 588 type: string 589 - name: dnsServers 590 schema: 591 openAPIV3Schema: 592 type: array 593 items: 594 # Schema of the array items. 595 type: string 596 ``` 597 598 Objects, maps and arrays can be used in patches either directly by referencing the variable name, 599 or by accessing individual fields. For example: 600 ```yaml 601 apiVersion: cluster.x-k8s.io/v1beta1 602 kind: ClusterClass 603 metadata: 604 name: docker-clusterclass-v0.1.0 605 spec: 606 ... 607 jsonPatches: 608 - op: add 609 path: /spec/template/spec/httpProxy/url 610 valueFrom: 611 # Use the url field of the httpProxy variable. 612 variable: httpProxy.url 613 - op: add 614 path: /spec/template/spec/customImage 615 valueFrom: 616 # Use the osImage field of the mdConfig variable for the current MD class. 617 template: "{{ (index .mdConfig .builtin.machineDeployment.class).osImage }}" 618 - op: add 619 path: /spec/template/spec/dnsServers 620 valueFrom: 621 # Use the entire dnsServers array. 622 variable: dnsServers 623 - op: add 624 path: /spec/template/spec/dnsServer 625 valueFrom: 626 # Use the first item of the dnsServers array. 627 variable: dnsServers[0] 628 ``` 629 630 **Tips & Tricks** 631 632 Complex variables can be used to make references in templates configurable, e.g. the `identityRef` used in `AzureCluster`. 633 Of course it's also possible to only make the name of the reference configurable, including restricting the valid values 634 to a pre-defined enum. 635 636 ```yaml 637 apiVersion: cluster.x-k8s.io/v1beta1 638 kind: ClusterClass 639 metadata: 640 name: azure-clusterclass-v0.1.0 641 spec: 642 ... 643 variables: 644 - name: clusterIdentityRef 645 schema: 646 openAPIV3Schema: 647 type: object 648 properties: 649 kind: 650 type: string 651 name: 652 type: string 653 ``` 654 655 Even if OpenAPI schema allows defining free form objects, e.g. 656 657 ```yaml 658 variables: 659 - name: freeFormObject 660 schema: 661 openAPIV3Schema: 662 type: object 663 ``` 664 665 User should be aware that the lack of the validation of users provided data could lead to problems 666 when those values are used in patch or when the generated templates are created (see e.g. 667 [6135](https://github.com/kubernetes-sigs/cluster-api/issues/6135)). 668 669 As a consequence we recommend avoiding this practice while we are considering alternatives to make 670 it explicit for the ClusterClass authors to opt-in in this feature, thus accepting the implied risks. 671 672 ### Using variable values in JSON patches 673 674 We already saw above that it's possible to use variable values in JSON patches. It's also 675 possible to calculate values via Go templating or to use hard-coded values. 676 677 ```yaml 678 apiVersion: cluster.x-k8s.io/v1beta1 679 kind: ClusterClass 680 metadata: 681 name: docker-clusterclass-v0.1.0 682 spec: 683 ... 684 patches: 685 - name: etcdImageTag 686 definitions: 687 - selector: 688 ... 689 jsonPatches: 690 - op: add 691 path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/etcd 692 valueFrom: 693 # This template is first rendered with Go templating, then parsed by 694 # a YAML/JSON parser and then used as value of the JSON patch. 695 # For example, if the variable etcdImageTag is set to `3.5.1-0` the 696 # .../clusterConfiguration/etcd field will be set to: 697 # {"local": {"imageTag": "3.5.1-0"}} 698 template: | 699 local: 700 imageTag: {{ .etcdImageTag }} 701 - name: imageRepository 702 definitions: 703 - selector: 704 ... 705 jsonPatches: 706 - op: add 707 path: /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/imageRepository 708 # This hard-coded value is used directly as value of the JSON patch. 709 value: "my.custom.registry" 710 ``` 711 712 <aside class="note"> 713 714 <h1>Variable paths</h1> 715 716 * Paths can be used in `.valueFrom.template` and `.valueFrom.variable` to access nested fields of arrays and objects. 717 * `.` is used to access a field of an object, e.g. `httpProxy.url`. 718 * `[i]` is used to access an array element, e.g. `dnsServers[0]`. 719 * Because of the way Go templates work, the paths in templates have to start with a dot. 720 721 </aside> 722 723 **Tips & Tricks** 724 725 Templates can be used to implement defaulting behavior during JSON patch value calculation. This can be used if the simple 726 constant default value which can be specified in the schema is not enough. 727 ```yaml 728 valueFrom: 729 # If .vnetName is set, it is used. Otherwise, we will use `{{.builtin.cluster.name}}-vnet`. 730 template: "{{ if .vnetName }}{{.vnetName}}{{else}}{{.builtin.cluster.name}}-vnet{{end}}" 731 ``` 732 When writing templates, a subset of functions from [the Sprig library](https://masterminds.github.io/sprig/) can be used to 733 write expressions, e.g., `{{ .name | upper }}`. Only functions that are guaranteed to evaluate to the same result 734 for a given input are allowed (e.g. `upper` or `max` can be used, while `now` or `randAlpha` cannot be used). 735 736 ### Optional patches 737 738 Patches can also be conditionally enabled. This can be done by configuring a Go template via `enabledIf`. 739 The patch is then only applied if the Go template evaluates to `true`. In the following example the `httpProxy` 740 patch is only applied if the `httpProxy` variable is set (and not empty). 741 742 ```yaml 743 apiVersion: cluster.x-k8s.io/v1beta1 744 kind: ClusterClass 745 metadata: 746 name: docker-clusterclass-v0.1.0 747 spec: 748 ... 749 variables: 750 - name: httpProxy 751 schema: 752 openAPIV3Schema: 753 type: string 754 patches: 755 - name: httpProxy 756 enabledIf: "{{ if .httpProxy }}true{{end}}" 757 definitions: 758 ... 759 ``` 760 761 **Tips & Tricks**: 762 763 Hard-coded values can be used to test the impact of a patch during development, gradually roll out patches, etc. . 764 ```yaml 765 enabledIf: false 766 ``` 767 768 A boolean variable can be used to enable/disable a patch (or "feature"). This can have opt-in or opt-out behavior 769 depending on the default value of the variable. 770 ```yaml 771 enabledIf: "{{ .httpProxyEnabled }}" 772 ``` 773 774 Of course the same is possible by adding a boolean variable to a configuration object. 775 ```yaml 776 enabledIf: "{{ .httpProxy.enabled }}" 777 ``` 778 779 Builtin variables can be leveraged to apply a patch only for a specific Kubernetes version. 780 ```yaml 781 enabledIf: '{{ semverCompare "1.21.1" .builtin.controlPlane.version }}' 782 ``` 783 784 With `semverCompare` and `coalesce` a feature can be enabled in newer versions of Kubernetes for both KubeadmConfigTemplate and KubeadmControlPlane. 785 ```yaml 786 enabledIf: '{{ semverCompare "^1.22.0" (coalesce .builtin.controlPlane.version .builtin.machineDeployment.version )}}' 787 ``` 788 789 <aside class="note"> 790 791 <h1>Builtin Variables</h1> 792 793 Please be aware that while you can use builtin variables, if you use for example a MachineDeployment-specific variable this 794 can mean that patches are only applied to some MachineDeployments. `enabledIf` is evaluated for each template that should be patched 795 individually. 796 797 </aside> 798 799 ### Version-aware patches 800 801 In some cases the ClusterClass authors want a patch to be computed according to the Kubernetes version in use. 802 803 While this is not a problem "per se" and it does not differ from writing any other patch, it is important 804 to keep in mind that there could be different Kubernetes version in a Cluster at any time, all of them 805 accessible via built in variables: 806 807 - `builtin.cluster.topology.version` defines the Kubernetes version from `cluster.topology`, and it acts 808 as the desired Kubernetes version for the entire cluster. However, during an upgrade workflow it could happen that 809 some objects in the Cluster are still at the older version. 810 - `builtin.controlPlane.version`, represent the desired version for the control plane object; usually this 811 version changes immediately after `cluster.topology.version` is updated (unless there are other operations 812 in progress preventing the upgrade to start). 813 - `builtin.machineDeployment.version`, represent the desired version for each specific MachineDeployment object; 814 this version changes only after the upgrade for the control plane is completed, and in case of many 815 MachineDeployments in the same cluster, they are upgraded sequentially. 816 817 This info should provide the bases for developing version-aware patches, allowing the patch author to determine when a 818 patch should adapt to the new Kubernetes version by choosing one of the above variables. In practice the 819 following rules applies to the most common use cases: 820 821 - When developing a version-aware patch for the control plane, `builtin.controlPlane.version` must be used. 822 - When developing a version-aware patch for MachineDeployments, `builtin.machineDeployment.version` must be used. 823 824 **Tips & Tricks**: 825 826 Sometimes users need to define variables to be used by version-aware patches, and in this case it is important 827 to keep in mind that there could be different Kubernetes versions in a Cluster at any time. 828 829 A simple approach to solve this problem is to define a map of version-aware variables, with the key of each item 830 being the Kubernetes version. Patch could then use the proper builtin variables as a lookup entry to fetch 831 the corresponding values for the Kubernetes version in use by each object. 832 833 ## JSON patches tips & tricks 834 835 JSON patches specification [RFC6902] requires that the target of 836 add operation must exist. 837 838 As a consequence ClusterClass authors should pay special attention when the following 839 conditions apply in order to prevent errors when a patch is applied: 840 841 * the patch tries to `add` a value to an **array** (which is a **slice** in the corresponding go struct) 842 * the slice was defined with `omitempty` 843 * the slice currently does not exist 844 845 A workaround in this particular case is to create the array in the patch instead of adding to the non-existing one. 846 When creating the slice, existing values would be overwritten so this should only be used when it does not exist. 847 848 The following example shows both cases to consider while writing a patch for adding a value to a slice. 849 This patch targets to add a file to the `files` slice of a `KubeadmConfigTemplate` which has [omitempty](https://github.com/kubernetes-sigs/cluster-api/blob/main/bootstrap/kubeadm/api/v1beta1/kubeadmconfig_types.go#L54) set. 850 851 {{#tabs name:"tab-configuration-patches" tabs:"Add to existing slice,Create slice"}} 852 {{#tab Add to existing slice}} 853 854 This patch **requires** the key `.spec.template.spec.files` to exist to succeed. 855 856 ```yaml 857 apiVersion: cluster.x-k8s.io/v1beta1 858 kind: ClusterClass 859 metadata: 860 name: my-clusterclass 861 spec: 862 ... 863 patches: 864 - name: add file 865 definitions: 866 - selector: 867 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 868 kind: KubeadmConfigTemplate 869 jsonPatches: 870 - op: add 871 path: /spec/template/spec/files/- 872 value: 873 content: Some content. 874 path: /some/file 875 --- 876 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 877 kind: KubeadmConfigTemplate 878 metadata: 879 name: "quick-start-default-worker-bootstraptemplate" 880 spec: 881 template: 882 spec: 883 ... 884 files: 885 - content: Some other content 886 path: /some/other/file 887 ``` 888 889 {{#/tab }} 890 {{#tab Create slice}} 891 892 This patch would **overwrite** an existing slice at `.spec.template.spec.files`. 893 894 ```yaml 895 apiVersion: cluster.x-k8s.io/v1beta1 896 kind: ClusterClass 897 metadata: 898 name: my-clusterclass 899 spec: 900 ... 901 patches: 902 - name: add file 903 definitions: 904 - selector: 905 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 906 kind: KubeadmConfigTemplate 907 jsonPatches: 908 - op: add 909 path: /spec/template/spec/files 910 value: 911 - content: Some content. 912 path: /some/file 913 --- 914 apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 915 kind: KubeadmConfigTemplate 916 metadata: 917 name: "quick-start-default-worker-bootstraptemplate" 918 spec: 919 template: 920 spec: 921 ... 922 ``` 923 924 {{#/tab }} 925 {{#/tabs }} 926 927 <!-- links --> 928 [Changing a ClusterClass]: ./change-clusterclass.md 929 [clusterctl alpha topology plan]: ../../../clusterctl/commands/alpha-topology-plan.md 930 [RFC6902]: https://datatracker.ietf.org/doc/html/rfc6902#appendix-A.12