sigs.k8s.io/cluster-api@v1.6.3/docs/book/src/tasks/bootstrap/kubeadm-bootstrap/index.md (about)

     1  # Cluster API bootstrap provider kubeadm
     2  ## What is the Cluster API bootstrap provider kubeadm?
     3  
     4  Cluster API bootstrap provider Kubeadm (CABPK) is a component responsible for generating a cloud-init script to
     5  turn a Machine into a Kubernetes Node. This implementation uses [kubeadm](https://github.com/kubernetes/kubeadm)
     6  for Kubernetes bootstrap.
     7  
     8  ### Resources
     9  
    10  * [design doc](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20190610-machine-states-preboot-bootstrapping.md)
    11  * [The Kubebuilder Book](https://book.kubebuilder.io)
    12  
    13  ## How does CABPK work?
    14  
    15  Assuming you have deployed the CAPI and CAPD controllers, create a `Cluster` object and its corresponding `DockerCluster`
    16  infrastructure object.
    17  
    18  ```yaml
    19  kind: DockerCluster
    20  apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    21  metadata:
    22    name: my-cluster-docker
    23  ---
    24  kind: Cluster
    25  apiVersion: cluster.x-k8s.io/v1beta1
    26  metadata:
    27    name: my-cluster
    28  spec:
    29    infrastructureRef:
    30      kind: DockerCluster
    31      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    32      name: my-cluster-docker
    33  ```
    34  
    35  Now you can start creating machines by defining a `Machine`, its corresponding `DockerMachine` object, and
    36  the `KubeadmConfig` bootstrap object.
    37  
    38  ```yaml
    39  kind: KubeadmConfig
    40  apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    41  metadata:
    42    name: my-control-plane1-config
    43  spec:
    44    initConfiguration:
    45      nodeRegistration: {} # node registration parameters are automatically injected by CAPD according to the kindest/node image in use.
    46    clusterConfiguration:
    47      controllerManager:
    48        extraArgs:
    49          enable-hostpath-provisioner: "true"
    50  ---
    51  kind: DockerMachine
    52  apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    53  metadata:
    54    name: my-control-plane1-docker
    55  ---
    56  kind: Machine
    57  apiVersion: cluster.x-k8s.io/v1beta1
    58  metadata:
    59    name: my-control-plane1
    60    labels:
    61      cluster.x-k8s.io/cluster-name: my-cluster
    62      cluster.x-k8s.io/control-plane: "true"
    63      set: controlplane
    64  spec:
    65    bootstrap:
    66      configRef:
    67        kind: KubeadmConfig
    68        apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    69        name: my-control-plane1-config
    70    infrastructureRef:
    71      kind: DockerMachine
    72      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    73      name: my-control-plane1-docker
    74    version: "v1.19.1"
    75  ```
    76  
    77  CABPK's main responsibility is to convert a `KubeadmConfig` bootstrap object into a cloud-init script that is
    78  going to turn a Machine into a Kubernetes Node using `kubeadm`.
    79  
    80  The cloud-init script will be saved into a secret `KubeadmConfig.Status.DataSecretName` and then the infrastructure provider
    81  (CAPD in this example) will pick up this value and proceed with the machine creation and the actual bootstrap.
    82  
    83  ### KubeadmConfig objects
    84  The `KubeadmConfig` object allows full control of Kubeadm init/join operations by exposing raw `InitConfiguration`,
    85  `ClusterConfiguration` and `JoinConfiguration` objects.
    86  
    87  `InitConfiguration` and `JoinConfiguration` exposes `Patches` field which can be used to specify the patches from a directory,
    88  this support is available from K8s 1.22 version onwards.
    89  
    90  CABPK will fill in some values if they are left empty with sensible defaults:
    91  
    92  | `KubeadmConfig` field                           | Default                                                      |
    93  | ----------------------------------------------- | ------------------------------------------------------------ |
    94  | `clusterConfiguration.KubernetesVersion`        | `Machine.Spec.Version`[1]                                     |
    95  | `clusterConfiguration.clusterName`              | `Cluster.metadata.name`                                      |
    96  | `clusterConfiguration.controlPlaneEndpoint`     | `Cluster.status.apiEndpoints[0]` |
    97  | `clusterConfiguration.networking.dnsDomain` | `Cluster.spec.clusterNetwork.serviceDomain`              |
    98  | `clusterConfiguration.networking.serviceSubnet` | `Cluster.spec.clusterNetwork.service.cidrBlocks[0]`              |
    99  | `clusterConfiguration.networking.podSubnet` | `Cluster.spec.clusterNetwork.pods.cidrBlocks[0]`              |
   100  | `joinConfiguration.discovery`                   | a short lived BootstrapToken generated by CABPK              |
   101  
   102  > IMPORTANT! overriding above defaults could lead to broken Clusters.
   103  
   104  [1] if both `clusterConfiguration.KubernetesVersion` and `Machine.Spec.Version` are empty, the latest Kubernetes
   105  version will be installed (as defined by the default kubeadm behavior). 
   106  #### Examples
   107  Valid combinations of configuration objects are:
   108  - for KCP, `InitConfiguration` and `ClusterConfiguration` for the first control plane node; `JoinConfiguration` for additional control plane nodes
   109  - for machine deployments, `JoinConfiguration` for worker nodes
   110  
   111  Bootstrap control plane node:
   112  ```yaml
   113  kind: KubeadmConfig
   114  apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
   115  metadata:
   116    name: my-control-plane1-config
   117  spec:
   118    initConfiguration:
   119      nodeRegistration:
   120        nodeRegistration: {} # node registration parameters are automatically injected by CAPD according to the kindest/node image in use.
   121    clusterConfiguration:
   122      controllerManager:
   123        extraArgs:
   124          enable-hostpath-provisioner: "true"
   125  ```
   126  
   127  Additional control plane nodes:
   128  ```yaml
   129  kind: KubeadmConfig
   130  apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
   131  metadata:
   132    name: my-control-plane2-config
   133  spec:
   134    joinConfiguration:
   135      nodeRegistration:
   136        nodeRegistration: {} # node registration parameters are automatically injected by CAPD according to the kindest/node image in use.
   137      controlPlane: {}
   138  ```
   139  
   140  worker nodes:
   141  ```yaml
   142  kind: KubeadmConfig
   143  apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
   144  metadata:
   145    name: my-worker1-config
   146  spec:
   147    joinConfiguration:
   148      nodeRegistration:
   149        nodeRegistration: {} # node registration parameters are automatically injected by CAPD according to the kindest/node image in use.
   150  ```
   151  
   152  ### Bootstrap Orchestration
   153  CABPK supports multiple control plane machines initing at the same time.
   154  The generation of cloud-init scripts of different machines is orchestrated in order to ensure a cluster
   155  bootstrap process that will be compliant with the correct Kubeadm init/join sequence. More in detail:
   156  1. cloud-config-data generation starts only after `Cluster.Status.InfrastructureReady` flag is set to `true`.
   157  2. at this stage, cloud-config-data will be generated for the first control plane machine only, keeping
   158  on hold additional control plane machines existing in the cluster, if any (kubeadm init).
   159  3. after the `ControlPlaneInitialized` conditions on the cluster object is set to true,
   160  the cloud-config-data for all the other machines are generated (kubeadm join/join —control-plane).
   161  
   162  ### Certificate Management
   163  The user can choose two approaches for certificate management:
   164  1. provide required certificate authorities (CAs) to use for `kubeadm init/kubeadm join --control-plane`; such CAs
   165  should be provided as a `Secrets` objects in the management cluster.
   166  2. let KCP to generate the necessary `Secrets` objects with a self-signed certificate authority for kubeadm
   167  
   168  See [here](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/) for more info about certificate management with kubeadm.
   169  
   170  ### Additional Features
   171  The `KubeadmConfig` object supports customizing the content of the config-data. The following examples illustrate how to specify these options. They should be adapted to fit your environment and use case.
   172  
   173  - `KubeadmConfig.Files` specifies additional files to be created on the machine, either with content inline or by referencing a secret.
   174  
   175      ```yaml
   176      files:
   177      - contentFrom:
   178          secret:
   179            key: node-cloud.json
   180            name: ${CLUSTER_NAME}-md-0-cloud-json
   181        owner: root:root
   182        path: /etc/kubernetes/cloud.json
   183        permissions: "0644"
   184      - path: /etc/kubernetes/cloud.json
   185        owner: "root:root"
   186        permissions: "0644"
   187        content: |
   188          {
   189            "cloud": "CustomCloud"
   190          }
   191      ```
   192  
   193  - `KubeadmConfig.PreKubeadmCommands` specifies a list of commands to be executed before `kubeadm init/join`
   194  
   195      ```yaml
   196      preKubeadmCommands:
   197        - hostname "{{ ds.meta_data.hostname }}"
   198        - echo "{{ ds.meta_data.hostname }}" >/etc/hostname
   199      ```
   200  
   201  - `KubeadmConfig.PostKubeadmCommands` same as above, but after `kubeadm init/join`
   202  
   203      ```yaml
   204      postKubeadmCommands:
   205        - echo "success" >/var/log/my-custom-file.log
   206      ```
   207  
   208  - `KubeadmConfig.Users` specifies a list of users to be created on the machine
   209  
   210      ```yaml
   211      users:
   212        - name: capiuser
   213          sshAuthorizedKeys:
   214          - '${SSH_AUTHORIZED_KEY}'
   215          sudo: ALL=(ALL) NOPASSWD:ALL
   216      ```
   217  
   218  - `KubeadmConfig.NTP` specifies NTP settings for the machine
   219  
   220    ```yaml
   221    ntp:
   222      servers:
   223        - IP_ADDRESS
   224      enabled: true
   225    ```
   226  
   227  - `KubeadmConfig.DiskSetup` specifies options for the creation of partition tables and file systems on devices.
   228  
   229    ```yaml
   230    diskSetup:
   231      filesystems:
   232      - device: /dev/disk/azure/scsi1/lun0
   233        extraOpts:
   234        - -E
   235        - lazy_itable_init=1,lazy_journal_init=1
   236        filesystem: ext4
   237        label: etcd_disk
   238      - device: ephemeral0.1
   239        filesystem: ext4
   240        label: ephemeral0
   241        replaceFS: ntfs
   242      partitions:
   243      - device: /dev/disk/azure/scsi1/lun0
   244        layout: true
   245        overwrite: false
   246        tableType: gpt
   247    ```
   248  
   249  - `KubeadmConfig.Mounts` specifies a list of mount points to be setup.
   250  
   251      ```yaml
   252      mounts:
   253      - - LABEL=etcd_disk
   254        - /var/lib/etcddisk
   255      ```
   256  
   257  - `KubeadmConfig.Verbosity` specifies the `kubeadm` log level verbosity
   258  
   259      ```yaml
   260      verbosity: 10
   261      ```
   262  
   263  - `KubeadmConfig.UseExperimentalRetryJoin` replaces a basic kubeadm command with a shell script with retries for joins. This will add about 40KB to userdata.
   264  
   265      ```yaml
   266      useExperimentalRetryJoin: true
   267      ```
   268  
   269  For more information on cloud-init options, see [cloud config examples](https://cloudinit.readthedocs.io/en/latest/topics/examples.html).