sigs.k8s.io/cluster-api@v1.7.1/docs/book/src/tasks/bootstrap/kubeadm-bootstrap/index.md (about)

     1  # Cluster API bootstrap provider kubeadm
     2  ## What is the Cluster API bootstrap provider kubeadm?
     3  
     4  Cluster API bootstrap provider Kubeadm (CABPK) is a component responsible for generating a cloud-init script to
     5  turn a Machine into a Kubernetes Node. This implementation uses [kubeadm](https://github.com/kubernetes/kubeadm)
     6  for Kubernetes bootstrap.
     7  
     8  ### Resources
     9  
    10  * [design doc](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20190610-machine-states-preboot-bootstrapping.md)
    11  * [The Kubebuilder Book](https://book.kubebuilder.io)
    12  
    13  ## How does CABPK work?
    14  
    15  Assuming you have deployed the CAPI and CAPD controllers, create a `Cluster` object and its corresponding `DockerCluster`
    16  infrastructure object.
    17  
    18  ```yaml
    19  kind: DockerCluster
    20  apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    21  metadata:
    22    name: my-cluster-docker
    23  ---
    24  kind: Cluster
    25  apiVersion: cluster.x-k8s.io/v1beta1
    26  metadata:
    27    name: my-cluster
    28  spec:
    29    infrastructureRef:
    30      kind: DockerCluster
    31      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    32      name: my-cluster-docker
    33  ```
    34  
    35  Now you can start creating machines by defining a `Machine`, its corresponding `DockerMachine` object, and
    36  the `KubeadmConfig` bootstrap object.
    37  
    38  ```yaml
    39  kind: KubeadmConfig
    40  apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    41  metadata:
    42    name: my-control-plane1-config
    43  spec:
    44    initConfiguration:
    45      nodeRegistration: {} # node registration parameters are automatically injected by CAPD according to the kindest/node image in use.
    46  ---
    47  kind: DockerMachine
    48  apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    49  metadata:
    50    name: my-control-plane1-docker
    51  ---
    52  kind: Machine
    53  apiVersion: cluster.x-k8s.io/v1beta1
    54  metadata:
    55    name: my-control-plane1
    56    labels:
    57      cluster.x-k8s.io/cluster-name: my-cluster
    58      cluster.x-k8s.io/control-plane: "true"
    59      set: controlplane
    60  spec:
    61    bootstrap:
    62      configRef:
    63        kind: KubeadmConfig
    64        apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    65        name: my-control-plane1-config
    66    infrastructureRef:
    67      kind: DockerMachine
    68      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    69      name: my-control-plane1-docker
    70    version: "v1.19.1"
    71  ```
    72  
    73  CABPK's main responsibility is to convert a `KubeadmConfig` bootstrap object into a cloud-init script that is
    74  going to turn a Machine into a Kubernetes Node using `kubeadm`.
    75  
    76  The cloud-init script will be saved into a secret `KubeadmConfig.Status.DataSecretName` and then the infrastructure provider
    77  (CAPD in this example) will pick up this value and proceed with the machine creation and the actual bootstrap.
    78  
    79  ### KubeadmConfig objects
    80  The `KubeadmConfig` object allows full control of Kubeadm init/join operations by exposing raw `InitConfiguration`,
    81  `ClusterConfiguration` and `JoinConfiguration` objects.
    82  
    83  `InitConfiguration` and `JoinConfiguration` exposes `Patches` field which can be used to specify the patches from a directory,
    84  this support is available from K8s 1.22 version onwards.
    85  
    86  CABPK will fill in some values if they are left empty with sensible defaults:
    87  
    88  | `KubeadmConfig` field                           | Default                                                      |
    89  | ----------------------------------------------- | ------------------------------------------------------------ |
    90  | `clusterConfiguration.KubernetesVersion`        | `Machine.Spec.Version`[1]                                     |
    91  | `clusterConfiguration.clusterName`              | `Cluster.metadata.name`                                      |
    92  | `clusterConfiguration.controlPlaneEndpoint`     | `Cluster.status.apiEndpoints[0]` |
    93  | `clusterConfiguration.networking.dnsDomain` | `Cluster.spec.clusterNetwork.serviceDomain`              |
    94  | `clusterConfiguration.networking.serviceSubnet` | `Cluster.spec.clusterNetwork.service.cidrBlocks[0]`              |
    95  | `clusterConfiguration.networking.podSubnet` | `Cluster.spec.clusterNetwork.pods.cidrBlocks[0]`              |
    96  | `joinConfiguration.discovery`                   | a short lived BootstrapToken generated by CABPK              |
    97  
    98  > IMPORTANT! overriding above defaults could lead to broken Clusters.
    99  
   100  [1] if both `clusterConfiguration.KubernetesVersion` and `Machine.Spec.Version` are empty, the latest Kubernetes
   101  version will be installed (as defined by the default kubeadm behavior). 
   102  #### Examples
   103  Valid combinations of configuration objects are:
   104  - for KCP, `InitConfiguration` and `ClusterConfiguration` for the first control plane node; `JoinConfiguration` for additional control plane nodes
   105  - for machine deployments, `JoinConfiguration` for worker nodes
   106  
   107  Bootstrap control plane node:
   108  ```yaml
   109  kind: KubeadmConfig
   110  apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
   111  metadata:
   112    name: my-control-plane1-config
   113  spec:
   114    initConfiguration:
   115      nodeRegistration:
   116        nodeRegistration: {} # node registration parameters are automatically injected by CAPD according to the kindest/node image in use.
   117  ```
   118  
   119  Additional control plane nodes:
   120  ```yaml
   121  kind: KubeadmConfig
   122  apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
   123  metadata:
   124    name: my-control-plane2-config
   125  spec:
   126    joinConfiguration:
   127      nodeRegistration:
   128        nodeRegistration: {} # node registration parameters are automatically injected by CAPD according to the kindest/node image in use.
   129      controlPlane: {}
   130  ```
   131  
   132  worker nodes:
   133  ```yaml
   134  kind: KubeadmConfig
   135  apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
   136  metadata:
   137    name: my-worker1-config
   138  spec:
   139    joinConfiguration:
   140      nodeRegistration:
   141        nodeRegistration: {} # node registration parameters are automatically injected by CAPD according to the kindest/node image in use.
   142  ```
   143  
   144  ### Bootstrap Orchestration
   145  CABPK supports multiple control plane machines initing at the same time.
   146  The generation of cloud-init scripts of different machines is orchestrated in order to ensure a cluster
   147  bootstrap process that will be compliant with the correct Kubeadm init/join sequence. More in detail:
   148  1. cloud-config-data generation starts only after `Cluster.Status.InfrastructureReady` flag is set to `true`.
   149  2. at this stage, cloud-config-data will be generated for the first control plane machine only, keeping
   150  on hold additional control plane machines existing in the cluster, if any (kubeadm init).
   151  3. after the `ControlPlaneInitialized` conditions on the cluster object is set to true,
   152  the cloud-config-data for all the other machines are generated (kubeadm join/join —control-plane).
   153  
   154  ### Certificate Management
   155  The user can choose two approaches for certificate management:
   156  1. provide required certificate authorities (CAs) to use for `kubeadm init/kubeadm join --control-plane`; such CAs
   157  should be provided as a `Secrets` objects in the management cluster.
   158  2. let KCP to generate the necessary `Secrets` objects with a self-signed certificate authority for kubeadm
   159  
   160  See [here](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/) for more info about certificate management with kubeadm.
   161  
   162  ### Additional Features
   163  The `KubeadmConfig` object supports customizing the content of the config-data. The following examples illustrate how to specify these options. They should be adapted to fit your environment and use case.
   164  
   165  - `KubeadmConfig.Files` specifies additional files to be created on the machine, either with content inline or by referencing a secret.
   166  
   167      ```yaml
   168      files:
   169      - contentFrom:
   170          secret:
   171            key: node-cloud.json
   172            name: ${CLUSTER_NAME}-md-0-cloud-json
   173        owner: root:root
   174        path: /etc/kubernetes/cloud.json
   175        permissions: "0644"
   176      - path: /etc/kubernetes/cloud.json
   177        owner: "root:root"
   178        permissions: "0644"
   179        content: |
   180          {
   181            "cloud": "CustomCloud"
   182          }
   183      ```
   184  
   185  - `KubeadmConfig.PreKubeadmCommands` specifies a list of commands to be executed before `kubeadm init/join`
   186  
   187      ```yaml
   188      preKubeadmCommands:
   189        - hostname "{{ ds.meta_data.hostname }}"
   190        - echo "{{ ds.meta_data.hostname }}" >/etc/hostname
   191      ```
   192  
   193  - `KubeadmConfig.PostKubeadmCommands` same as above, but after `kubeadm init/join`
   194  
   195      ```yaml
   196      postKubeadmCommands:
   197        - echo "success" >/var/log/my-custom-file.log
   198      ```
   199  
   200  - `KubeadmConfig.Users` specifies a list of users to be created on the machine
   201  
   202      ```yaml
   203      users:
   204        - name: capiuser
   205          sshAuthorizedKeys:
   206          - '${SSH_AUTHORIZED_KEY}'
   207          sudo: ALL=(ALL) NOPASSWD:ALL
   208      ```
   209  
   210  - `KubeadmConfig.NTP` specifies NTP settings for the machine
   211  
   212    ```yaml
   213    ntp:
   214      servers:
   215        - IP_ADDRESS
   216      enabled: true
   217    ```
   218  
   219  - `KubeadmConfig.DiskSetup` specifies options for the creation of partition tables and file systems on devices.
   220  
   221    ```yaml
   222    diskSetup:
   223      filesystems:
   224      - device: /dev/disk/azure/scsi1/lun0
   225        extraOpts:
   226        - -E
   227        - lazy_itable_init=1,lazy_journal_init=1
   228        filesystem: ext4
   229        label: etcd_disk
   230      - device: ephemeral0.1
   231        filesystem: ext4
   232        label: ephemeral0
   233        replaceFS: ntfs
   234      partitions:
   235      - device: /dev/disk/azure/scsi1/lun0
   236        layout: true
   237        overwrite: false
   238        tableType: gpt
   239    ```
   240  
   241  - `KubeadmConfig.Mounts` specifies a list of mount points to be setup.
   242  
   243      ```yaml
   244      mounts:
   245      - - LABEL=etcd_disk
   246        - /var/lib/etcddisk
   247      ```
   248  
   249  - `KubeadmConfig.Verbosity` specifies the `kubeadm` log level verbosity
   250  
   251      ```yaml
   252      verbosity: 10
   253      ```
   254  
   255  - `KubeadmConfig.UseExperimentalRetryJoin` replaces a basic kubeadm command with a shell script with retries for joins. This will add about 40KB to userdata.
   256  
   257      ```yaml
   258      useExperimentalRetryJoin: true
   259      ```
   260  
   261  For more information on cloud-init options, see [cloud config examples](https://cloudinit.readthedocs.io/en/latest/topics/examples.html).