sigs.k8s.io/cluster-api-provider-azure@v1.17.0/docs/book/src/managed/managedcluster-join-vmss.md (about)

     1  ## Joining self-managed VMSS nodes to an AKS control plane
     2  
     3  <aside class="note warning">
     4  
     5  <h1> Warning </h1>
     6  
     7  This is not an officially supported AKS scenario. It is meant to facilitate development and testing of alpha/beta Kubernetes features. Please use at your own risk.
     8  
     9  </aside>
    10  
    11  ### Installing Addons
    12  
    13  In order for the nodes to become ready, you'll need to install Cloud Provider Azure and a CNI.
    14  
    15  AKS will install Cloud Provider Azure on the self-managed nodes as long as they have the appropriate labels. You can add the required label on the nodes by running the following command on the AKS cluster:
    16  
    17  ```bash
    18  kubectl label node <node name> kubernetes.azure.com/cluster=<nodeResourceGroupName>
    19  ```
    20  
    21  Repeat this for each node in the MachinePool.
    22  
    23  <aside class="note">
    24  
    25  <h1> Warning </h1>
    26  
    27  Note: CAPI does not currently support propagating labels from the MachinePool to the nodes, in the future this could be part of the MachinePool definition.
    28  
    29  </aside>
    30  
    31  For the CNI, you can install the CNI of your choice. For example, to install Azure CNI, run the following command on the AKS cluster:
    32  
    33  ```bash
    34  kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/cluster-api-provider-azure/main/templates/addons/azure-cni-v1.yaml
    35  ```
    36  
    37  ### Notes
    38  
    39  Some notes about how this works under the hood:
    40  
    41  - CAPZ will fetch the kubeconfig for the AKS cluster and store it in a secret named `${CLUSTER_NAME}-kubeconfig` in the management cluster. That secret is then used for discovery by the `KubeadmConfig` resource.
    42  - You can customize the `MachinePool`, `AzureMachinePool`, and `KubeadmConfig` resources to your liking. The example above is just a starting point. Note that the key configurations to keep are in the `KubeadmConfig` resource, namely the `files`, `joinConfiguration`, and `preKubeadmCommands` sections.
    43  - The `KubeadmConfig` resource will be used to generate a `kubeadm join` command that will be executed on each node in the VMSS. It uses the cluster kubeconfig for discovery. The `kubeadm init phase upload-config all` is run as a preKubeadmCommand to ensure that the kubeadm and kubelet configurations are uploaded to a ConfigMap. This step would normally be done by the `kubeadm init` command, but since we're not running `kubeadm init` we need to do it manually.
    44  
    45  ### Creating the MachinePool
    46  
    47  You can add a self-managed VMSS node pool to any CAPZ-managed AKS cluster by applying the following resources to the management cluster:
    48  
    49  ```yaml
    50  apiVersion: cluster.x-k8s.io/v1beta1
    51  kind: MachinePool
    52  metadata:
    53    name: ${CLUSTER_NAME}-vmss
    54    namespace: default
    55  spec:
    56    clusterName: ${CLUSTER_NAME}
    57    replicas: ${WORKER_MACHINE_COUNT}
    58    template:
    59      spec:
    60        bootstrap:
    61          configRef:
    62            apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    63            kind: KubeadmConfig
    64            name: ${CLUSTER_NAME}-vmss
    65        clusterName: ${CLUSTER_NAME}
    66        infrastructureRef:
    67          apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    68          kind: AzureMachinePool
    69          name: ${CLUSTER_NAME}-vmss
    70        version: ${KUBERNETES_VERSION}
    71  ---
    72  apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    73  kind: AzureMachinePool
    74  metadata:
    75    name: ${CLUSTER_NAME}-vmss
    76    namespace: default
    77  spec:
    78    location: ${AZURE_LOCATION}
    79    strategy:
    80      rollingUpdate:
    81        deletePolicy: Oldest
    82        maxSurge: 25%
    83        maxUnavailable: 1
    84      type: RollingUpdate
    85    template:
    86      osDisk:
    87        diskSizeGB: 30
    88        managedDisk:
    89          storageAccountType: Premium_LRS
    90        osType: Linux
    91      sshPublicKey: ${AZURE_SSH_PUBLIC_KEY_B64:=""}
    92      vmSize: ${AZURE_NODE_MACHINE_TYPE}
    93  ---
    94  apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    95  kind: KubeadmConfig
    96  metadata:
    97    name: ${CLUSTER_NAME}-vmss
    98    namespace: default
    99  spec:
   100    files:
   101    - contentFrom:
   102        secret:
   103          key: worker-node-azure.json
   104          name: ${CLUSTER_NAME}-vmss-azure-json
   105      owner: root:root
   106      path: /etc/kubernetes/azure.json
   107      permissions: "0644"
   108    - contentFrom:
   109        secret:
   110          key: value
   111          name: ${CLUSTER_NAME}-kubeconfig
   112      owner: root:root
   113      path: /etc/kubernetes/admin.conf
   114      permissions: "0644"
   115    joinConfiguration:
   116      discovery:
   117        file:
   118          kubeConfigPath: /etc/kubernetes/admin.conf
   119      nodeRegistration:
   120        kubeletExtraArgs:
   121          cloud-provider: external
   122        name: '{{ ds.meta_data["local_hostname"] }}'
   123    preKubeadmCommands:
   124    - kubeadm init phase upload-config all
   125    ```