sigs.k8s.io/cluster-api-provider-aws@v1.5.5/docs/book/src/topics/ignition-support.md (about)

     1  # Ignition support
     2  
     3  - **Feature status:** Experimental
     4  - **Feature gate:** BootstrapFormatIgnition=true
     5  
     6  The default configuration engine for bootstrapping workload cluster machines is [cloud-init][cloud-init].
     7  **Ignition** is an alternative engine used by Linux distributions such as [Flatcar Container Linux][flatcar]
     8  and [Fedora CoreOS][fedora-coreos] and therefore should be used when choosing an Ignition-based distribution as
     9  the underlying OS for workload clusters.
    10  
    11  <aside class="note warning">
    12  
    13  <h1>Note</h1>
    14  
    15  This initial implementation uses Ignition **v2** and was tested with **Flatcar Container Linux** only.
    16  Future releases are expected to add Ignition **v3** support and cover more Linux distributions.
    17  
    18  </aside>
    19  
    20  This document explains how Ignition support works.
    21  
    22  For more generic information, see [Cluster API documentation on Ignition Bootstrap configuration][cabpk].
    23  
    24  ## Overview
    25  
    26  By default machine controller stores EC2 instance user data using SSM to store it encrypted, which underneath
    27  use multi part mime types, which are [unlikely to be supported](https://github.com/coreos/ignition/issues/1072)
    28  by Ignition.
    29  
    30  EC2 user data is also limited to 64 KB, which is often not enough to provision Kubernetes controlplane because
    31  of the size of required certificates and configuration files.
    32  
    33  To address those limitations CAPA can create and use S3 Bucket to store encrypted user data, which will be then
    34  pulled by the instances during provisioning.
    35  
    36  ## IAM Permissions
    37  
    38  To manage S3 Buckets and objects inside them, CAPA controllers require additional IAM permissions.
    39  
    40  If you use `clusterawsadm` for managing the IAM roles, you can use the configuration below to create S3-related
    41  IAM permissions.
    42  
    43  ``` yaml
    44  apiVersion: bootstrap.aws.infrastructure.cluster.x-k8s.io/v1beta1
    45  kind: AWSIAMConfiguration
    46  spec:
    47    s3Buckets:
    48      enable: true
    49  ```
    50  
    51  See [Using clusterawsadm to fulfill prerequisites](./using-clusterawsadm-to-fulfill-prerequisites.md) for more
    52  details.
    53  
    54  ## Enabling EXP_BOOTSTRAP_FORMAT_IGNITION feature gate
    55  
    56  When deploying CAPA using `clusterctl`, make sure you set `BOOTSTRAP_FORMAT_IGNITION=true` and
    57  `EXP_KUBEADM_BOOTSTRAP_FORMAT_IGNITION=true `environment variables to enable experimental Ignition bootstrap
    58  support.
    59  
    60  ``` sh
    61  # Enable the feature gates controlling Ignition bootstrap.
    62  export EXP_KUBEADM_BOOTSTRAP_FORMAT_IGNITION=true # Used by the kubeadm bootstrap provider.
    63  export EXP_BOOTSTRAP_FORMAT_IGNITION=true # Used by the AWS provider.
    64  
    65  # Initialize the management cluster.
    66  clusterctl init --infrastructure aws
    67  ```
    68  
    69  ## Bucket and object management
    70  
    71  When you want to use Ignition user data format for you machines, you need to configure your cluster to
    72  specify which S3 bucket to use. Controller will then make sure that the bucket exists and that required policies
    73  are in place.
    74  
    75  See the configuration snippet below to learn how to configure `AWSCluster` to manage S3 bucket.
    76  
    77  ``` yaml
    78  apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    79  kind: AWSCluster
    80  spec:
    81    s3Bucket:
    82      controlPlaneIAMInstanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io
    83      name: cluster-api-provider-aws-unique-suffix
    84      nodesIAMInstanceProfiles:
    85      - nodes.cluster-api-provider-aws.sigs.k8s.io
    86  ```
    87  
    88  Buckets are safe to be reused between clusters.
    89  
    90  After successful machine provisioning, bootstrap data is removed from the bucket.
    91  
    92  During cluster removal, if S3 bucket is empty, it will be removed as well.
    93  
    94  ## Bucket naming
    95  
    96  Bucket naming must follow [S3 Bucket naming rules][bucket-naming-rules].
    97  
    98  In addition, by default `clusterawsadm` creates IAM roles to only allow interacting with buckets with
    99  `cluster-api-provider-aws-` prefix to reduce the permissions of CAPA controller, so all bucket names should
   100  use this prefix.
   101  
   102  To change it, use `spec.s3Buckets.namePrefix` field in `AWSIAMConfiguration`.
   103  
   104  ```yaml
   105  apiVersion: bootstrap.aws.infrastructure.cluster.x-k8s.io/v1beta1
   106  kind: AWSIAMConfiguration
   107  spec:
   108    s3Buckets:
   109      namePrefix: my-custom-secure-bucket-prefix-
   110  ```
   111  
   112  ## Supported bootstrap providers
   113  
   114  At the moment only [CABPK][cabpk] is known to support producing bootstrap data in Ignition format.
   115  
   116  ## Trying it out
   117  
   118  If you want to test Ignition support, use `flatcar` cluster flavor.
   119  
   120  ## Other bootstrap providers
   121  
   122  If you want to use Ignition support with custom bootstrap provider which supports producing Ignition bootstrap
   123  data, ensure that bootstrap provider sets the `format` field in machine bootstrap secret to `ignition`. This
   124  information is used by the machine controller to determine which user data format to use for the instances.
   125  
   126  [bucket-naming-rules]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html
   127  [cloud-init]: https://cloudinit.readthedocs.io/
   128  [flatcar]: https://www.flatcar-linux.org/docs/latest/provisioning/ignition/
   129  [fedora-coreos]: https://docs.fedoraproject.org/en-US/fedora-coreos/producing-ign/
   130  [cabpk]: https://cluster-api.sigs.k8s.io/tasks/experimental-features/ignition.html