github.com/cilium/cilium@v1.16.2/clustermesh-apiserver/README.md (about)

     1  # API server for Cilium ClusterMesh
     2  
     3  Cilium uses a clustermesh-apiserver when multiple clusters are connected in clustermesh, or
     4  when external workloads are connected to the Cilium cluster. If neither is used, then
     5  clustermesh-apiserver is never required.
     6  
     7  Since etcd is used in a clustermesh for data synchronization, an etcd server container
     8  is deployed within clustermesh-apiserver pod.
     9  
    10  When used in an External Workloads setup, it also creates a CiliumNode and
    11  CiliumEndpoint resources for each workload name and allocates its identity.
    12  
    13  Note: `ipv4-alloc-cidr` set in the CiliumExternalWorkload object spec is currently unused.
    14  IP address tied to the CiliumEndpoint and CiliumNode is the one that is registered by
    15  cilium-agent (IP address of the external workload).
    16  
    17  The API server itself performs the following operations:
    18  
    19  ### K8s synchronization
    20  
    21  It performs the job of synchronizing CiliumIdentites, CiliumEndpoints,
    22  CiliumNodes and Kubernetes services from k8s datastore to the KVStore (etcd).
    23  
    24  ### Heartbeat update
    25  
    26  Cilium's heartbeat path key stored in the KVStore is periodically updated by
    27  the API server with the current time so that Cilium Agents can correctly
    28  validate KVStore updates. The key for this heartbeat is
    29  `cilium/.heartbeat`.
    30  
    31  ## Deploy the clustermesh-apiserver
    32  
    33  Clustermesh-apiserver is automatically deployed when External
    34  Workloads support or clustermesh is enabled using either Helm or the cilium-cli tool.
    35  
    36  Users are required to set both `cluster.name` and a non-zero `cluster.id` in Helm or
    37  `cilium install --cluster-name <name> --cluster-id <id>`. Otherwise, clustermesh will
    38  not be correctly established.
    39  
    40  `clustermesh-apiserver` service type defaults to `NodePort`. Depending on
    41  your k8s provider it may be beneficial to change this to `LoadBalancer`.
    42  
    43  ### Deploy using cilium-cli:
    44  
    45     ```
    46     $ cilium clustermesh enable
    47     ```
    48  
    49  #### Connect Cilium clusters in to a clustermesh
    50  
    51     ```
    52     $ cilium --context "${CONTEXT1}" clustermesh connect --destination-context "${CONTEXT2}"
    53     ```
    54     Note: `clustermesh connect` command needs to be run for every new cluster (context) that joins clustermesh.
    55  
    56  #### Wait for clustermesh status to be ready
    57  
    58     ```
    59     $ cilium --context "${CONTEXT1}" clustermesh status --wait
    60     ```
    61  
    62  ### Deploy using helm:
    63  
    64     ```
    65     $ helm install cilium ... \
    66       --set clustermesh.useAPIServer=true \
    67     ```
    68  
    69  Additionally, if your load balancer can give you a static IP address, it may be
    70  specified like so:
    71  
    72     ```
    73     $ helm install cilium ... \
    74       --set clustermesh.apiserver.service.loadBalancerIP=xxx.xxx.xxx.xxx \
    75     ```
    76  
    77  Clustermesh-apiserver is deployed as a standard k8s deployment with multiple
    78  containers. You can check that both clustermesh-apiserver and etcd server are present:
    79  
    80     ```
    81     $ kubectl get pods -l k8s-app=clustermesh-apiserver \
    82       -o jsonpath='{range .items[*].spec.containers[*]}{.image}{"\n"}{end}'
    83     quay.io/coreos/etcd:v3.5.4
    84     quay.io/cilium/clustermesh-apiserver:v1.10.2
    85     ```
    86  #### Connect Cilium clusters in to a clustermesh
    87  
    88  In helm installation clusters have to be connected in 2 steps:
    89  
    90  1. Extract a `cilium-clustermesh` secret from each cluster to be applied in another cluster:
    91  
    92     ```
    93     $ contrib/k8s/k8s-extract-clustermesh-nodeport-secret.sh > cluster1-secret.json
    94     ```
    95  
    96     Repeat this step in all your clusters, storing the outputs into different files.
    97  
    98  3. Apply secrets from all other clusters in each of your clusters, e.g., on cluster1:
    99  
   100     ```
   101     $ contrib/k8s/k8s-import-clustermesh-secrets.sh cluster2-secret.json cluster3-secret.json ...
   102     ```