github.com/datadog/cilium@v1.6.12/Documentation/concepts/terminology.rst (about) 1 .. only:: not (epub or latex or html) 2 3 WARNING: You are looking at unreleased Cilium documentation. 4 Please use the official rendered version released here: 5 http://docs.cilium.io 6 7 *********** 8 Terminology 9 *********** 10 11 12 .. _label: 13 .. _labels: 14 15 Labels 16 ====== 17 18 Labels are a generic, flexible and highly scalable way of addressing a large 19 set of resources as they allow for arbitrary grouping and creation of sets. 20 Whenever something needs to be described, addressed or selected, it is done 21 based on labels: 22 23 - `Endpoints` are assigned labels as derived from the container runtime, 24 orchestration system, or other sources. 25 - `Network policies` select pairs of `endpoints` which are allowed to 26 communicate based on labels. The policies themselves are identified by labels 27 as well. 28 29 What is a Label? 30 ---------------- 31 32 A label is a pair of strings consisting of a ``key`` and ``value``. A label can 33 be formatted as a single string with the format ``key=value``. The key portion 34 is mandatory and must be unique. This is typically achieved by using the 35 reverse domain name notion, e.g. ``io.cilium.mykey=myvalue``. The value portion 36 is optional and can be omitted, e.g. ``io.cilium.mykey``. 37 38 Key names should typically consist of the character set ``[a-z0-9-.]``. 39 40 When using labels to select resources, both the key and the value must match, 41 e.g. when a policy should be applied to all endpoints with the label 42 ``my.corp.foo`` then the label ``my.corp.foo=bar`` will not match the 43 selector. 44 45 Label Source 46 ------------ 47 48 A label can be derived from various sources. For example, an `endpoint`_ will 49 derive the labels associated to the container by the local container runtime as 50 well as the labels associated with the pod as provided by Kubernetes. As these 51 two label namespaces are not aware of each other, this may result in 52 conflicting label keys. 53 54 To resolve this potential conflict, Cilium prefixes all label keys with 55 ``source:`` to indicate the source of the label when importing labels, e.g. 56 ``k8s:role=frontend``, ``container:user=joe``, ``k8s:role=backend``. This means 57 that when you run a Docker container using ``docker run [...] -l foo=bar``, the 58 label ``container:foo=bar`` will appear on the Cilium endpoint representing the 59 container. Similarly, a Kubernetes pod started with the label ``foo: bar`` 60 will be represented with a Cilium endpoint associated with the label 61 ``k8s:foo=bar``. A unique name is allocated for each potential source. The 62 following label sources are currently supported: 63 64 - ``container:`` for labels derived from the local container runtime 65 - ``k8s:`` for labels derived from Kubernetes 66 - ``mesos:`` for labels derived from Mesos 67 - ``reserved:`` for special reserved labels, see :ref:`reserved_labels`. 68 - ``unspec:`` for labels with unspecified source 69 70 When using labels to identify other resources, the source can be included to 71 limit matching of labels to a particular type. If no source is provided, the 72 label source defaults to ``any:`` which will match all labels regardless of 73 their source. If a source is provided, the source of the selecting and matching 74 labels need to match. 75 76 .. _endpoint: 77 .. _endpoints: 78 79 Endpoint 80 ========= 81 82 Cilium makes application containers available on the network by assigning them 83 IP addresses. Multiple application containers can share the same IP address; a 84 typical example for this model is a Kubernetes `Pod`. All application containers 85 which share a common address are grouped together in what Cilium refers to as 86 an endpoint. 87 88 Allocating individual IP addresses enables the use of the entire Layer 4 port 89 range by each endpoint. This essentially allows multiple application containers 90 running on the same cluster node to all bind to well known ports such as ``80`` 91 without causing any conflicts. 92 93 The default behavior of Cilium is to assign both an IPv6 and IPv4 address to 94 every endpoint. However, this behavior can be configured to only allocate an 95 IPv6 address with the ``--enable-ipv4=false`` option. If both an IPv6 and IPv4 96 address are assigned, either address can be used to reach the endpoint. The 97 same behavior will apply with regard to policy rules, load-balancing, etc. See 98 :ref:`address_management` for more details. 99 100 Identification 101 -------------- 102 103 For identification purposes, Cilium assigns an internal endpoint id to all 104 endpoints on a cluster node. The endpoint id is unique within the context of 105 an individual cluster node. 106 107 .. _endpoint id: 108 109 Endpoint Metadata 110 ----------------- 111 112 An endpoint automatically derives metadata from the application containers 113 associated with the endpoint. The metadata can then be used to identify the 114 endpoint for security/policy, load-balancing and routing purposes. 115 116 The source of the metadata will depend on the orchestration system and 117 container runtime in use. The following metadata retrieval mechanisms are 118 currently supported: 119 120 +---------------------+---------------------------------------------------+ 121 | System | Description | 122 +=====================+===================================================+ 123 | Kubernetes | Pod labels (via k8s API) | 124 +---------------------+---------------------------------------------------+ 125 | Mesos | Labels (via CNI) | 126 +---------------------+---------------------------------------------------+ 127 | containerd (Docker) | Container labels (via Docker API) | 128 +---------------------+---------------------------------------------------+ 129 130 Metadata is attached to endpoints in the form of `labels`. 131 132 The following example launches a container with the label ``app=benchmark`` 133 which is then associated with the endpoint. The label is prefixed with 134 ``container:`` to indicate that the label was derived from the container 135 runtime. 136 137 :: 138 139 $ docker run --net cilium -d -l app=benchmark tgraf/netperf 140 aaff7190f47d071325e7af06577f672beff64ccc91d2b53c42262635c063cf1c 141 $ cilium endpoint list 142 ENDPOINT POLICY IDENTITY LABELS (source:key[=value]) IPv6 IPv4 STATUS 143 ENFORCEMENT 144 62006 Disabled 257 container:app=benchmark f00d::a00:20f:0:f236 10.15.116.202 ready 145 146 147 An endpoint can have metadata associated from multiple sources. A typical 148 example is a Kubernetes cluster which uses containerd as the container runtime. 149 Endpoints will derive Kubernetes pod labels (prefixed with the ``k8s:`` source 150 prefix) and containerd labels (prefixed with ``container:`` source prefix). 151 152 .. _identity: 153 154 Identity 155 ======== 156 157 All `endpoints` are assigned an identity. The identity is what is used to enforce 158 basic connectivity between endpoints. In traditional networking terminology, 159 this would be equivalent to Layer 3 enforcement. 160 161 An identity is identified by `labels` and is given a cluster wide unique 162 identifier. The endpoint is assigned the identity which matches the endpoint's 163 `security relevant labels`, i.e. all endpoints which share the same set of 164 `security relevant labels` will share the same identity. This concept allows to 165 scale policy enforcement to a massive number of endpoints as many individual 166 endpoints will typically share the same set of security `labels` as applications 167 are scaled. 168 169 What is an Identity? 170 -------------------- 171 172 The identity of an endpoint is derived based on the `labels` associated with 173 the pod or container which are derived to the `endpoint`_. When a pod or 174 container is started, Cilium will create an `endpoint`_ based on the event 175 received by the container runtime to represent the pod or container on the 176 network. As a next step, Cilium will resolve the identity of the `endpoint`_ 177 created. Whenever the `labels` of the pod or container change, the identity is 178 reconfirmed and automatically modified as required. 179 180 .. _security relevant labels: 181 182 Security Relevant Labels 183 ------------------------ 184 185 Not all `labels` associated with a container or pod are meaningful when 186 deriving the `identity`. Labels may be used to store metadata such as the 187 timestamp when a container was launched. Cilium requires to know which labels 188 are meaningful and are subject to being considered when deriving the identity. 189 For this purpose, the user is required to specify a list of string prefixes of 190 meaningful labels. The standard behavior is to include all labels which start 191 with the prefix ``id.``, e.g. ``id.service1``, ``id.service2``, 192 ``id.groupA.service44``. The list of meaningful label prefixes can be specified 193 when starting the agent. 194 195 .. _reserved_labels: 196 197 Special Identities 198 ------------------ 199 200 All endpoints which are managed by Cilium will be assigned an identity. In 201 order to allow communication to network endpoints which are not managed by 202 Cilium, special identities exist to represent those. Special reserved 203 identities are prefixed with the string ``reserved:``. 204 205 +---------------------+---------------------------------------------------+ 206 | Identity | Description | 207 +=====================+===================================================+ 208 | reserved:unknown | The identity could not be derived. | 209 +---------------------+---------------------------------------------------+ 210 | reserved:host | The collection of all cluster hosts. Any traffic | 211 | | that originates from or is designated to one of | 212 | | the IPs of any host in the cluster is assigned the| 213 | | reserved:host identity. | 214 +---------------------+---------------------------------------------------+ 215 | reserved:world | Any network endpoint outside of the cluster | 216 +---------------------+---------------------------------------------------+ 217 | reserved:health | This is health checking traffic generated by | 218 | | Cilium agents. | 219 +---------------------+---------------------------------------------------+ 220 | reserved:init | An endpoint for which the identity has not yet | 221 | | been resolved is assigned the init identity. | 222 | | This represents the phase of an endpoint in which | 223 | | some of the metadata required to derive the | 224 | | security identity is still missing. This is | 225 | | typically the case in the bootstrapping phase. | 226 | | | 227 | | The init identity is only allocated if the labels | 228 | | of the endpoint are not known at creation time. | 229 | | This can be the case for the Docker plugin. | 230 +---------------------+---------------------------------------------------+ 231 | reserved:unmanaged | An endpoint that is not managed by Cilium, e.g. | 232 | | a Kubernetes pod that was launched before Cilium | 233 | | was installed. | 234 +---------------------+---------------------------------------------------+ 235 236 Well-known Identities 237 --------------------- 238 239 The following is a list of well-known identities which Cilium is aware of 240 automatically and will hand out a security identity without requiring to 241 contact any external dependencies such as the kvstore. The purpose of this is 242 to allow bootstrapping Cilium and enable network connectivity with policy 243 enforcement in the cluster for essential services without depending on any 244 dependencies. 245 246 ======================== =================== ==================== ================= =========== ============================================================================ 247 Deployment Namespace ServiceAccount Cluster Name Numeric ID Labels 248 ======================== =================== ==================== ================= =========== ============================================================================ 249 cilium-etcd-operator <cilium-namespace> cilium-etcd-operator <cilium-cluster> 107 ``name=cilium-etcd-operator``, ``io.cilium/app=etcd-operator`` 250 etcd-operator <cilium-namespace> cilium-etcd-sa <cilium-cluster> 100 ``io.cilium/app=etcd-operator`` 251 cilium-etcd <cilium-namespace> default <cilium-cluster> 101 ``app=etcd``, ``etcd_cluster=cilium-etcd``, ``io.cilium/app=etcd-operator`` 252 kube-dns kube-system kube-dns <cilium-cluster> 102 ``k8s-app=kube-dns`` 253 kube-dns (EKS) kube-system kube-dns <cilium-cluster> 103 ``k8s-app=kube-dns``, ``eks.amazonaws.com/component=kube-dns`` 254 core-dns kube-system coredns <cilium-cluster> 104 ``k8s-app=kube-dns`` 255 core-dns (EKS) kube-system coredns <cilium-cluster> 106 ``k8s-app=kube-dns``, ``eks.amazonaws.com/component=coredns`` 256 cilium-operator <cilium-namespace> cilium-operator <cilium-cluster> 105 ``name=cilium-operator``, ``io.cilium/app=operator`` 257 ======================== =================== ==================== ================= =========== ============================================================================ 258 259 *Note*: if ``cilium-cluster`` is not defined with the ``cluster-name`` option, 260 the default value will be set to "``default``". 261 262 Identity Management in the Cluster 263 ---------------------------------- 264 265 Identities are valid in the entire cluster which means that if several pods or 266 containers are started on several cluster nodes, all of them will resolve and 267 share a single identity if they share the identity relevant labels. This 268 requires coordination between cluster nodes. 269 270 .. image:: ../images/identity_store.png 271 :align: center 272 273 The operation to resolve an endpoint identity is performed with the help of the 274 distributed key-value store which allows to perform atomic operations in the 275 form *generate a new unique identifier if the following value has not been seen 276 before*. This allows each cluster node to create the identity relevant subset 277 of labels and then query the key-value store to derive the identity. Depending 278 on whether the set of labels has been queried before, either a new identity 279 will be created, or the identity of the initial query will be returned. 280 281 Node 282 ==== 283 284 Cilium refers to a node as an individual member of a cluster. Each node must be 285 running the ``cilium-agent`` and will operate in a mostly autonomous manner. 286 Synchronization of state between Cilium agent's running on different nodes is 287 kept to a minimum for simplicity and scale. It occurs exclusively via the 288 Key-Value store or with packet metadata. 289 290 Node Address 291 ------------ 292 293 Cilium will automatically detect the node's IPv4 and IPv6 address. The detected 294 node address is printed out when the ``cilium-agent`` starts: 295 296 :: 297 298 Local node-name: worker0 299 Node-IPv6: f00d::ac10:14:0:1 300 External-Node IPv4: 172.16.0.20 301 Internal-Node IPv4: 10.200.28.238 302