github.com/cilium/cilium@v1.16.2/Documentation/gettingstarted/terminology.rst (about) 1 .. only:: not (epub or latex or html) 2 3 WARNING: You are looking at unreleased Cilium documentation. 4 Please use the official rendered version released here: 5 https://docs.cilium.io 6 7 *********** 8 Terminology 9 *********** 10 11 12 .. _label: 13 .. _labels: 14 15 Labels 16 ====== 17 18 Labels are a generic, flexible and highly scalable way of addressing a large 19 set of resources as they allow for arbitrary grouping and creation of sets. 20 Whenever something needs to be described, addressed or selected, it is done 21 based on labels: 22 23 - `Endpoints` are assigned labels as derived from the container runtime, 24 orchestration system, or other sources. 25 - `Network policies` select pairs of `endpoints` which are allowed to 26 communicate based on labels. The policies themselves are identified by labels 27 as well. 28 29 What is a Label? 30 ---------------- 31 32 A label is a pair of strings consisting of a ``key`` and ``value``. A label can 33 be formatted as a single string with the format ``key=value``. The key portion 34 is mandatory and must be unique. This is typically achieved by using the 35 reverse domain name notion, e.g. ``io.cilium.mykey=myvalue``. The value portion 36 is optional and can be omitted, e.g. ``io.cilium.mykey``. 37 38 Key names should typically consist of the character set ``[a-z0-9-.]``. 39 40 When using labels to select resources, both the key and the value must match, 41 e.g. when a policy should be applied to all endpoints with the label 42 ``my.corp.foo`` then the label ``my.corp.foo=bar`` will not match the 43 selector. 44 45 Label Source 46 ------------ 47 48 A label can be derived from various sources. For example, an `endpoint`_ will 49 derive the labels associated to the container by the local container runtime as 50 well as the labels associated with the pod as provided by Kubernetes. As these 51 two label namespaces are not aware of each other, this may result in 52 conflicting label keys. 53 54 To resolve this potential conflict, Cilium prefixes all label keys with 55 ``source:`` to indicate the source of the label when importing labels, e.g. 56 ``k8s:role=frontend``, ``container:user=joe``, ``k8s:role=backend``. This means 57 that when you run a Docker container using ``docker run [...] -l foo=bar``, the 58 label ``container:foo=bar`` will appear on the Cilium endpoint representing the 59 container. Similarly, a Kubernetes pod started with the label ``foo: bar`` 60 will be represented with a Cilium endpoint associated with the label 61 ``k8s:foo=bar``. A unique name is allocated for each potential source. The 62 following label sources are currently supported: 63 64 - ``container:`` for labels derived from the local container runtime 65 - ``k8s:`` for labels derived from Kubernetes 66 - ``reserved:`` for special reserved labels, see :ref:`reserved_labels`. 67 - ``unspec:`` for labels with unspecified source 68 69 When using labels to identify other resources, the source can be included to 70 limit matching of labels to a particular type. If no source is provided, the 71 label source defaults to ``any:`` which will match all labels regardless of 72 their source. If a source is provided, the source of the selecting and matching 73 labels need to match. 74 75 .. _endpoint: 76 .. _endpoints: 77 78 Endpoint 79 ========= 80 81 Cilium makes application containers available on the network by assigning them 82 IP addresses. Multiple application containers can share the same IP address; a 83 typical example for this model is a Kubernetes :term:`Pod`. All application containers 84 which share a common address are grouped together in what Cilium refers to as 85 an endpoint. 86 87 Allocating individual IP addresses enables the use of the entire Layer 4 port 88 range by each endpoint. This essentially allows multiple application containers 89 running on the same cluster node to all bind to well known ports such as ``80`` 90 without causing any conflicts. 91 92 The default behavior of Cilium is to assign both an IPv6 and IPv4 address to 93 every endpoint. However, this behavior can be configured to only allocate an 94 IPv6 address with the ``--enable-ipv4=false`` option. If both an IPv6 and IPv4 95 address are assigned, either address can be used to reach the endpoint. The 96 same behavior will apply with regard to policy rules, load-balancing, etc. See 97 :ref:`address_management` for more details. 98 99 Identification 100 -------------- 101 102 For identification purposes, Cilium assigns an internal endpoint id to all 103 endpoints on a cluster node. The endpoint id is unique within the context of 104 an individual cluster node. 105 106 .. _endpoint id: 107 108 Endpoint Metadata 109 ----------------- 110 111 An endpoint automatically derives metadata from the application containers 112 associated with the endpoint. The metadata can then be used to identify the 113 endpoint for security/policy, load-balancing and routing purposes. 114 115 The source of the metadata will depend on the orchestration system and 116 container runtime in use. The following metadata retrieval mechanisms are 117 currently supported: 118 119 +---------------------+---------------------------------------------------+ 120 | System | Description | 121 +=====================+===================================================+ 122 | Kubernetes | Pod labels (via k8s API) | 123 +---------------------+---------------------------------------------------+ 124 | containerd (Docker) | Container labels (via Docker API) | 125 +---------------------+---------------------------------------------------+ 126 127 Metadata is attached to endpoints in the form of `labels`. 128 129 The following example launches a container with the label ``app=benchmark`` 130 which is then associated with the endpoint. The label is prefixed with 131 ``container:`` to indicate that the label was derived from the container 132 runtime. 133 134 .. code-block:: shell-session 135 136 $ docker run --net cilium -d -l app=benchmark tgraf/netperf 137 aaff7190f47d071325e7af06577f672beff64ccc91d2b53c42262635c063cf1c 138 $ cilium-dbg endpoint list 139 ENDPOINT POLICY IDENTITY LABELS (source:key[=value]) IPv6 IPv4 STATUS 140 ENFORCEMENT 141 62006 Disabled 257 container:app=benchmark f00d::a00:20f:0:f236 10.15.116.202 ready 142 143 144 An endpoint can have metadata associated from multiple sources. A typical 145 example is a Kubernetes cluster which uses containerd as the container runtime. 146 Endpoints will derive Kubernetes pod labels (prefixed with the ``k8s:`` source 147 prefix) and containerd labels (prefixed with ``container:`` source prefix). 148 149 .. _identity: 150 151 Identity 152 ======== 153 154 All `endpoints` are assigned an identity. The identity is what is used to enforce 155 basic connectivity between endpoints. In traditional networking terminology, 156 this would be equivalent to Layer 3 enforcement. 157 158 An identity is identified by `labels` and is given a cluster wide unique 159 identifier. The endpoint is assigned the identity which matches the endpoint's 160 `security relevant labels`, i.e. all endpoints which share the same set of 161 `security relevant labels` will share the same identity. This concept allows to 162 scale policy enforcement to a massive number of endpoints as many individual 163 endpoints will typically share the same set of security `labels` as applications 164 are scaled. 165 166 What is an Identity? 167 -------------------- 168 169 The identity of an endpoint is derived based on the `labels` associated with 170 the pod or container which are derived to the `endpoint`_. When a pod or 171 container is started, Cilium will create an `endpoint`_ based on the event 172 received by the container runtime to represent the pod or container on the 173 network. As a next step, Cilium will resolve the identity of the `endpoint`_ 174 created. Whenever the `labels` of the pod or container change, the identity is 175 reconfirmed and automatically modified as required. 176 177 .. _security relevant labels: 178 179 Security Relevant Labels 180 ------------------------ 181 182 Not all `labels` associated with a container or pod are meaningful when 183 deriving the `identity`. Labels may be used to store metadata such as the 184 timestamp when a container was launched. Cilium requires to know which labels 185 are meaningful and are subject to being considered when deriving the identity. 186 For this purpose, the user is required to specify a list of string prefixes of 187 meaningful labels. The standard behavior is to include all labels which start 188 with the prefix ``id.``, e.g. ``id.service1``, ``id.service2``, 189 ``id.groupA.service44``. The list of meaningful label prefixes can be specified 190 when starting the agent. 191 192 .. _reserved_labels: 193 194 Special Identities 195 ------------------ 196 197 All endpoints which are managed by Cilium will be assigned an identity. In 198 order to allow communication to network endpoints which are not managed by 199 Cilium, special identities exist to represent those. Special reserved 200 identities are prefixed with the string ``reserved:``. 201 202 +-----------------------------+------------+---------------------------------------------------+ 203 | Identity | Numeric ID | Description | 204 +=============================+============+===================================================+ 205 | ``reserved:unknown`` | 0 | The identity could not be derived. | 206 +-----------------------------+------------+---------------------------------------------------+ 207 | ``reserved:host`` | 1 | The local host. Any traffic that originates from | 208 | | | or is designated to one of the local host IPs. | 209 +-----------------------------+------------+---------------------------------------------------+ 210 | ``reserved:world`` | 2 | Any network endpoint outside of the cluster | 211 +-----------------------------+------------+---------------------------------------------------+ 212 | ``reserved:unmanaged`` | 3 | An endpoint that is not managed by Cilium, e.g. | 213 | | | a Kubernetes pod that was launched before Cilium | 214 | | | was installed. | 215 +-----------------------------+------------+---------------------------------------------------+ 216 | ``reserved:health`` | 4 | This is health checking traffic generated by | 217 | | | Cilium agents. | 218 +-----------------------------+------------+---------------------------------------------------+ 219 | ``reserved:init`` | 5 | An endpoint for which the identity has not yet | 220 | | | been resolved is assigned the init identity. | 221 | | | This represents the phase of an endpoint in which | 222 | | | some of the metadata required to derive the | 223 | | | security identity is still missing. This is | 224 | | | typically the case in the bootstrapping phase. | 225 | | | | 226 | | | The init identity is only allocated if the labels | 227 | | | of the endpoint are not known at creation time. | 228 | | | This can be the case for the Docker plugin. | 229 +-----------------------------+------------+---------------------------------------------------+ 230 | ``reserved:remote-node`` | 6 | The collection of all remote cluster hosts. | 231 | | | Any traffic that originates from or is designated | 232 | | | to one of the IPs of any host in any connected | 233 | | | cluster other than the local node. | 234 +-----------------------------+------------+---------------------------------------------------+ 235 | ``reserved:kube-apiserver`` | 7 | Remote node(s) which have backend(s) serving the | 236 | | | kube-apiserver running. | 237 +-----------------------------+------------+---------------------------------------------------+ 238 | ``reserved:ingress`` | 8 | Given to the IPs used as the source address for | 239 | | | connections from Ingress proxies. | 240 +-----------------------------+------------+---------------------------------------------------+ 241 242 Well-known Identities 243 --------------------- 244 245 The following is a list of well-known identities which Cilium is aware of 246 automatically and will hand out a security identity without requiring to 247 contact any external dependencies such as the kvstore. The purpose of this is 248 to allow bootstrapping Cilium and enable network connectivity with policy 249 enforcement in the cluster for essential services without depending on any 250 dependencies. 251 252 ======================== =================== ==================== ================= =========== ============================================================================ 253 Deployment Namespace ServiceAccount Cluster Name Numeric ID Labels 254 ======================== =================== ==================== ================= =========== ============================================================================ 255 kube-dns kube-system kube-dns <cilium-cluster> 102 ``k8s-app=kube-dns`` 256 kube-dns (EKS) kube-system kube-dns <cilium-cluster> 103 ``k8s-app=kube-dns``, ``eks.amazonaws.com/component=kube-dns`` 257 core-dns kube-system coredns <cilium-cluster> 104 ``k8s-app=kube-dns`` 258 core-dns (EKS) kube-system coredns <cilium-cluster> 106 ``k8s-app=kube-dns``, ``eks.amazonaws.com/component=coredns`` 259 cilium-operator <cilium-namespace> cilium-operator <cilium-cluster> 105 ``name=cilium-operator``, ``io.cilium/app=operator`` 260 ======================== =================== ==================== ================= =========== ============================================================================ 261 262 *Note*: if ``cilium-cluster`` is not defined with the ``cluster-name`` option, 263 the default value will be set to "``default``". 264 265 Identity Management in the Cluster 266 ---------------------------------- 267 268 Identities are valid in the entire cluster which means that if several pods or 269 containers are started on several cluster nodes, all of them will resolve and 270 share a single identity if they share the identity relevant labels. This 271 requires coordination between cluster nodes. 272 273 .. image:: ../images/identity_store.png 274 :align: center 275 276 The operation to resolve an endpoint identity is performed with the help of the 277 distributed key-value store which allows to perform atomic operations in the 278 form *generate a new unique identifier if the following value has not been seen 279 before*. This allows each cluster node to create the identity relevant subset 280 of labels and then query the key-value store to derive the identity. Depending 281 on whether the set of labels has been queried before, either a new identity 282 will be created, or the identity of the initial query will be returned. 283 284 .. _node: 285 286 Node 287 ==== 288 289 Cilium refers to a node as an individual member of a cluster. Each node must be 290 running the ``cilium-agent`` and will operate in a mostly autonomous manner. 291 Synchronization of state between Cilium agents running on different nodes is 292 kept to a minimum for simplicity and scale. It occurs exclusively via the 293 Key-Value store or with packet metadata. 294 295 Node Address 296 ------------ 297 298 Cilium will automatically detect the node's IPv4 and IPv6 address. The detected 299 node address is printed out when the ``cilium-agent`` starts: 300 301 :: 302 303 Local node-name: worker0 304 Node-IPv6: f00d::ac10:14:0:1 305 External-Node IPv4: 172.16.0.20 306 Internal-Node IPv4: 10.200.28.238 307