github.com/looshlee/cilium@v1.6.12/Documentation/kvstore.rst (about)

     1  Key-Value Store
     2  ###############
     3  
     4  Cilium uses an external key-value store to exchange information across multiple
     5  Cilium instances:
     6  
     7  Layout
     8  ======
     9  
    10  All data is stored under a common key prefix:
    11  
    12  ===================== ====================
    13  Prefix                Description
    14  ===================== ====================
    15  ``cilium/``           All keys share this common prefix.
    16  ``cilium/state/``     State stored by agents, data is automatically recreated on removal or corruption.
    17  ===================== ====================
    18  
    19  
    20  Cluster Nodes
    21  -------------
    22  
    23  Every agent will register itself as a node in the kvstore and make the
    24  following information available to other agents:
    25  
    26  - Name
    27  - IP addresses of the node
    28  - Health checking IP addresses
    29  - Allocation range of endpoints on the node
    30  
    31  ============================================================ ====================
    32  Key                                                          Value
    33  ============================================================ ====================
    34  ``cilium/state/nodes/v1/<cluster>/<node>``                   node.Node_
    35  ============================================================ ====================
    36  
    37  .. _node.Node: https://godoc.org/github.com/cilium/cilium/pkg/node#Node
    38  
    39  All node keys are attached to a lease owned by the agent of the respective
    40  node.
    41  
    42  
    43  Services
    44  --------
    45  
    46  All Kubernetes services are mirrored into the kvstore by the Cilium operator. This is
    47  required to implement multi cluster service discovery.
    48  
    49  ============================================================= ====================
    50  Key                                                           Value
    51  ============================================================= ====================
    52  ``cilium/state/services/v1/<cluster>/<namespace>/<service>``  service.ClusterService_
    53  ============================================================= ====================
    54  
    55  .. _service.ClusterService: https://godoc.org/github.com/cilium/cilium/pkg/service#ClusterService
    56  
    57  Identities
    58  ----------
    59  
    60  Any time a new endpoint is started on a Cilium node, it will determine whether
    61  the labels for the endpoint are unique and allocate an identity for that set of
    62  labels. These identities are only meaningful within the local cluster.
    63  
    64  ============================================================= ====================
    65  Key                                                           Value
    66  ============================================================= ====================
    67  ``cilium/state/identities/v1/id/<identity>``                  labels.LabelArray_
    68  ``cilium/state/identities/v1/value/<labels>/<node>``          identity.NumericIdentity_
    69  ============================================================= ====================
    70  
    71  .. _identity.NumericIdentity: https://godoc.org/github.com/cilium/cilium/pkg/identity#NumericIdentity
    72  .. _labels.LabelArray: https://godoc.org/github.com/cilium/cilium/pkg/labels#LabelArray
    73  
    74  Endpoints
    75  ---------
    76  
    77  All endpoint IPs and corresponding identities are mirrored to the kvstore by
    78  the agent on the node where the endpoint is launched, to allow peer nodes to
    79  configure egress policies to endpoints backed by these IPs.
    80  
    81  ============================================================= ====================
    82  Key                                                           Value
    83  ============================================================= ====================
    84  ``cilium/state/ip/v1/<cluster>/<ip>``                         identity.IPIdentityPair_
    85  ============================================================= ====================
    86  
    87  .. _identity.IPIdentityPair: https://godoc.org/github.com/cilium/cilium/pkg/identity#IPIdentityPair
    88  
    89  Leases
    90  ======
    91  
    92  With a few exceptions, all keys in the key-value store are owned by a
    93  particular agent running on a node. All such keys have a lease attached. The
    94  lease is renewed automatically. When the lease expires, the key is removed from
    95  the key-value store. This guarantees that keys are removed from the key-value
    96  store in the event that an agent dies on a particular and never reappears.
    97  
    98  The lease lifetime is set to 15 minutes. The exact expiration behavior is
    99  dependent on the kvstore implementation but the expiration typically occurs
   100  after double the lifetime
   101  
   102  Debugging
   103  =========
   104  
   105  The contents stored in the kvstore can be queued and manipulate using the
   106  ``cilium kvstore`` command. For additional details, see the command reference.
   107  
   108  Example:
   109  
   110  .. code:: bash
   111  
   112          $ cilium kvstore get --recursive cilium/state/nodes/
   113          cilium/state/nodes/v1/default/runtime1 => {"Name":"runtime1","IPAddresses":[{"AddressType":"InternalIP","IP":"10.0.2.15"}],"IPv4AllocCIDR":{"IP":"10.11.0.0","Mask":"//8AAA=="},"IPv6AllocCIDR":{"IP":"f00d::a0f:0:0:0","Mask":"//////////////////8AAA=="},"IPv4HealthIP":"","IPv6HealthIP":""}