github.com/datadog/cilium@v1.6.12/Documentation/concepts/datapath.rst (about)

     1  .. only:: not (epub or latex or html)
     2  
     3      WARNING: You are looking at unreleased Cilium documentation.
     4      Please use the official rendered version released here:
     5      http://docs.cilium.io
     6  
     7  .. _concepts_datapath:
     8  
     9  ********
    10  Datapath
    11  ********
    12  
    13  .. _aws_eni_datapath:
    14  
    15  AWS ENI
    16  =======
    17  
    18  The AWS ENI datapath is enabled when Cilium is run with the option
    19  ``--ipam=eni``. It is a special purpose datapath that is useful when running
    20  Cilium in an AWS environment.
    21  
    22  Advantages of the model
    23  -----------------------
    24  
    25  * Pods are assigned ENI IPs which are directly routable in the AWS VPC. This
    26    simplifies communication of pod traffic within VPCs and avoids the need for
    27    SNAT.
    28  
    29  * Pod IPs are assigned a security group. The security groups for pods are
    30    configured per node which allows to create node pools and give different
    31    security group assignments to different pods. See section :ref:`ipam_eni` for
    32    more details.
    33  
    34  Disadvantages of this model
    35  ---------------------------
    36  
    37  * The number of ENI IPs is limited per instance. The limit depends on the EC2
    38    instance type. This can become a problem when attempting to run a larger
    39    number of pods on very small instance types.
    40  
    41  * Allocation of ENIs and ENI IPs requires interaction with the EC2 API which is
    42    subject to rate limiting. This is primarily mitigated via the operator
    43    design, see section :ref:`ipam_eni` for more details.
    44  
    45  Architecture
    46  ------------
    47  
    48  Ingress
    49  ~~~~~~~
    50  
    51  1. Traffic is received on one of the ENIs attached to the instance which is
    52     represented on the node as interface ``ethN``.
    53  
    54  2. An IP routing rule ensures that traffic to all local pod IPs is done using
    55     the main routing table:
    56  
    57     .. code-block:: bash
    58  
    59         20:	from all to 192.168.105.44 lookup main
    60  
    61  3. The main routing table contains an exact match route to steer traffic into a
    62     veth pair which is hooked into the pod:
    63  
    64     .. code-block:: bash
    65  
    66         192.168.105.44 dev lxc5a4def8d96c5
    67  
    68  4. All traffic passing ``lxc5a4def8d96c5`` on the way into the pod is subject
    69     to Cilium's BPF program to enforce network policies, provide service reverse
    70     load-balancing, and visibility.
    71  
    72  Egress
    73  ~~~~~~
    74  
    75  1. The pod's network namespace contains a default route which points to the
    76     node's router IP via the veth pair which is named ``eth0`` inside of the pod
    77     and ``lxcXXXXXX`` in the host namespace. The router IP is allocated from the
    78     ENI space, allowing for sending of ICMP errors from the router IP for Path
    79     MTU purposes.
    80  
    81  2. After passing through the veth pair and before reaching the Linux routing
    82     layer, all traffic is subject to Cilium's BPF program to enforce network
    83     policies, implement load-balancing and provide networking features.
    84  
    85  3. An IP routing rule ensures that traffic from individual endpoints are using
    86     a routing table specific to the ENI from which the endpoint IP was
    87     allocated:
    88  
    89     .. code-block:: bash
    90  
    91         30:	from 192.168.105.44 to 192.168.0.0/16 lookup 92
    92  
    93  4. The ENI specific routing table contains a default route which redirects
    94     to the router of the VPC via the ENI interface:
    95  
    96     .. code-block:: bash
    97  
    98         default via 192.168.0.1 dev eth2
    99         192.168.0.1 dev eth2
   100  
   101  
   102  Configuration
   103  -------------
   104  
   105  The AWS ENI datapath is enabled by setting the following option:
   106  
   107  .. code-block: yaml
   108  
   109          ipam: eni
   110          blacklist-conflicting-routes: "false"
   111          enable-endpoint-routes: "true"
   112          auto-create-cilium-node-resource: "true"
   113          egress-masquerade-interfaces: eth+
   114  
   115  * ``ipam: eni`` Enables the ENI specific IPAM backend and indicates to the
   116    datapath that ENI IPs will be used.
   117  
   118  * ``blacklist-conflicting-routes: "false"`` disables blacklisting of local
   119    routes. This is required as routes will exist covering ENI IPs pointing to
   120    interfaces that are not owned by Cilium. If blacklisting is not disabled, all
   121    ENI IPs would be considered used by another networking component.
   122  
   123  * ``enable-endpoint-routes: "true"`` enables direct routing to the ENI
   124    veth pairs without requiring to route via the ``cilium_host`` interface.
   125  
   126  * ``auto-create-cilium-node-resource: "true"`` enables the automatic creation of
   127    the ``CiliumNode`` custom resource with all required ENI parameters. It is
   128    possible to disable this and provide the custom resource manually.
   129  
   130  * ``egress-masquerade-interfaces: eth+`` is the interface selector of all
   131    interfaces which are subject to masquerading. Masquerading can be disabled
   132    entirely with ``masquerade: "false"``.
   133  
   134  See the section :ref:`ipam_eni` for details on how to configure ENI IPAM
   135  specific parameters.
   136