github.com/kubearmor/cilium@v1.6.12/README.rst (about) 1 |logo| 2 3 |cii| |build-status| |pulls| |slack| |go-report| |go-doc| |rtd| |apache| |gpl| 4 5 Cilium is open source software for providing and transparently securing network 6 connectivity and loadbalancing between application workloads such as 7 application containers or processes. Cilium operates at Layer 3/4 to provide 8 traditional networking and security services as well as Layer 7 to protect and 9 secure use of modern application protocols such as HTTP, gRPC and Kafka. Cilium 10 is integrated into common orchestration frameworks such as Kubernetes and Mesos. 11 12 A new Linux kernel technology called BPF is at the foundation of Cilium. It 13 supports dynamic insertion of BPF bytecode into the Linux kernel at various 14 integration points such as: network IO, application sockets, and tracepoints to 15 implement security, networking and visibility logic. BPF is highly efficient 16 and flexible. To learn more about BPF, read more in our extensive 17 `BPF and XDP Reference Guide`_. 18 19 .. image:: https://cdn.rawgit.com/cilium/cilium/master/Documentation/images/cilium-arch.png 20 :align: center 21 22 Functionality Overview 23 ====================== 24 25 .. begin-functionality-overview 26 27 Protect and secure APIs transparently 28 ------------------------------------- 29 30 Ability to secure modern application protocols such as REST/HTTP, gRPC and 31 Kafka. Traditional firewalls operates at Layer 3 and 4. A protocol running on a 32 particular port is either completely trusted or blocked entirely. Cilium 33 provides the ability to filter on individual application protocol requests such 34 as: 35 36 - Allow all HTTP requests with method ``GET`` and path ``/public/.*``. Deny all 37 other requests. 38 - Allow ``service1`` to produce on Kafka topic ``topic1`` and ``service2`` to 39 consume on ``topic1``. Reject all other Kafka messages. 40 - Require the HTTP header ``X-Token: [0-9]+`` to be present in all REST calls. 41 42 See the section `Layer 7 Policy`_ in our documentation for the latest list of 43 supported protocols and examples on how to use it. 44 45 Secure service to service communication based on identities 46 ----------------------------------------------------------- 47 48 Modern distributed applications rely on technologies such as application 49 containers to facilitate agility in deployment and scale out on demand. This 50 results in a large number of application containers to be started in a short 51 period of time. Typical container firewalls secure workloads by filtering on 52 source IP addresses and destination ports. This concept requires the firewalls 53 on all servers to be manipulated whenever a container is started anywhere in 54 the cluster. 55 56 In order to avoid this situation which limits scale, Cilium assigns a security 57 identity to groups of application containers which share identical security 58 policies. The identity is then associated with all network packets emitted by 59 the application containers, allowing to validate the identity at the receiving 60 node. Security identity management is performed using a key-value store. 61 62 Secure access to and from external services 63 ------------------------------------------- 64 65 Label based security is the tool of choice for cluster internal access control. 66 In order to secure access to and from external services, traditional CIDR based 67 security policies for both ingress and egress are supported. This allows to 68 limit access to and from application containers to particular IP ranges. 69 70 Simple Networking 71 ----------------- 72 73 A simple flat Layer 3 network with the ability to span multiple clusters 74 connects all application containers. IP allocation is kept simple by using host 75 scope allocators. This means that each host can allocate IPs without any 76 coordination between hosts. 77 78 The following multi node networking models are supported: 79 80 * **Overlay:** Encapsulation-based virtual network spanning all hosts. 81 Currently VXLAN and Geneve are baked in but all encapsulation formats 82 supported by Linux can be enabled. 83 84 When to use this mode: This mode has minimal infrastructure and integration 85 requirements. It works on almost any network infrastructure as the only 86 requirement is IP connectivity between hosts which is typically already 87 given. 88 89 * **Native Routing:** Use of the regular routing table of the Linux host. 90 The network is required to be capable to route the IP addresses of the 91 application containers. 92 93 When to use this mode: This mode is for advanced users and requires some 94 awareness of the underlying networking infrastructure. This mode works well 95 with: 96 97 - Native IPv6 networks 98 - In conjunction with cloud network routers 99 - If you are already running routing daemons 100 101 Load balancing 102 -------------- 103 104 Distributed load balancing for traffic between application containers and to 105 external services. The loadbalancing is implemented using BPF using efficient 106 hashtables allowing for almost unlimited scale and supports direct server 107 return (DSR) if the loadbalancing operation is not performed on the source 108 host. 109 *Note: load balancing requires connection tracking to be enabled. This is the 110 default.* 111 112 Monitoring and Troubleshooting 113 ------------------------------ 114 115 The ability to gain visibility and to troubleshoot issues is fundamental to the 116 operation of any distributed system. While we learned to love tools like 117 ``tcpdump`` and ``ping`` and while they will always find a special place in our 118 hearts, we strive to provide better tooling for troubleshooting. This includes 119 tooling to provide: 120 121 - Event monitoring with metadata: When a packet is dropped, the tool doesn't 122 just report the source and destination IP of the packet, the tool provides 123 the full label information of both the sender and receiver among a lot of 124 other information. 125 126 - Policy decision tracing: Why is a packet being dropped or a request rejected. 127 The policy tracing framework allows to trace the policy decision process for 128 both, running workloads and based on arbitrary label definitions. 129 130 - Metrics export via Prometheus: Key metrics are exported via Prometheus for 131 integration with your existing dashboards. 132 133 Integrations 134 ------------ 135 136 * Network plugin integrations: CNI_, libnetwork_ 137 * Container runtime events: containerd_ 138 * Kubernetes: NetworkPolicy_, Labels_, Ingress_, Service_ 139 140 .. _CNI: https://github.com/containernetworking/cni 141 .. _libnetwork: https://github.com/docker/libnetwork 142 .. _containerd: https://github.com/containerd/containerd 143 .. _service: https://kubernetes.io/docs/concepts/services-networking/service/ 144 .. _Ingress: https://kubernetes.io/docs/concepts/services-networking/ingress/ 145 .. _NetworkPolicy: https://kubernetes.io/docs/concepts/services-networking/network-policies/ 146 .. _Labels: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/ 147 .. _`Layer 7 Policy`: http://docs.cilium.io/en/stable/policy/#layer-7 148 149 .. end-functionality-overview 150 151 Getting Started 152 =============== 153 154 * `Why Cilium?`_ 155 * `Getting Started`_ 156 * `Architecture and Concepts`_ 157 * `Installing Cilium`_ 158 * `Frequently Asked Questions`_ 159 * Contributing_ 160 161 What is eBPF and XDP? 162 ===================== 163 164 Berkeley Packet Filter (BPF) is a Linux kernel bytecode interpreter originally 165 introduced to filter network packets, e.g. for tcpdump and socket filters. The 166 BPF instruction set and surrounding architecture has recently been 167 significantly reworked with additional data structures such as hash tables and 168 arrays for keeping state as well as additional actions to support packet 169 mangling, forwarding, encapsulation, etc. Furthermore, a compiler back end for 170 LLVM allows for programs to be written in C and compiled into BPF instructions. 171 An in-kernel verifier ensures that BPF programs are safe to run and a JIT 172 compiler converts the BPF bytecode to CPU architecture specific instructions 173 for native execution efficiency. BPF programs can be run at various hooking 174 points in the kernel such as for incoming packets, outgoing packets, system 175 calls, kprobes, uprobes, tracepoints, etc. 176 177 BPF continues to evolve and gain additional capabilities with each new Linux 178 release. Cilium leverages BPF to perform core data path filtering, mangling, 179 monitoring and redirection, and requires BPF capabilities that are in any Linux 180 kernel version 4.8.0 or newer (the latest current stable Linux kernel is 181 4.14.x). 182 183 Many Linux distributions including CoreOS, Debian, Docker's LinuxKit, Fedora, 184 openSUSE and Ubuntu already ship kernel versions >= 4.8.x. You can check your Linux 185 kernel version by running ``uname -a``. If you are not yet running a recent 186 enough kernel, check the Documentation of your Linux distribution on how to run 187 Linux kernel 4.9.x or later. 188 189 To read up on the necessary kernel versions to run the BPF runtime, see the 190 section Prerequisites_. 191 192 .. image:: https://cdn.rawgit.com/cilium/cilium/master/Documentation/images/bpf-overview.png 193 :align: center 194 195 XDP is a further step in evolution and enables to run a specific flavor of BPF 196 programs from the network driver with direct access to the packet's DMA buffer. 197 This is, by definition, the earliest possible point in the software stack, 198 where programs can be attached to in order to allow for a programmable, high 199 performance packet processor in the Linux kernel networking data path. 200 201 Further information about BPF and XDP targeted for developers can be found in 202 the `BPF and XDP Reference Guide`_. 203 204 205 Further Reading 206 =============== 207 208 .. further-reading-begin 209 210 Related Material 211 ---------------- 212 213 * `k8s-snowflake: Configs and scripts for bootstrapping an opinionated 214 Kubernetes cluster anywhere using Cilium plugin 215 <https://github.com/jessfraz/k8s-snowflake>`_ 216 * `Using Cilium for NetworkPolicy: Kubernetes documentation on how to use Cilium 217 to implement NetworkPolicy 218 <https://kubernetes.io/docs/tasks/administer-cluster/cilium-network-policy/>`_ 219 220 Presentations 221 ------------- 222 223 * DockerCon, Austin TX, Apr 2017 - Cilium - Network and Application Security with BPF and XDP: `Slides 224 <https://www.slideshare.net/ThomasGraf5/dockercon-2017-cilium-network-and-application-security-with-bpf-and-xdp>`__, `Video <https://www.youtube.com/watch?v=ilKlmTDdFgk>`__ 225 * CNCF/KubeCon Meetup, Berlin, Mar 2017 - Linux Native, HTTP Aware Network Security: 226 `Slides <https://www.slideshare.net/ThomasGraf5/linux-native-http-aware-network-security>`__, `Video <https://www.youtube.com/watch?v=Yf_INdTWIHI>`__ 227 * Docker Distributed Systems Summit, Berlin, Oct 2016: 228 `Slides <http://www.slideshare.net/Docker/cilium-bpf-xdp-for-containers-66969823>`__, `Video <https://www.youtube.com/watch?v=TnJF7ht3ZYc&list=PLkA60AVN3hh8oPas3cq2VA9xB7WazcIgs&index=7>`__ 229 * NetDev1.2, Tokyo, Sep 2016 - cls_bpf/eBPF updates since netdev 1.1: `Slides <http://borkmann.ch/talks/2016_tcws.pdf>`__, `Video <https://youtu.be/gwzaKXWIelc?t=12m55s>`__ 230 * NetDev1.2, Tokyo, Sep 2016 - Advanced programmability and recent updates with tc’s cls_bpf: `Slides <http://borkmann.ch/talks/2016_netdev2.pdf>`__, `Video <https://www.youtube.com/watch?v=GwT9hRiqdUo>`__ 231 * ContainerCon NA, Toronto, Aug 2016 - Fast IPv6 container networking with BPF & XDP: `Slides <http://www.slideshare.net/ThomasGraf5/cilium-fast-ipv6-container-networking-with-bpf-and-xdp>`__ 232 233 Podcasts 234 -------- 235 236 * Software Gone Wild by Ivan Pepelnjak, Oct 2016: `Blog <http://blog.ipspace.net/2016/10/fast-linux-packet-forwarding-with.html>`__, `MP3 <http://media.blubrry.com/ipspace/stream.ipspace.net/nuggets/podcast/Show_64-Cilium_with_Thomas_Graf.mp3>`__ 237 * OVS Orbit by Ben Pfaff, May 2016: `Blog <https://ovsorbit.benpfaff.org/#e4>`__, `MP3 <https://ovsorbit.benpfaff.org/episode-4.mp3>`__ 238 239 Community blog posts 240 -------------------- 241 242 * `Cilium for Network and Application Security with BPF and XDP, Apr 2017 243 <https://blog.scottlowe.org/2017/04/18/black-belt-cilium/>`_ 244 * `Cilium, BPF and XDP, Google Open Source Blog, Nov 2016 245 <https://opensource.googleblog.com/2016/11/cilium-networking-and-security.html>`_ 246 247 .. further-reading-end 248 249 Community 250 ========= 251 252 Slack 253 ----- 254 255 Join the Cilium `Slack channel <https://cilium.herokuapp.com/>`_ to chat with 256 Cilium developers and other Cilium users. This is a good place to learn about 257 Cilium, ask questions, and share your experiences. 258 259 Special Interest Groups (SIG) 260 ----------------------------- 261 262 See `Special Interest groups 263 <https://docs.cilium.io/en/stable/community/#special-interest-groups>`_ for a list of all SIGs and their meeting times. 264 265 Weekly Developer meeting 266 ------------------------ 267 * The developer community is hanging out on zoom on a weekly basis to chat. 268 Everybody is welcome. 269 * Weekly, Monday, 9:00 am PT, 12:00 pm (noon) ET, 6:00 pm CEST 270 * `Join zoom <https://zoom.us/j/328820525>`_ 271 272 License 273 ======= 274 275 The cilium user space components are licensed under the 276 `Apache License, Version 2.0 <LICENSE>`_. The BPF code templates are licensed 277 under the `General Public License, Version 2.0 <bpf/COPYING>`_. 278 279 .. _`Why Cilium?`: http://docs.cilium.io/en/stable/intro/#why-cilium 280 .. _`Getting Started`: http://docs.cilium.io/en/stable/gettingstarted/ 281 .. _`Architecture and Concepts`: http://docs.cilium.io/en/stable/concepts/ 282 .. _`Installing Cilium`: http://docs.cilium.io/en/stable/gettingstarted/#installation 283 .. _`Frequently Asked Questions`: https://github.com/cilium/cilium/issues?utf8=%E2%9C%93&q=is%3Aissue+label%3Akind%2Fquestion+ 284 .. _Contributing: http://docs.cilium.io/en/stable/contributing/development/ 285 .. _Prerequisites: http://docs.cilium.io/en/doc-1.0/install/system_requirements 286 .. _`BPF and XDP Reference Guide`: http://docs.cilium.io/en/stable/bpf/ 287 288 .. |logo| image:: https://cdn.rawgit.com/cilium/cilium/master/Documentation/images/logo.svg 289 :alt: Cilium Logo 290 :width: 350px 291 292 .. |build-status| image:: https://jenkins.cilium.io/job/cilium-ginkgo/job/cilium/job/master/badge/icon 293 :alt: Build Status 294 :scale: 100% 295 :target: https://jenkins.cilium.io/job/cilium-ginkgo/job/cilium/job/master/ 296 297 .. |go-report| image:: https://goreportcard.com/badge/github.com/cilium/cilium 298 :alt: Go Report Card 299 :target: https://goreportcard.com/report/github.com/cilium/cilium 300 301 .. |go-doc| image:: https://godoc.org/github.com/cilium/cilium?status.svg 302 :alt: GoDoc 303 :target: https://godoc.org/github.com/cilium/cilium 304 305 .. |rtd| image:: https://readthedocs.org/projects/docs/badge/?version=latest 306 :alt: Read the Docs 307 :target: http://docs.cilium.io/ 308 309 .. |apache| image:: https://img.shields.io/badge/license-Apache-blue.svg 310 :alt: Apache licensed 311 :target: https://github.com/cilium/cilium/blob/master/LICENSE 312 313 .. |gpl| image:: https://img.shields.io/badge/license-GPL-blue.svg 314 :alt: GPL licensed 315 :target: https://github.com/cilium/cilium/blob/master/bpf/COPYING 316 317 .. |slack| image:: https://cilium.herokuapp.com/badge.svg 318 :alt: Join the Cilium slack channel 319 :target: https://cilium.herokuapp.com/ 320 321 .. |cii| image:: https://bestpractices.coreinfrastructure.org/projects/1269/badge 322 :alt: CII Best Practices 323 :target: https://bestpractices.coreinfrastructure.org/projects/1269 324 325 .. |pulls| image:: https://img.shields.io/docker/pulls/cilium/cilium.svg 326 :alt: Cilium pulls 327 :target: https://hub.docker.com/r/cilium/cilium/tags/