github.com/aporeto-inc/trireme-lib@v10.358.0+incompatible/docs/docker_host_networks.md

github.com/aporeto-inc/trireme-lib@v10.358.0+incompatible/docs/docker_host_networks.md (about)

1 # Docker Host Networks and Trireme
2
3 Docker and most other container systems have several options for networking.
4 In most cases, people will choose to use bridge networks or some overlay.
5 However, for some particular use cases, there is a need for direct access to
6 a container to the host network. For example, Consul has a use case of
7 exposing a DNS server using host networks. The RedHat OpenShift router or
8 several ingress implementations in Kubernetes instantiate the ingress proxy
9 on a host network so that they can fix its IP address.
10 (see https://docs.docker.com/engine/userguide/networking/
11 or
12 https://docs.openshift.com/enterprise/3.0/install_config/install/deploy_router.html
13 for some examples).
14
15 In general exposing a container directly on the host network poses some
16 challenges from a security perspective. The container is essentially
17 attached to the host namespace and can freely interact with any endpoint
18 with which the host network can communicate. This situation can be
19 especially dangerous for an ingress proxy that is exposed to the Internet.
20 Even a non-privileged container with host network access can create a
21 security vulnerability.
22
23 Trireme de-couples security from networking and since it treats containers
24 and Linux processes as equal, it can give you some additional protections
25 when your implementation requires access to the host network. Essentially
26 with Trireme, you can still isolate the container from a networking
27 perspective, even though it uses the same IP address and ports as your host.
28
29 ## TL;DR Show me how
30 To illustrate how to use Trireme with Docker host networks, we will use the
31 Trireme example with default settings.
32 1. Download and build trireme-example from
33 https://go.aporeto.io/trireme-example
34 Follow the instructions in the Readme file and make sure all the
35 dependencies are installed in your system.
36 2. Start trireme-example in one window
37
38 ```bash
39 % sudo trireme-example daemon --hybrid
40 ```
41
42 3. Start an Nginx container with host network access. In other words, you
43 Nginx container will be accessible through the host interface without any
44 network address translations.
45
46 ```bash
47 % docker run -l app=nginx --net=host -d nginx
48 ```
49
50 If you want to verify that your container is running in host mode, just
51 issue the command
52
53 ```bash
54 % docker inspect <container id> | grep NetworkMode
55 "NetworkMode" : "host"
56 ```
57
58 4. Despite the fact that your container is running in host mode, it is
59 still protected by Trireme and it cannot be actually accessed. The
60 default policy in Trireme allows two containers or processes to interact
61 only if they are both protected by Trireme and they have the same labels.
62 Try:
63
64 ```bash
65 % curl http://127.0.0.1
66 ```
67
68 This should fail.
69
70 5. Instantiate a curl container now with the same labels as the Nginx
71 container that we just started.
72
73 ```bash
74 % docker run -l app=nginx -it nhoag/curl
75 ```
76
77 Assuming that your local docker bridge is at 172.17.0.1 (the default in
78 docker) the nginx container should be accessible through the bridge IP.
79 Initiate a curl command to the bridge IP
80
81 ```bash
82 root@b84a73c6d5ba:# curl http://172.17.0.1
83 ```
84
85 You will see that curl succeeded in this case. You can now exit from the
86 curl container.
87
88 6. Since Trireme also supports Linux Processes through we can actually
89 access the container from the host as well, provided that we use Trireme
90 to control the network capabilities of the Linux process. From your host
91 cell, just issue the command:
92
93 ```bash
94 trireme-example run --label=app=nginx curl -- http://127.0.0.1
95 ```
96
97 This command should succeed and you should see the Nginx output. Note, that we start the curl process with the same labels as the Nginx container.
98
99 ## What Did We Do?
100
101 We started a docker container with the net=host parameter. The effect of
102 this parameter is that the container uses the host network namespace and
103 has direct access to the interfaces and the network of the host. Doing
104 that without extra controls poses security risks. Trireme allows you to
105 protect even containers started in the host network namespace. Since the
106 container was protected by default by Trireme we instantiated another
107 container and a Linux process and demonstrated how to use the Trireme
108 policies to control which network or Linux process can interact with the
109 host network container.
110
111 ## Trireme and Host Networking Architecture
112
113 As we explained in some of the previous sections, Trireme treats containers
114 and Linux processes equally from a network security perspective. Trireme
115 can apply granular policy equally well to a container or a Linux process.
116
117 When a container is activated in host network mode, Trireme detects this
118 activation. Instead of giving full access to the container, it treats it as
119 a Linux process. It gets the first process (Pid : 1 ) that is run in the
120 container and places it on a dedicated net_cls cgroup as it would do with
121 Linux processes. All subsequent processes instantiated/forked inside the
122 container inherit by default the same policy. It can then apply the
123 granular policy to the particular container, even though the container is
124 in the same namespace as the host network. This policy does not affect
125 any other process or container running on the same host.
126
127 This capability makes Trireme very useful in environments that you need
128 to implement some containers with host network mode.
129
130 ## Taking it to the extreme
131 The Trireme isolation for host networks is of course not as strong as a
132 full network namespace isolation. Containers will still share several
133 others of the stack. However, in several cases, users are looking for a
134 less granular isolation, since networking is a pain. See
135 (https://medium.com/@copyconstruct/schedulers-kubernetes-and-nomad-b0f2e14a896)
136 for some discussion on the topic.
137
138 Even the original Borg architecture used this approach to minimize network
139 complexities. From the corresponding ACM article
140
141 "All containers running on a Borg machine share the host’s IP address, so
142 Borg assigns the containers unique port numbers as part of the scheduling
143 process."
144
145 Although this is not optimal, there are several implementations of docker
146 in production environments that have decided to take the risk and
147 instantiate all containers in host network mode since they don't want to
148 adapt their applications to the concept of either random ports (docker
149 bridge approach) or IP per container (Kubernetes default). They just did not
150 want to deal with the complexities of networking. One can not discount
151 the pragmatic operational reasons that are leading teams down this path.
152
153 Obviously, there is an isolation risk with this decision and Trireme can
154 bridge this gap. By using Trireme as the network policy mechanism, we have
155 decoupled security from the network. Security is delegated to an end-to-end
156 authorization function, and the fact that containers live in the same
157 network namespace does not affect the capability of Trireme to provide
158 strong isolation.
159
160 Therefore, one can use Trireme to implement a container based system
161 without the need for complex networking. Yes, there are architectural
162 limitations in some of these choices, but nevertheless it is useful in
163 some environments.