github.com/docker/docker-ce@v17.12.1-ce-rc2+incompatible/components/cli/experimental/vlan-networks.md (about) 1 # Ipvlan Network Driver 2 3 ### Getting Started 4 5 The Ipvlan driver is currently in experimental mode in order to incubate Docker users use cases and vet the implementation to ensure a hardened, production ready driver in a future release. Libnetwork now gives users total control over both IPv4 and IPv6 addressing. The VLAN driver builds on top of that in giving operators complete control of layer 2 VLAN tagging and even Ipvlan L3 routing for users interested in underlay network integration. For overlay deployments that abstract away physical constraints see the [multi-host overlay ](https://docs.docker.com/engine/userguide/networking/get-started-overlay/) driver. 6 7 Ipvlan is a new twist on the tried and true network virtualization technique. The Linux implementations are extremely lightweight because rather than using the traditional Linux bridge for isolation, they are simply associated to a Linux Ethernet interface or sub-interface to enforce separation between networks and connectivity to the physical network. 8 9 Ipvlan offers a number of unique features and plenty of room for further innovations with the various modes. Two high level advantages of these approaches are, the positive performance implications of bypassing the Linux bridge and the simplicity of having fewer moving parts. Removing the bridge that traditionally resides in between the Docker host NIC and container interface leaves a simple setup consisting of container interfaces, attached directly to the Docker host interface. This result is easy access for external facing services as there is no need for port mappings in these scenarios. 10 11 ### Pre-Requisites 12 13 - The examples on this page are all single host and setup using Docker experimental builds that can be installed with the following instructions: [Install Docker experimental](https://github.com/docker/docker/tree/master/experimental) 14 15 - All of the examples can be performed on a single host running Docker. Any examples using a sub-interface like `eth0.10` can be replaced with `eth0` or any other valid parent interface on the Docker host. Sub-interfaces with a `.` are created on the fly. `-o parent` interfaces can also be left out of the `docker network create` all together and the driver will create a `dummy` interface that will enable local host connectivity to perform the examples. 16 17 - Kernel requirements: 18 19 - To check your current kernel version, use `uname -r` to display your kernel version 20 - Ipvlan Linux kernel v4.2+ (support for earlier kernels exists but is buggy) 21 22 ### Ipvlan L2 Mode Example Usage 23 24 An example of the ipvlan `L2` mode topology is shown in the following image. The driver is specified with `-d driver_name` option. In this case `-d ipvlan`. 25 26 ![Simple Ipvlan L2 Mode Example](images/ipvlan_l2_simple.png) 27 28 The parent interface in the next example `-o parent=eth0` is configured as followed: 29 30 ``` 31 $ ip addr show eth0 32 3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 33 inet 192.168.1.250/24 brd 192.168.1.255 scope global eth0 34 ``` 35 36 Use the network from the host's interface as the `--subnet` in the `docker network create`. The container will be attached to the same network as the host interface as set via the `-o parent=` option. 37 38 Create the ipvlan network and run a container attaching to it: 39 40 ``` 41 # Ipvlan (-o ipvlan_mode= Defaults to L2 mode if not specified) 42 $ docker network create -d ipvlan \ 43 --subnet=192.168.1.0/24 \ 44 --gateway=192.168.1.1 \ 45 -o ipvlan_mode=l2 \ 46 -o parent=eth0 db_net 47 48 # Start a container on the db_net network 49 $ docker run --net=db_net -it --rm alpine /bin/sh 50 51 # NOTE: the containers can NOT ping the underlying host interfaces as 52 # they are intentionally filtered by Linux for additional isolation. 53 ``` 54 55 The default mode for Ipvlan is `l2`. If `-o ipvlan_mode=` are left unspecified, the default mode will be used. Similarly, if the `--gateway` is left empty, the first usable address on the network will be set as the gateway. For example, if the subnet provided in the network create is `--subnet=192.168.1.0/24` then the gateway the container receives is `192.168.1.1`. 56 57 To help understand how this mode interacts with other hosts, the following figure shows the same layer 2 segment between two Docker hosts that applies to and Ipvlan L2 mode. 58 59 ![Multiple Ipvlan Hosts](images/macvlan-bridge-ipvlan-l2.png) 60 61 The following will create the exact same network as the network `db_net` created prior, with the driver defaults for `--gateway=192.168.1.1` and `-o ipvlan_mode=l2`. 62 63 ``` 64 # Ipvlan (-o ipvlan_mode= Defaults to L2 mode if not specified) 65 $ docker network create -d ipvlan \ 66 --subnet=192.168.1.0/24 \ 67 -o parent=eth0 db_net_ipv 68 69 # Start a container with an explicit name in daemon mode 70 $ docker run --net=db_net_ipv --name=ipv1 -itd alpine /bin/sh 71 72 # Start a second container and ping using the container name 73 # to see the docker included name resolution functionality 74 $ docker run --net=db_net_ipv --name=ipv2 -it --rm alpine /bin/sh 75 $ ping -c 4 ipv1 76 77 # NOTE: the containers can NOT ping the underlying host interfaces as 78 # they are intentionally filtered by Linux for additional isolation. 79 ``` 80 81 The drivers also support the `--internal` flag that will completely isolate containers on a network from any communications external to that network. Since network isolation is tightly coupled to the network's parent interface the result of leaving the `-o parent=` option off of a `docker network create` is the exact same as the `--internal` option. If the parent interface is not specified or the `--internal` flag is used, a netlink type `dummy` parent interface is created for the user and used as the parent interface effectively isolating the network completely. 82 83 The following two `docker network create` examples result in identical networks that you can attach container to: 84 85 ``` 86 # Empty '-o parent=' creates an isolated network 87 $ docker network create -d ipvlan \ 88 --subnet=192.168.10.0/24 isolated1 89 90 # Explicit '--internal' flag is the same: 91 $ docker network create -d ipvlan \ 92 --subnet=192.168.11.0/24 --internal isolated2 93 94 # Even the '--subnet=' can be left empty and the default 95 # IPAM subnet of 172.18.0.0/16 will be assigned 96 $ docker network create -d ipvlan isolated3 97 98 $ docker run --net=isolated1 --name=cid1 -it --rm alpine /bin/sh 99 $ docker run --net=isolated2 --name=cid2 -it --rm alpine /bin/sh 100 $ docker run --net=isolated3 --name=cid3 -it --rm alpine /bin/sh 101 102 # To attach to any use `docker exec` and start a shell 103 $ docker exec -it cid1 /bin/sh 104 $ docker exec -it cid2 /bin/sh 105 $ docker exec -it cid3 /bin/sh 106 ``` 107 108 ### Ipvlan 802.1q Trunk L2 Mode Example Usage 109 110 Architecturally, Ipvlan L2 mode trunking is the same as Macvlan with regard to gateways and L2 path isolation. There are nuances that can be advantageous for CAM table pressure in ToR switches, one MAC per port and MAC exhaustion on a host's parent NIC to name a few. The 802.1q trunk scenario looks the same. Both modes adhere to tagging standards and have seamless integration with the physical network for underlay integration and hardware vendor plugin integrations. 111 112 Hosts on the same VLAN are typically on the same subnet and almost always are grouped together based on their security policy. In most scenarios, a multi-tier application is tiered into different subnets because the security profile of each process requires some form of isolation. For example, hosting your credit card processing on the same virtual network as the frontend webserver would be a regulatory compliance issue, along with circumventing the long standing best practice of layered defense in depth architectures. VLANs or the equivocal VNI (Virtual Network Identifier) when using the Overlay driver, are the first step in isolating tenant traffic. 113 114 ![Docker VLANs in Depth](images/vlans-deeper-look.png) 115 116 The Linux sub-interface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the sub-interface. Parent interfaces such as `eth0` are not deleted, only sub-interfaces with a netlink parent index > 0. 117 118 For the driver to add/delete the vlan sub-interfaces the format needs to be `interface_name.vlan_tag`. Other sub-interface naming can be used as the specified parent, but the link will not be deleted automatically when `docker network rm` is invoked. 119 120 The option to use either existing parent vlan sub-interfaces or let Docker manage them enables the user to either completely manage the Linux interfaces and networking or let Docker create and delete the Vlan parent sub-interfaces (netlink `ip link`) with no effort from the user. 121 122 For example: use `eth0.10` to denote a sub-interface of `eth0` tagged with the vlan id of `10`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.10 type vlan id 10`. 123 124 The example creates the vlan tagged networks and then start two containers to test connectivity between containers. Different Vlans cannot ping one another without a router routing between the two networks. The default namespace is not reachable per ipvlan design in order to isolate container namespaces from the underlying host. 125 126 **Vlan ID 20** 127 128 In the first network tagged and isolated by the Docker host, `eth0.20` is the parent interface tagged with vlan id `20` specified with `-o parent=eth0.20`. Other naming formats can be used, but the links need to be added and deleted manually using `ip link` or Linux configuration files. As long as the `-o parent` exists anything can be used if compliant with Linux netlink. 129 130 ``` 131 # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged 132 $ docker network create -d ipvlan \ 133 --subnet=192.168.20.0/24 \ 134 --gateway=192.168.20.1 \ 135 -o parent=eth0.20 ipvlan20 136 137 # in two separate terminals, start a Docker container and the containers can now ping one another. 138 $ docker run --net=ipvlan20 -it --name ivlan_test1 --rm alpine /bin/sh 139 $ docker run --net=ipvlan20 -it --name ivlan_test2 --rm alpine /bin/sh 140 ``` 141 142 **Vlan ID 30** 143 144 In the second network, tagged and isolated by the Docker host, `eth0.30` is the parent interface tagged with vlan id `30` specified with `-o parent=eth0.30`. The `ipvlan_mode=` defaults to l2 mode `ipvlan_mode=l2`. It can also be explicitly set with the same result as shown in the next example. 145 146 ``` 147 # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged. 148 $ docker network create -d ipvlan \ 149 --subnet=192.168.30.0/24 \ 150 --gateway=192.168.30.1 \ 151 -o parent=eth0.30 \ 152 -o ipvlan_mode=l2 ipvlan30 153 154 # in two separate terminals, start a Docker container and the containers can now ping one another. 155 $ docker run --net=ipvlan30 -it --name ivlan_test3 --rm alpine /bin/sh 156 $ docker run --net=ipvlan30 -it --name ivlan_test4 --rm alpine /bin/sh 157 ``` 158 159 The gateway is set inside of the container as the default gateway. That gateway would typically be an external router on the network. 160 161 ``` 162 $$ ip route 163 default via 192.168.30.1 dev eth0 164 192.168.30.0/24 dev eth0 src 192.168.30.2 165 ``` 166 167 Example: Multi-Subnet Ipvlan L2 Mode starting two containers on the same subnet and pinging one another. In order for the `192.168.114.0/24` to reach `192.168.116.0/24` it requires an external router in L2 mode. L3 mode can route between subnets that share a common `-o parent=`. 168 169 Secondary addresses on network routers are common as an address space becomes exhausted to add another secondary to an L3 vlan interface or commonly referred to as a "switched virtual interface" (SVI). 170 171 ``` 172 $ docker network create -d ipvlan \ 173 --subnet=192.168.114.0/24 --subnet=192.168.116.0/24 \ 174 --gateway=192.168.114.254 --gateway=192.168.116.254 \ 175 -o parent=eth0.114 \ 176 -o ipvlan_mode=l2 ipvlan114 177 178 $ docker run --net=ipvlan114 --ip=192.168.114.10 -it --rm alpine /bin/sh 179 $ docker run --net=ipvlan114 --ip=192.168.114.11 -it --rm alpine /bin/sh 180 ``` 181 182 A key takeaway is, operators have the ability to map their physical network into their virtual network for integrating containers into their environment with no operational overhauls required. NetOps simply drops an 802.1q trunk into the Docker host. That virtual link would be the `-o parent=` passed in the network creation. For untagged (non-VLAN) links, it is as simple as `-o parent=eth0` or for 802.1q trunks with VLAN IDs each network gets mapped to the corresponding VLAN/Subnet from the network. 183 184 An example being, NetOps provides VLAN ID and the associated subnets for VLANs being passed on the Ethernet link to the Docker host server. Those values are simply plugged into the `docker network create` commands when provisioning the Docker networks. These are persistent configurations that are applied every time the Docker engine starts which alleviates having to manage often complex configuration files. The network interfaces can also be managed manually by being pre-created and docker networking will never modify them, simply use them as parent interfaces. Example mappings from NetOps to Docker network commands are as follows: 185 186 - VLAN: 10, Subnet: 172.16.80.0/24, Gateway: 172.16.80.1 187 188 - `--subnet=172.16.80.0/24 --gateway=172.16.80.1 -o parent=eth0.10` 189 190 - VLAN: 20, IP subnet: 172.16.50.0/22, Gateway: 172.16.50.1 191 192 - `--subnet=172.16.50.0/22 --gateway=172.16.50.1 -o parent=eth0.20 ` 193 194 - VLAN: 30, Subnet: 10.1.100.0/16, Gateway: 10.1.100.1 195 196 - `--subnet=10.1.100.0/16 --gateway=10.1.100.1 -o parent=eth0.30` 197 198 ### IPVlan L3 Mode Example 199 200 IPVlan will require routes to be distributed to each endpoint. The driver only builds the Ipvlan L3 mode port and attaches the container to the interface. Route distribution throughout a cluster is beyond the initial implementation of this single host scoped driver. In L3 mode, the Docker host is very similar to a router starting new networks in the container. They are on networks that the upstream network will not know about without route distribution. For those curious how Ipvlan L3 will fit into container networking see the following examples. 201 202 ![Docker Ipvlan L2 Mode](images/ipvlan-l3.png) 203 204 Ipvlan L3 mode drops all broadcast and multicast traffic. This reason alone makes Ipvlan L3 mode a prime candidate for those looking for massive scale and predictable network integrations. It is predictable and in turn will lead to greater uptimes because there is no bridging involved. Bridging loops have been responsible for high profile outages that can be hard to pinpoint depending on the size of the failure domain. This is due to the cascading nature of BPDUs (Bridge Port Data Units) that are flooded throughout a broadcast domain (VLAN) to find and block topology loops. Eliminating bridging domains, or at the least, keeping them isolated to a pair of ToRs (top of rack switches) will reduce hard to troubleshoot bridging instabilities. Ipvlan L2 modes is well suited for isolated VLANs only trunked into a pair of ToRs that can provide a loop-free non-blocking fabric. The next step further is to route at the edge via Ipvlan L3 mode that reduces a failure domain to a local host only. 205 206 - L3 mode needs to be on a separate subnet as the default namespace since it requires a netlink route in the default namespace pointing to the Ipvlan parent interface. 207 208 - The parent interface used in this example is `eth0` and it is on the subnet `192.168.1.0/24`. Notice the `docker network` is **not** on the same subnet as `eth0`. 209 210 - Unlike ipvlan l2 modes, different subnets/networks can ping one another as long as they share the same parent interface `-o parent=`. 211 212 ``` 213 $$ ip a show eth0 214 3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 215 link/ether 00:50:56:39:45:2e brd ff:ff:ff:ff:ff:ff 216 inet 192.168.1.250/24 brd 192.168.1.255 scope global eth0 217 ``` 218 219 - A traditional gateway doesn't mean much to an L3 mode Ipvlan interface since there is no broadcast traffic allowed. Because of that, the container default gateway simply points to the containers `eth0` device. See below for CLI output of `ip route` or `ip -6 route` from inside an L3 container for details. 220 221 The mode ` -o ipvlan_mode=l3` must be explicitly specified since the default ipvlan mode is `l2`. 222 223 The following example does not specify a parent interface. The network drivers will create a dummy type link for the user rather than rejecting the network creation and isolating containers from only communicating with one another. 224 225 ``` 226 # Create the Ipvlan L3 network 227 $ docker network create -d ipvlan \ 228 --subnet=192.168.214.0/24 \ 229 --subnet=10.1.214.0/24 \ 230 -o ipvlan_mode=l3 ipnet210 231 232 # Test 192.168.214.0/24 connectivity 233 $ docker run --net=ipnet210 --ip=192.168.214.10 -itd alpine /bin/sh 234 $ docker run --net=ipnet210 --ip=10.1.214.10 -itd alpine /bin/sh 235 236 # Test L3 connectivity from 10.1.214.0/24 to 192.168.212.0/24 237 $ docker run --net=ipnet210 --ip=192.168.214.9 -it --rm alpine ping -c 2 10.1.214.10 238 239 # Test L3 connectivity from 192.168.212.0/24 to 10.1.214.0/24 240 $ docker run --net=ipnet210 --ip=10.1.214.9 -it --rm alpine ping -c 2 192.168.214.10 241 242 ``` 243 244 Notice there is no `--gateway=` option in the network create. The field is ignored if one is specified `l3` mode. Take a look at the container routing table from inside of the container: 245 246 ``` 247 # Inside an L3 mode container 248 $$ ip route 249 default dev eth0 250 192.168.214.0/24 dev eth0 src 192.168.214.10 251 ``` 252 253 In order to ping the containers from a remote Docker host or the container be able to ping a remote host, the remote host or the physical network in between need to have a route pointing to the host IP address of the container's Docker host eth interface. More on this as we evolve the Ipvlan `L3` story. 254 255 ### Dual Stack IPv4 IPv6 Ipvlan L2 Mode 256 257 - Not only does Libnetwork give you complete control over IPv4 addressing, but it also gives you total control over IPv6 addressing as well as feature parity between the two address families. 258 259 - The next example will start with IPv6 only. Start two containers on the same VLAN `139` and ping one another. Since the IPv4 subnet is not specified, the default IPAM will provision a default IPv4 subnet. That subnet is isolated unless the upstream network is explicitly routing it on VLAN `139`. 260 261 ``` 262 # Create a v6 network 263 $ docker network create -d ipvlan \ 264 --subnet=2001:db8:abc2::/64 --gateway=2001:db8:abc2::22 \ 265 -o parent=eth0.139 v6ipvlan139 266 267 # Start a container on the network 268 $ docker run --net=v6ipvlan139 -it --rm alpine /bin/sh 269 270 ``` 271 272 View the container eth0 interface and v6 routing table: 273 274 ``` 275 # Inside the IPv6 container 276 $$ ip a show eth0 277 75: eth0@if55: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 278 link/ether 00:50:56:2b:29:40 brd ff:ff:ff:ff:ff:ff 279 inet 172.18.0.2/16 scope global eth0 280 valid_lft forever preferred_lft forever 281 inet6 2001:db8:abc4::250:56ff:fe2b:2940/64 scope link 282 valid_lft forever preferred_lft forever 283 inet6 2001:db8:abc2::1/64 scope link nodad 284 valid_lft forever preferred_lft forever 285 286 $$ ip -6 route 287 2001:db8:abc4::/64 dev eth0 proto kernel metric 256 288 2001:db8:abc2::/64 dev eth0 proto kernel metric 256 289 default via 2001:db8:abc2::22 dev eth0 metric 1024 290 ``` 291 292 Start a second container and ping the first container's v6 address. 293 294 ``` 295 # Test L2 connectivity over IPv6 296 $ docker run --net=v6ipvlan139 -it --rm alpine /bin/sh 297 298 # Inside the second IPv6 container 299 $$ ip a show eth0 300 75: eth0@if55: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 301 link/ether 00:50:56:2b:29:40 brd ff:ff:ff:ff:ff:ff 302 inet 172.18.0.3/16 scope global eth0 303 valid_lft forever preferred_lft forever 304 inet6 2001:db8:abc4::250:56ff:fe2b:2940/64 scope link tentative dadfailed 305 valid_lft forever preferred_lft forever 306 inet6 2001:db8:abc2::2/64 scope link nodad 307 valid_lft forever preferred_lft forever 308 309 $$ ping6 2001:db8:abc2::1 310 PING 2001:db8:abc2::1 (2001:db8:abc2::1): 56 data bytes 311 64 bytes from 2001:db8:abc2::1%eth0: icmp_seq=0 ttl=64 time=0.044 ms 312 64 bytes from 2001:db8:abc2::1%eth0: icmp_seq=1 ttl=64 time=0.058 ms 313 314 2 packets transmitted, 2 packets received, 0% packet loss 315 round-trip min/avg/max/stddev = 0.044/0.051/0.058/0.000 ms 316 ``` 317 318 The next example with setup a dual stack IPv4/IPv6 network with an example VLAN ID of `140`. 319 320 Next create a network with two IPv4 subnets and one IPv6 subnets, all of which have explicit gateways: 321 322 ``` 323 $ docker network create -d ipvlan \ 324 --subnet=192.168.140.0/24 --subnet=192.168.142.0/24 \ 325 --gateway=192.168.140.1 --gateway=192.168.142.1 \ 326 --subnet=2001:db8:abc9::/64 --gateway=2001:db8:abc9::22 \ 327 -o parent=eth0.140 \ 328 -o ipvlan_mode=l2 ipvlan140 329 ``` 330 331 Start a container and view eth0 and both v4 & v6 routing tables: 332 333 ``` 334 $ docker run --net=ipvlan140 --ip6=2001:db8:abc2::51 -it --rm alpine /bin/sh 335 336 $ ip a show eth0 337 78: eth0@if77: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 338 link/ether 00:50:56:2b:29:40 brd ff:ff:ff:ff:ff:ff 339 inet 192.168.140.2/24 scope global eth0 340 valid_lft forever preferred_lft forever 341 inet6 2001:db8:abc4::250:56ff:fe2b:2940/64 scope link 342 valid_lft forever preferred_lft forever 343 inet6 2001:db8:abc9::1/64 scope link nodad 344 valid_lft forever preferred_lft forever 345 346 $$ ip route 347 default via 192.168.140.1 dev eth0 348 192.168.140.0/24 dev eth0 proto kernel scope link src 192.168.140.2 349 350 $$ ip -6 route 351 2001:db8:abc4::/64 dev eth0 proto kernel metric 256 352 2001:db8:abc9::/64 dev eth0 proto kernel metric 256 353 default via 2001:db8:abc9::22 dev eth0 metric 1024 354 ``` 355 356 Start a second container with a specific `--ip4` address and ping the first host using IPv4 packets: 357 358 ``` 359 $ docker run --net=ipvlan140 --ip=192.168.140.10 -it --rm alpine /bin/sh 360 ``` 361 362 **Note**: Different subnets on the same parent interface in Ipvlan `L2` mode cannot ping one another. That requires a router to proxy-arp the requests with a secondary subnet. However, Ipvlan `L3` will route the unicast traffic between disparate subnets as long as they share the same `-o parent` parent link. 363 364 ### Dual Stack IPv4 IPv6 Ipvlan L3 Mode 365 366 **Example:** IpVlan L3 Mode Dual Stack IPv4/IPv6, Multi-Subnet w/ 802.1q Vlan Tag:118 367 368 As in all of the examples, a tagged VLAN interface does not have to be used. The sub-interfaces can be swapped with `eth0`, `eth1`, `bond0` or any other valid interface on the host other then the `lo` loopback. 369 370 The primary difference you will see is that L3 mode does not create a default route with a next-hop but rather sets a default route pointing to `dev eth` only since ARP/Broadcasts/Multicast are all filtered by Linux as per the design. Since the parent interface is essentially acting as a router, the parent interface IP and subnet needs to be different from the container networks. That is the opposite of bridge and L2 modes, which need to be on the same subnet (broadcast domain) in order to forward broadcast and multicast packets. 371 372 ``` 373 # Create an IPv6+IPv4 Dual Stack Ipvlan L3 network 374 # Gateways for both v4 and v6 are set to a dev e.g. 'default dev eth0' 375 $ docker network create -d ipvlan \ 376 --subnet=192.168.110.0/24 \ 377 --subnet=192.168.112.0/24 \ 378 --subnet=2001:db8:abc6::/64 \ 379 -o parent=eth0 \ 380 -o ipvlan_mode=l3 ipnet110 381 382 383 # Start a few of containers on the network (ipnet110) 384 # in separate terminals and check connectivity 385 $ docker run --net=ipnet110 -it --rm alpine /bin/sh 386 # Start a second container specifying the v6 address 387 $ docker run --net=ipnet110 --ip6=2001:db8:abc6::10 -it --rm alpine /bin/sh 388 # Start a third specifying the IPv4 address 389 $ docker run --net=ipnet110 --ip=192.168.112.30 -it --rm alpine /bin/sh 390 # Start a 4th specifying both the IPv4 and IPv6 addresses 391 $ docker run --net=ipnet110 --ip6=2001:db8:abc6::50 --ip=192.168.112.50 -it --rm alpine /bin/sh 392 ``` 393 394 Interface and routing table outputs are as follows: 395 396 ``` 397 $$ ip a show eth0 398 63: eth0@if59: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 399 link/ether 00:50:56:2b:29:40 brd ff:ff:ff:ff:ff:ff 400 inet 192.168.112.2/24 scope global eth0 401 valid_lft forever preferred_lft forever 402 inet6 2001:db8:abc4::250:56ff:fe2b:2940/64 scope link 403 valid_lft forever preferred_lft forever 404 inet6 2001:db8:abc6::10/64 scope link nodad 405 valid_lft forever preferred_lft forever 406 407 # Note the default route is simply the eth device because ARPs are filtered. 408 $$ ip route 409 default dev eth0 scope link 410 192.168.112.0/24 dev eth0 proto kernel scope link src 192.168.112.2 411 412 $$ ip -6 route 413 2001:db8:abc4::/64 dev eth0 proto kernel metric 256 414 2001:db8:abc6::/64 dev eth0 proto kernel metric 256 415 default dev eth0 metric 1024 416 ``` 417 418 *Note:* There may be a bug when specifying `--ip6=` addresses when you delete a container with a specified v6 address and then start a new container with the same v6 address it throws the following like the address isn't properly being released to the v6 pool. It will fail to unmount the container and be left dead. 419 420 ``` 421 docker: Error response from daemon: Address already in use. 422 ``` 423 424 ### Manually Creating 802.1q Links 425 426 **Vlan ID 40** 427 428 If a user does not want the driver to create the vlan sub-interface it simply needs to exist prior to the `docker network create`. If you have sub-interface naming that is not `interface.vlan_id` it is honored in the `-o parent=` option again as long as the interface exists and is up. 429 430 Links, when manually created, can be named anything as long as they exist when the network is created. Manually created links do not get deleted regardless of the name when the network is deleted with `docker network rm`. 431 432 ``` 433 # create a new sub-interface tied to dot1q vlan 40 434 $ ip link add link eth0 name eth0.40 type vlan id 40 435 436 # enable the new sub-interface 437 $ ip link set eth0.40 up 438 439 # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged 440 $ docker network create -d ipvlan \ 441 --subnet=192.168.40.0/24 \ 442 --gateway=192.168.40.1 \ 443 -o parent=eth0.40 ipvlan40 444 445 # in two separate terminals, start a Docker container and the containers can now ping one another. 446 $ docker run --net=ipvlan40 -it --name ivlan_test5 --rm alpine /bin/sh 447 $ docker run --net=ipvlan40 -it --name ivlan_test6 --rm alpine /bin/sh 448 ``` 449 450 **Example:** Vlan sub-interface manually created with any name: 451 452 ``` 453 # create a new sub interface tied to dot1q vlan 40 454 $ ip link add link eth0 name foo type vlan id 40 455 456 # enable the new sub-interface 457 $ ip link set foo up 458 459 # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged 460 $ docker network create -d ipvlan \ 461 --subnet=192.168.40.0/24 --gateway=192.168.40.1 \ 462 -o parent=foo ipvlan40 463 464 # in two separate terminals, start a Docker container and the containers can now ping one another. 465 $ docker run --net=ipvlan40 -it --name ivlan_test5 --rm alpine /bin/sh 466 $ docker run --net=ipvlan40 -it --name ivlan_test6 --rm alpine /bin/sh 467 ``` 468 469 Manually created links can be cleaned up with: 470 471 ``` 472 $ ip link del foo 473 ``` 474 475 As with all of the Libnetwork drivers, they can be mixed and matched, even as far as running 3rd party ecosystem drivers in parallel for maximum flexibility to the Docker user.