github.com/maier/nomad@v0.4.1-0.20161110003312-a9e3d0b8549d/website/source/docs/drivers/docker.html.md (about) 1 --- 2 layout: "docs" 3 page_title: "Drivers: Docker" 4 sidebar_current: "docs-drivers-docker" 5 description: |- 6 The Docker task driver is used to run Docker based tasks. 7 --- 8 9 # Docker Driver 10 11 Name: `docker` 12 13 The `docker` driver provides a first-class Docker workflow on Nomad. The Docker 14 driver handles downloading containers, mapping ports, and starting, watching, 15 and cleaning up after containers. 16 17 ## Task Configuration 18 19 ```hcl 20 task "webservice" { 21 driver = "docker" 22 23 config { 24 image = "redis:3.2" 25 labels { 26 group = "webservice-cache" 27 } 28 } 29 } 30 ``` 31 32 The `docker` driver supports the following configuration in the job spec: 33 34 * `image` - The Docker image to run. The image may include a tag or custom URL 35 and should include `https://` if required. By default it will be fetched from 36 Docker Hub. 37 38 ```hcl 39 config { 40 image = "https://hub.docker.internal/redis:3.2" 41 } 42 ``` 43 44 * `load` - (Optional) A list of paths to image archive files. If 45 this key is not specified, Nomad assumes the `image` is hosted on a repository 46 and attempts to pull the image. The `artifact` blocks can be specified to 47 download each of the archive files. The equivalent of `docker load -i path` 48 would be run on each of the archive files. 49 50 ```hcl 51 artifact { 52 source = "http://path.to/redis.tar" 53 } 54 config { 55 load = ["redis.tar"] 56 image = "redis" 57 } 58 ``` 59 60 * `command` - (Optional) The command to run when starting the container. 61 62 ```hcl 63 config { 64 command = "my-command" 65 } 66 ``` 67 68 * `args` - (Optional) A list of arguments to the optional `command`. If no 69 `command` is specified, the args are passed directly to the container. 70 References to environment variables or any [interpretable Nomad 71 variables](/docs/runtime/interpolation.html) will be interpreted before 72 launching the task. For example: 73 74 ```hcl 75 config { 76 args = [ 77 "-bind", "${NOMAD_PORT_http}", 78 "${nomad.datacenter}", 79 "${MY_ENV}", 80 "${meta.foo}", 81 ] 82 } 83 ``` 84 85 * `labels` - (Optional) A key-value map of labels to set to the containers on 86 start. 87 88 ```hcl 89 config { 90 labels { 91 foo = "bar" 92 zip = "zap" 93 } 94 } 95 ``` 96 97 * `privileged` - (Optional) `true` or `false` (default). Privileged mode gives 98 the container access to devices on the host. Note that this also requires the 99 nomad agent and docker daemon to be configured to allow privileged 100 containers. 101 102 * `ipc_mode` - (Optional) The IPC mode to be used for the container. The default 103 is `none` for a private IPC namespace. Other values are `host` for sharing 104 the host IPC namespace or the name or id of an existing container. Note that 105 it is not possible to refer to Docker containers started by Nomad since their 106 names are not known in advance. Note that setting this option also requires the 107 Nomad agent to be configured to allow privileged containers. 108 109 * `pid_mode` - (Optional) `host` or not set (default). Set to `host` to share 110 the PID namespace with the host. Note that this also requires the Nomad agent 111 to be configured to allow privileged containers. 112 113 * `uts_mode` - (Optional) `host` or not set (default). Set to `host` to share 114 the UTS namespace with the host. Note that this also requires the Nomad agent 115 to be configured to allow privileged containers. 116 117 * `userns_mode` - (Optional) `host` or not set (default). Set to `host` to use 118 the host's user namespace when user namespace remapping is enabled on the 119 docker daemon. 120 121 * `network_mode` - (Optional) The network mode to be used for the container. In 122 order to support userspace networking plugins in Docker 1.9 this accepts any 123 value. The default is `bridge` for all operating systems but Windows, which 124 defaults to `nat`. Other networking modes may not work without additional 125 configuration on the host (which is outside the scope of Nomad). Valid values 126 pre-docker 1.9 are `default`, `bridge`, `host`, `none`, or `container:name`. 127 See below for more details. 128 129 * `hostname` - (Optional) The hostname to assign to the container. When 130 launching more than one of a task (using `count`) with this option set, every 131 container the task starts will have the same hostname. 132 133 * `dns_servers` - (Optional) A list of DNS servers for the container to use 134 (e.g. ["8.8.8.8", "8.8.4.4"]). *Docker API v1.10 and above only* 135 136 * `dns_search_domains` - (Optional) A list of DNS search domains for the container 137 to use. 138 139 * `SSL` - (Optional) If this is set to true, Nomad uses SSL to talk to the 140 repository. The default value is `true`. 141 142 * `port_map` - (Optional) A key-value map of port labels (see below). 143 144 * `auth` - (Optional) Provide authentication for a private registry (see below). 145 146 * `tty` - (Optional) `true` or `false` (default). Allocate a pseudo-TTY for the 147 container. 148 149 * `interactive` - (Optional) `true` or `false` (default). Keep STDIN open on 150 the container. 151 152 * `shm_size` - (Optional) The size (bytes) of /dev/shm for the container. 153 154 * `logging` - (Optional) A key-value map of Docker logging options. The default 155 value is `syslog`. 156 157 ```hcl 158 config { 159 logging { 160 type = "fluentd" 161 config { 162 fluentd-address = "localhost:24224" 163 } 164 } 165 } 166 ``` 167 168 * `volumes` - (Optional) A list of `host_path:container_path` strings to bind 169 host paths to container paths. Mounting host paths outside of the alloc 170 directory tasks normally have access to can be disabled on clients by setting 171 the `docker.volumes.enabled` option set to false. 172 173 ```hcl 174 config { 175 volumes = [ 176 # Use absolute paths to mount arbitrary paths on the host 177 "/path/on/host:/path/in/container", 178 179 # Use relative paths to rebind paths already in the allocation dir 180 "relative/to/alloc:/also/in/container" 181 ] 182 } 183 ``` 184 185 * `work_dir` - (Optional) The working directory inside the container. 186 187 ### Container Name 188 189 Nomad creates a container after pulling an image. Containers are named 190 `{taskName}-{allocId}`. This is necessary in order to place more than one 191 container from the same task on a host (e.g. with count > 1). This also means 192 that each container's name is unique across the cluster. 193 194 This is not configurable. 195 196 ### Authentication 197 198 If you want to pull from a private repo (for example on dockerhub or quay.io), 199 you will need to specify credentials in your job via the `auth` option. 200 201 The `auth` object supports the following keys: 202 203 * `username` - (Optional) The account username. 204 205 * `password` - (Optional) The account password. 206 207 * `email` - (Optional) The account email. 208 209 * `server_address` - (Optional) The server domain/IP without the protocol. 210 Docker Hub is used by default. 211 212 Example: 213 214 ```hcl 215 task "example" { 216 driver = "docker" 217 218 config { 219 image = "secret/service" 220 221 auth { 222 username = "dockerhub_user" 223 password = "dockerhub_password" 224 } 225 } 226 } 227 ``` 228 229 !> **Be Careful!** At this time these credentials are stored in Nomad in plain 230 text. Secrets management will be added in a later release. 231 232 ## Networking 233 234 Docker supports a variety of networking configurations, including using host 235 interfaces, SDNs, etc. Nomad uses `bridged` networking by default, like Docker. 236 237 You can specify other networking options, including custom networking plugins 238 in Docker 1.9. **You may need to perform additional configuration on the host 239 in order to make these work.** This additional configuration is outside the 240 scope of Nomad. 241 242 ### Allocating Ports 243 244 You can allocate ports to your task using the port syntax described on the 245 [networking page](/docs/job-specification/network.html). Here is a recap: 246 247 ```hcl 248 task "example" { 249 driver = "docker" 250 251 resources { 252 network { 253 port "http" {} 254 port "https" {} 255 } 256 } 257 } 258 ``` 259 260 ### Forwarding and Exposing Ports 261 262 A Docker container typically specifies which port a service will listen on by 263 specifying the `EXPOSE` directive in the `Dockerfile`. 264 265 Because dynamic ports will not match the ports exposed in your Dockerfile, 266 Nomad will automatically expose all of the ports it allocates to your 267 container. 268 269 These ports will be identified via environment variables. For example: 270 271 ``` 272 port "http" {} 273 ``` 274 275 If Nomad allocates port `23332` to your task for `http`, `23332` will be 276 automatically exposed and forwarded to your container, and the driver will set 277 an environment variable `NOMAD_PORT_http` with the value `23332` that you can 278 read inside your container. 279 280 This provides an easy way to use the `host` networking option for better 281 performance. 282 283 ### Using the Port Map 284 285 If you prefer to use the traditional port-mapping method, you can specify the 286 `port_map` option in your job specification. It looks like this: 287 288 ```hcl 289 task "example" { 290 driver = "docker" 291 292 config { 293 image = "redis" 294 295 port_map { 296 redis = 6379 297 } 298 } 299 300 resources { 301 network { 302 mbits = 20 303 port "redis" {} 304 } 305 } 306 } 307 ``` 308 309 If Nomad allocates port `23332` to your task, the Docker driver will 310 automatically setup the port mapping from `23332` on the host to `6379` in your 311 container, so it will just work! 312 313 Note that by default this only works with `bridged` networking mode. It may 314 also work with custom networking plugins which implement the same API for 315 expose and port forwarding. 316 317 ### Networking Protocols 318 319 The Docker driver configures ports on both the `tcp` and `udp` protocols. 320 321 This is not configurable. 322 323 ### Other Networking Modes 324 325 Some networking modes like `container` or `none` will require coordination 326 outside of Nomad. First-class support for these options may be improved later 327 through Nomad plugins or dynamic job configuration. 328 329 ## Client Requirements 330 331 Nomad requires Docker to be installed and running on the host alongside the 332 Nomad agent. Nomad was developed against Docker `1.8.2` and `1.9`. 333 334 By default Nomad communicates with the Docker daemon using the daemon's unix 335 socket. Nomad will need to be able to read/write to this socket. If you do not 336 run Nomad as root, make sure you add the Nomad user to the Docker group so 337 Nomad can communicate with the Docker daemon. 338 339 For example, on Ubuntu you can use the `usermod` command to add the `vagrant` 340 user to the `docker` group so you can run Nomad without root: 341 342 sudo usermod -G docker -a vagrant 343 344 For the best performance and security features you should use recent versions 345 of the Linux Kernel and Docker daemon. 346 347 ## Client Configuration 348 349 The `docker` driver has the following [client configuration 350 options](/docs/agent/configuration/client.html#options): 351 352 * `docker.endpoint` - Defaults to `unix:///var/run/docker.sock`. You will need 353 to customize this if you use a non-standard socket (HTTP or another 354 location). 355 356 * `docker.auth.config` - Allows an operator to specify a JSON file which is in 357 the dockercfg format containing authentication information for a private registry. 358 359 * `docker.tls.cert` - Path to the server's certificate file (`.pem`). Specify 360 this along with `docker.tls.key` and `docker.tls.ca` to use a TLS client to 361 connect to the docker daemon. `docker.endpoint` must also be specified or 362 this setting will be ignored. 363 364 * `docker.tls.key` - Path to the client's private key (`.pem`). Specify this 365 along with `docker.tls.cert` and `docker.tls.ca` to use a TLS client to 366 connect to the docker daemon. `docker.endpoint` must also be specified or 367 this setting will be ignored. 368 369 * `docker.tls.ca` - Path to the server's CA file (`.pem`). Specify this along 370 with `docker.tls.cert` and `docker.tls.key` to use a TLS client to connect to 371 the docker daemon. `docker.endpoint` must also be specified or this setting 372 will be ignored. 373 374 * `docker.cleanup.image` Defaults to `true`. Changing this to `false` will 375 prevent Nomad from removing images from stopped tasks. 376 377 * `docker.volumes.enabled`: Defaults to `true`. Allows tasks to bind host paths 378 (`volumes`) inside their container. Binding relative paths is always allowed 379 and will be resolved relative to the allocation's directory. 380 381 * `docker.volumes.selinuxlabel`: Allows the operator to set a SELinux 382 label to the allocation and task local bind-mounts to containers. If used 383 with `docker.volumes.enabled` set to false, the labels will still be applied 384 to the standard binds in the container. 385 386 * `docker.privileged.enabled` Defaults to `false`. Changing this to `true` will 387 allow containers to use `privileged` mode, which gives the containers full 388 access to the host's devices. Note that you must set a similar setting on the 389 Docker daemon for this to work. 390 391 Note: When testing or using the `-dev` flag you can use `DOCKER_HOST`, 392 `DOCKER_TLS_VERIFY`, and `DOCKER_CERT_PATH` to customize Nomad's behavior. If 393 `docker.endpoint` is set Nomad will **only** read client configuration from the 394 config file. 395 396 An example is given below: 397 398 ```hcl 399 client { 400 options = { 401 "docker.cleanup.image" = "false" 402 } 403 } 404 ``` 405 406 ## Client Attributes 407 408 The `docker` driver will set the following client attributes: 409 410 * `driver.docker` - This will be set to "1", indicating the driver is 411 available. 412 * `driver.docker.version` - This will be set to version of the docker server. 413 414 Here is an example of using these properties in a job file: 415 416 ```hcl 417 job "docs" { 418 # Require docker version higher than 1.2. 419 constraint { 420 attribute = "${driver.docker.version}" 421 operator = ">" 422 version = "1.2" 423 } 424 } 425 ``` 426 427 ## Resource Isolation 428 429 ### CPU 430 431 Nomad limits containers' CPU based on CPU shares. CPU shares allow containers 432 to burst past their CPU limits. CPU limits will only be imposed when there is 433 contention for resources. When the host is under load your process may be 434 throttled to stabilize QOS depending on how many shares it has. You can see how 435 many CPU shares are available to your process by reading `NOMAD_CPU_LIMIT`. 436 1000 shares are approximately equal to 1Ghz. 437 438 Please keep the implications of CPU shares in mind when you load test workloads 439 on Nomad. 440 441 ### Memory 442 443 Nomad limits containers' memory usage based on total virtual memory. This means 444 that containers scheduled by Nomad cannot use swap. This is to ensure that a 445 swappy process does not degrade performance for other workloads on the same 446 host. 447 448 Since memory is not an elastic resource, you will need to make sure your 449 container does not exceed the amount of memory allocated to it, or it will be 450 terminated or crash when it tries to malloc. A process can inspect its memory 451 limit by reading `NOMAD_MEMORY_LIMIT`, but will need to track its own memory 452 usage. Memory limit is expressed in megabytes so 1024 = 1Gb. 453 HIO!!! 454 455 ### IO 456 457 Nomad's Docker integration does not currently provide QOS around network or 458 filesystem IO. These will be added in a later release. 459 460 ### Security 461 462 Docker provides resource isolation by way of 463 [cgroups and namespaces](https://docs.docker.com/introduction/understanding-docker/#the-underlying-technology). 464 Containers essentially have a virtual file system all to themselves. If you 465 need a higher degree of isolation between processes for security or other 466 reasons, it is recommended to use full virtualization like 467 [QEMU](/docs/drivers/qemu.html). 468 469 ## Docker For Mac Caveats 470 471 Docker For Mac runs docker inside a small VM and then allows access to parts of 472 the host filesystem into that VM. At present, nomad uses a syslog server bound to 473 a unix socket within a path that both the host and the VM can access to forward 474 log messages back to nomad. But at present, Docker For Mac does not work for 475 unix domain sockets (https://github.com/docker/for-mac/issues/483) in one of 476 these shared paths. 477 478 As a result, using nomad with the docker driver on OS X/macOS will work, but no 479 logs will be available to nomad. Users must use the native docker facilities to 480 examine the logs of any jobs running under docker. 481 482 In the future, we will resolve this issue, one way or another.