github.com/maier/nomad@v0.4.1-0.20161110003312-a9e3d0b8549d/website/source/docs/drivers/docker.html.md (about)

     1  ---
     2  layout: "docs"
     3  page_title: "Drivers: Docker"
     4  sidebar_current: "docs-drivers-docker"
     5  description: |-
     6    The Docker task driver is used to run Docker based tasks.
     7  ---
     8  
     9  # Docker Driver
    10  
    11  Name: `docker`
    12  
    13  The `docker` driver provides a first-class Docker workflow on Nomad. The Docker
    14  driver handles downloading containers, mapping ports, and starting, watching,
    15  and cleaning up after containers.
    16  
    17  ## Task Configuration
    18  
    19  ```hcl
    20  task "webservice" {
    21    driver = "docker"
    22  
    23    config {
    24      image = "redis:3.2"
    25      labels {
    26        group = "webservice-cache"
    27      }
    28    }
    29  }
    30  ```
    31  
    32  The `docker` driver supports the following configuration in the job spec:
    33  
    34  * `image` - The Docker image to run. The image may include a tag or custom URL
    35    and should include `https://` if required. By default it will be fetched from
    36    Docker Hub.
    37  
    38      ```hcl
    39      config {
    40        image = "https://hub.docker.internal/redis:3.2"
    41      }
    42      ```
    43  
    44  * `load` - (Optional) A list of paths to image archive files. If
    45    this key is not specified, Nomad assumes the `image` is hosted on a repository
    46    and attempts to pull the image. The `artifact` blocks can be specified to
    47    download each of the archive files. The equivalent of `docker load -i path`
    48    would be run on each of the archive files.
    49  
    50      ```hcl
    51      artifact {
    52        source = "http://path.to/redis.tar"
    53      }
    54      config {
    55        load = ["redis.tar"]
    56        image = "redis"
    57      }
    58      ```
    59  
    60  * `command` - (Optional) The command to run when starting the container.
    61  
    62      ```hcl
    63      config {
    64        command = "my-command"
    65      }
    66      ```
    67  
    68  * `args` - (Optional) A list of arguments to the optional `command`. If no
    69    `command` is specified, the args are passed directly to the container.
    70    References to environment variables or any [interpretable Nomad
    71    variables](/docs/runtime/interpolation.html) will be interpreted before
    72    launching the task. For example:
    73  
    74      ```hcl
    75      config {
    76        args = [
    77          "-bind", "${NOMAD_PORT_http}",
    78          "${nomad.datacenter}",
    79          "${MY_ENV}",
    80          "${meta.foo}",
    81        ]
    82      }
    83      ```
    84  
    85  * `labels` - (Optional) A key-value map of labels to set to the containers on
    86    start.
    87  
    88      ```hcl
    89      config {
    90        labels {
    91          foo = "bar"
    92          zip = "zap"
    93        }
    94      }
    95      ```
    96  
    97  * `privileged` - (Optional) `true` or `false` (default). Privileged mode gives
    98    the container access to devices on the host. Note that this also requires the
    99    nomad agent and docker daemon to be configured to allow privileged
   100    containers.
   101  
   102  * `ipc_mode` - (Optional) The IPC mode to be used for the container. The default
   103    is `none` for a private IPC namespace. Other values are `host` for sharing
   104    the host IPC namespace or the name or id of an existing container. Note that
   105    it is not possible to refer to Docker containers started by Nomad since their
   106    names are not known in advance. Note that setting this option also requires the
   107    Nomad agent to be configured to allow privileged containers.
   108  
   109  * `pid_mode` - (Optional) `host` or not set (default). Set to `host` to share
   110    the PID namespace with the host. Note that this also requires the Nomad agent
   111    to be configured to allow privileged containers.
   112  
   113  * `uts_mode` - (Optional) `host` or not set (default). Set to `host` to share
   114    the UTS namespace with the host. Note that this also requires the Nomad agent
   115    to be configured to allow privileged containers.
   116  
   117  * `userns_mode` - (Optional) `host` or not set (default). Set to `host` to use
   118    the host's user namespace when user namespace remapping is enabled on the
   119    docker daemon.
   120  
   121  * `network_mode` - (Optional) The network mode to be used for the container. In
   122    order to support userspace networking plugins in Docker 1.9 this accepts any
   123    value. The default is `bridge` for all operating systems but Windows, which
   124    defaults to `nat`. Other networking modes may not work without additional
   125    configuration on the host (which is outside the scope of Nomad).  Valid values
   126    pre-docker 1.9 are `default`, `bridge`, `host`, `none`, or `container:name`.
   127    See below for more details.
   128  
   129  * `hostname` - (Optional) The hostname to assign to the container. When
   130    launching more than one of a task (using `count`) with this option set, every
   131    container the task starts will have the same hostname.
   132  
   133  * `dns_servers` - (Optional) A list of DNS servers for the container to use
   134    (e.g. ["8.8.8.8", "8.8.4.4"]). *Docker API v1.10 and above only*
   135  
   136  * `dns_search_domains` - (Optional) A list of DNS search domains for the container
   137    to use.
   138  
   139  * `SSL` - (Optional) If this is set to true, Nomad uses SSL to talk to the
   140    repository. The default value is `true`.
   141  
   142  * `port_map` - (Optional) A key-value map of port labels (see below).
   143  
   144  * `auth` - (Optional) Provide authentication for a private registry (see below).
   145  
   146  * `tty` - (Optional) `true` or `false` (default). Allocate a pseudo-TTY for the
   147    container.
   148  
   149  * `interactive` - (Optional) `true` or `false` (default). Keep STDIN open on
   150    the container.
   151  
   152  * `shm_size` - (Optional) The size (bytes) of /dev/shm for the container.
   153  
   154  * `logging` - (Optional) A key-value map of Docker logging options. The default
   155    value is `syslog`.
   156  
   157      ```hcl
   158      config {
   159        logging {
   160          type = "fluentd"
   161          config {
   162            fluentd-address = "localhost:24224"
   163          }
   164        }
   165      }
   166      ```
   167  
   168  * `volumes` - (Optional) A list of `host_path:container_path` strings to bind
   169    host paths to container paths. Mounting host paths outside of the alloc
   170    directory tasks normally have access to can be disabled on clients by setting
   171    the `docker.volumes.enabled` option set to false.
   172  
   173      ```hcl
   174      config {
   175        volumes = [
   176          # Use absolute paths to mount arbitrary paths on the host
   177          "/path/on/host:/path/in/container",
   178  
   179          # Use relative paths to rebind paths already in the allocation dir
   180          "relative/to/alloc:/also/in/container"
   181        ]
   182      }
   183      ```
   184  
   185  * `work_dir` - (Optional) The working directory inside the container.
   186  
   187  ### Container Name
   188  
   189  Nomad creates a container after pulling an image. Containers are named
   190  `{taskName}-{allocId}`. This is necessary in order to place more than one
   191  container from the same task on a host (e.g. with count > 1). This also means
   192  that each container's name is unique across the cluster.
   193  
   194  This is not configurable.
   195  
   196  ### Authentication
   197  
   198  If you want to pull from a private repo (for example on dockerhub or quay.io),
   199  you will need to specify credentials in your job via the `auth` option.
   200  
   201  The `auth` object supports the following keys:
   202  
   203  * `username` - (Optional) The account username.
   204  
   205  * `password` - (Optional) The account password.
   206  
   207  * `email` - (Optional) The account email.
   208  
   209  * `server_address` - (Optional) The server domain/IP without the protocol.
   210    Docker Hub is used by default.
   211  
   212  Example:
   213  
   214  ```hcl
   215  task "example" {
   216    driver = "docker"
   217  
   218    config {
   219      image = "secret/service"
   220  
   221      auth {
   222        username = "dockerhub_user"
   223        password = "dockerhub_password"
   224      }
   225    }
   226  }
   227  ```
   228  
   229  !> **Be Careful!** At this time these credentials are stored in Nomad in plain
   230  text. Secrets management will be added in a later release.
   231  
   232  ## Networking
   233  
   234  Docker supports a variety of networking configurations, including using host
   235  interfaces, SDNs, etc. Nomad uses `bridged` networking by default, like Docker.
   236  
   237  You can specify other networking options, including custom networking plugins
   238  in Docker 1.9. **You may need to perform additional configuration on the host
   239  in order to make these work.** This additional configuration is outside the
   240  scope of Nomad.
   241  
   242  ### Allocating Ports
   243  
   244  You can allocate ports to your task using the port syntax described on the
   245  [networking page](/docs/job-specification/network.html). Here is a recap:
   246  
   247  ```hcl
   248  task "example" {
   249    driver = "docker"
   250  
   251    resources {
   252      network {
   253        port "http" {}
   254        port "https" {}
   255      }
   256    }
   257  }
   258  ```
   259  
   260  ### Forwarding and Exposing Ports
   261  
   262  A Docker container typically specifies which port a service will listen on by
   263  specifying the `EXPOSE` directive in the `Dockerfile`.
   264  
   265  Because dynamic ports will not match the ports exposed in your Dockerfile,
   266  Nomad will automatically expose all of the ports it allocates to your
   267  container.
   268  
   269  These ports will be identified via environment variables. For example:
   270  
   271  ```
   272  port "http" {}
   273  ```
   274  
   275  If Nomad allocates port `23332` to your task for `http`, `23332` will be
   276  automatically exposed and forwarded to your container, and the driver will set
   277  an environment variable `NOMAD_PORT_http` with the value `23332` that you can
   278  read inside your container.
   279  
   280  This provides an easy way to use the `host` networking option for better
   281  performance.
   282  
   283  ### Using the Port Map
   284  
   285  If you prefer to use the traditional port-mapping method, you can specify the
   286  `port_map` option in your job specification. It looks like this:
   287  
   288  ```hcl
   289  task "example" {
   290    driver = "docker"
   291  
   292    config {
   293      image = "redis"
   294  
   295      port_map {
   296        redis = 6379
   297      }
   298    }
   299  
   300    resources {
   301      network {
   302        mbits = 20
   303        port "redis" {}
   304      }
   305    }
   306  }
   307  ```
   308  
   309  If Nomad allocates port `23332` to your task, the Docker driver will
   310  automatically setup the port mapping from `23332` on the host to `6379` in your
   311  container, so it will just work!
   312  
   313  Note that by default this only works with `bridged` networking mode. It may
   314  also work with custom networking plugins which implement the same API for
   315  expose and port forwarding.
   316  
   317  ### Networking Protocols
   318  
   319  The Docker driver configures ports on both the `tcp` and `udp` protocols.
   320  
   321  This is not configurable.
   322  
   323  ### Other Networking Modes
   324  
   325  Some networking modes like `container` or `none` will require coordination
   326  outside of Nomad. First-class support for these options may be improved later
   327  through Nomad plugins or dynamic job configuration.
   328  
   329  ## Client Requirements
   330  
   331  Nomad requires Docker to be installed and running on the host alongside the
   332  Nomad agent. Nomad was developed against Docker `1.8.2` and `1.9`.
   333  
   334  By default Nomad communicates with the Docker daemon using the daemon's unix
   335  socket. Nomad will need to be able to read/write to this socket. If you do not
   336  run Nomad as root, make sure you add the Nomad user to the Docker group so
   337  Nomad can communicate with the Docker daemon.
   338  
   339  For example, on Ubuntu you can use the `usermod` command to add the `vagrant`
   340  user to the `docker` group so you can run Nomad without root:
   341  
   342      sudo usermod -G docker -a vagrant
   343  
   344  For the best performance and security features you should use recent versions
   345  of the Linux Kernel and Docker daemon.
   346  
   347  ## Client Configuration
   348  
   349  The `docker` driver has the following [client configuration
   350  options](/docs/agent/configuration/client.html#options):
   351  
   352  * `docker.endpoint` - Defaults to `unix:///var/run/docker.sock`. You will need
   353    to customize this if you use a non-standard socket (HTTP or another
   354    location).
   355  
   356  * `docker.auth.config` - Allows an operator to specify a JSON file which is in
   357    the dockercfg format containing authentication information for a private registry.
   358  
   359  * `docker.tls.cert` - Path to the server's certificate file (`.pem`). Specify
   360    this along with `docker.tls.key` and `docker.tls.ca` to use a TLS client to
   361    connect to the docker daemon. `docker.endpoint` must also be specified or
   362    this setting will be ignored.
   363  
   364  * `docker.tls.key` - Path to the client's private key (`.pem`). Specify this
   365    along with `docker.tls.cert` and `docker.tls.ca` to use a TLS client to
   366    connect to the docker daemon. `docker.endpoint` must also be specified or
   367    this setting will be ignored.
   368  
   369  * `docker.tls.ca` - Path to the server's CA file (`.pem`). Specify this along
   370    with `docker.tls.cert` and `docker.tls.key` to use a TLS client to connect to
   371    the docker daemon. `docker.endpoint` must also be specified or this setting
   372    will be ignored.
   373  
   374  * `docker.cleanup.image` Defaults to `true`. Changing this to `false` will
   375    prevent Nomad from removing images from stopped tasks.
   376  
   377  * `docker.volumes.enabled`: Defaults to `true`. Allows tasks to bind host paths
   378    (`volumes`) inside their container. Binding relative paths is always allowed
   379    and will be resolved relative to the allocation's directory.
   380  
   381  * `docker.volumes.selinuxlabel`: Allows the operator to set a SELinux
   382    label to the allocation and task local bind-mounts to containers. If used
   383    with `docker.volumes.enabled` set to false, the labels will still be applied
   384    to the standard binds in the container.
   385  
   386  * `docker.privileged.enabled` Defaults to `false`. Changing this to `true` will
   387    allow containers to use `privileged` mode, which gives the containers full
   388    access to the host's devices. Note that you must set a similar setting on the
   389    Docker daemon for this to work.
   390  
   391  Note: When testing or using the `-dev` flag you can use `DOCKER_HOST`,
   392  `DOCKER_TLS_VERIFY`, and `DOCKER_CERT_PATH` to customize Nomad's behavior. If
   393  `docker.endpoint` is set Nomad will **only** read client configuration from the
   394  config file.
   395  
   396  An example is given below:
   397  
   398  ```hcl
   399  client {
   400    options = {
   401      "docker.cleanup.image" = "false"
   402    }
   403  }
   404  ```
   405  
   406  ## Client Attributes
   407  
   408  The `docker` driver will set the following client attributes:
   409  
   410  * `driver.docker` - This will be set to "1", indicating the driver is
   411    available.
   412  * `driver.docker.version` - This will be set to version of the docker server.
   413  
   414  Here is an example of using these properties in a job file:
   415  
   416  ```hcl
   417  job "docs" {
   418    # Require docker version higher than 1.2.
   419    constraint {
   420      attribute = "${driver.docker.version}"
   421      operator  = ">"
   422      version   = "1.2"
   423    }
   424  }
   425  ```
   426  
   427  ## Resource Isolation
   428  
   429  ### CPU
   430  
   431  Nomad limits containers' CPU based on CPU shares. CPU shares allow containers
   432  to burst past their CPU limits. CPU limits will only be imposed when there is
   433  contention for resources. When the host is under load your process may be
   434  throttled to stabilize QOS depending on how many shares it has. You can see how
   435  many CPU shares are available to your process by reading `NOMAD_CPU_LIMIT`.
   436  1000 shares are approximately equal to 1Ghz.
   437  
   438  Please keep the implications of CPU shares in mind when you load test workloads
   439  on Nomad.
   440  
   441  ### Memory
   442  
   443  Nomad limits containers' memory usage based on total virtual memory. This means
   444  that containers scheduled by Nomad cannot use swap. This is to ensure that a
   445  swappy process does not degrade performance for other workloads on the same
   446  host.
   447  
   448  Since memory is not an elastic resource, you will need to make sure your
   449  container does not exceed the amount of memory allocated to it, or it will be
   450  terminated or crash when it tries to malloc. A process can inspect its memory
   451  limit by reading `NOMAD_MEMORY_LIMIT`, but will need to track its own memory
   452  usage. Memory limit is expressed in megabytes so 1024 = 1Gb.
   453  HIO!!!
   454  
   455  ### IO
   456  
   457  Nomad's Docker integration does not currently provide QOS around network or
   458  filesystem IO. These will be added in a later release.
   459  
   460  ### Security
   461  
   462  Docker provides resource isolation by way of
   463  [cgroups and namespaces](https://docs.docker.com/introduction/understanding-docker/#the-underlying-technology).
   464  Containers essentially have a virtual file system all to themselves. If you
   465  need a higher degree of isolation between processes for security or other
   466  reasons, it is recommended to use full virtualization like
   467  [QEMU](/docs/drivers/qemu.html).
   468  
   469  ## Docker For Mac Caveats
   470  
   471  Docker For Mac runs docker inside a small VM and then allows access to parts of
   472  the host filesystem into that VM. At present, nomad uses a syslog server bound to
   473  a unix socket within a path that both the host and the VM can access to forward
   474  log messages back to nomad. But at present, Docker For Mac does not work for
   475  unix domain sockets (https://github.com/docker/for-mac/issues/483) in one of
   476  these shared paths.
   477  
   478  As a result, using nomad with the docker driver on OS X/macOS will work, but no
   479  logs will be available to nomad. Users must use the native docker facilities to
   480  examine the logs of any jobs running under docker.
   481  
   482  In the future, we will resolve this issue, one way or another.