github.com/taylorchu/nomad@v0.5.3-rc1.0.20170407200202-db11e7dd7b55/website/source/docs/drivers/docker.html.md (about)

     1  ---
     2  layout: "docs"
     3  page_title: "Drivers: Docker"
     4  sidebar_current: "docs-drivers-docker"
     5  description: |-
     6    The Docker task driver is used to run Docker based tasks.
     7  ---
     8  
     9  # Docker Driver
    10  
    11  Name: `docker`
    12  
    13  The `docker` driver provides a first-class Docker workflow on Nomad. The Docker
    14  driver handles downloading containers, mapping ports, and starting, watching,
    15  and cleaning up after containers.
    16  
    17  ## Task Configuration
    18  
    19  ```hcl
    20  task "webservice" {
    21    driver = "docker"
    22  
    23    config {
    24      image = "redis:3.2"
    25      labels {
    26        group = "webservice-cache"
    27      }
    28    }
    29  }
    30  ```
    31  
    32  The `docker` driver supports the following configuration in the job spec:
    33  
    34  * `image` - The Docker image to run. The image may include a tag or custom URL
    35    and should include `https://` if required. By default it will be fetched from
    36    Docker Hub. If the tag is omitted or equal to `latest` the driver will always
    37    try to pull the image. If the image to be pulled exists in a registry that
    38    requires authentication credentials must be provided to Nomad. Please see the
    39    [Authentication section](#authentication).
    40  
    41      ```hcl
    42      config {
    43        image = "https://hub.docker.internal/redis:3.2"
    44      }
    45      ```
    46  
    47  * `load` - (Optional) A list of paths to image archive files. If
    48    this key is not specified, Nomad assumes the `image` is hosted on a repository
    49    and attempts to pull the image. The `artifact` blocks can be specified to
    50    download each of the archive files. The equivalent of `docker load -i path`
    51    would be run on each of the archive files.
    52  
    53      ```hcl
    54      artifact {
    55        source = "http://path.to/redis.tar"
    56      }
    57      config {
    58        load = ["redis.tar"]
    59        image = "redis"
    60      }
    61      ```
    62  
    63  * `command` - (Optional) The command to run when starting the container.
    64  
    65      ```hcl
    66      config {
    67        command = "my-command"
    68      }
    69      ```
    70  
    71  * `args` - (Optional) A list of arguments to the optional `command`. If no
    72    `command` is specified, the args are passed directly to the container.
    73    References to environment variables or any [interpretable Nomad
    74    variables](/docs/runtime/interpolation.html) will be interpreted before
    75    launching the task. For example:
    76  
    77      ```hcl
    78      config {
    79        args = [
    80          "-bind", "${NOMAD_PORT_http}",
    81          "${nomad.datacenter}",
    82          "${MY_ENV}",
    83          "${meta.foo}",
    84        ]
    85      }
    86      ```
    87  
    88  * `labels` - (Optional) A key-value map of labels to set to the containers on
    89    start.
    90  
    91      ```hcl
    92      config {
    93        labels {
    94          foo = "bar"
    95          zip = "zap"
    96        }
    97      }
    98      ```
    99  
   100  * `privileged` - (Optional) `true` or `false` (default). Privileged mode gives
   101    the container access to devices on the host. Note that this also requires the
   102    nomad agent and docker daemon to be configured to allow privileged
   103    containers.
   104  
   105  * `ipc_mode` - (Optional) The IPC mode to be used for the container. The default
   106    is `none` for a private IPC namespace. Other values are `host` for sharing
   107    the host IPC namespace or the name or id of an existing container. Note that
   108    it is not possible to refer to Docker containers started by Nomad since their
   109    names are not known in advance. Note that setting this option also requires the
   110    Nomad agent to be configured to allow privileged containers.
   111  
   112  * `pid_mode` - (Optional) `host` or not set (default). Set to `host` to share
   113    the PID namespace with the host. Note that this also requires the Nomad agent
   114    to be configured to allow privileged containers.
   115  
   116  * `uts_mode` - (Optional) `host` or not set (default). Set to `host` to share
   117    the UTS namespace with the host. Note that this also requires the Nomad agent
   118    to be configured to allow privileged containers.
   119  
   120  * `userns_mode` - (Optional) `host` or not set (default). Set to `host` to use
   121    the host's user namespace when user namespace remapping is enabled on the
   122    docker daemon.
   123  
   124  * `network_mode` - (Optional) The network mode to be used for the container. In
   125    order to support userspace networking plugins in Docker 1.9 this accepts any
   126    value. The default is `bridge` for all operating systems but Windows, which
   127    defaults to `nat`. Other networking modes may not work without additional
   128    configuration on the host (which is outside the scope of Nomad).  Valid values
   129    pre-docker 1.9 are `default`, `bridge`, `host`, `none`, or `container:name`.
   130    See below for more details.
   131  
   132  * `network_aliases` - (Optional) A list of network-scoped aliases, provide a way for a
   133    container to be discovered by an alternate name by any other container within
   134    the scope of a particular network. Network-scoped alias is supported only for
   135    containers in user defined networks
   136  
   137      ```hcl
   138      config {
   139        network_mode = "user-network"
   140        network_aliases = [
   141          "${NOMAD_TASK_NAME}",
   142          "${NOMAD_TASK_NAME}-${NOMAD_ALLOC_INDEX}"
   143        ]
   144      }
   145      ```
   146  
   147  * `ipv4_address` - (Optional) The IPv4 address to be used for the container when
   148    using user defined networks. Requires docker 1.13.0 or greater.
   149  
   150  * `ipv6_address` - (Optional) The IPv6 address to be used for the container when
   151    using user defined networks. Requires docker 1.13.0 or greater.
   152  
   153  * `hostname` - (Optional) The hostname to assign to the container. When
   154    launching more than one of a task (using `count`) with this option set, every
   155    container the task starts will have the same hostname.
   156  
   157  * `dns_servers` - (Optional) A list of DNS servers for the container to use
   158    (e.g. ["8.8.8.8", "8.8.4.4"]). *Docker API v1.10 and above only*
   159  
   160  * `dns_search_domains` - (Optional) A list of DNS search domains for the container
   161    to use.
   162  
   163  * `SSL` - (Optional) If this is set to true, Nomad uses SSL to talk to the
   164    repository. The default value is `true`. **Deprecated as of 0.5.3**
   165  
   166  * `port_map` - (Optional) A key-value map of port labels (see below).
   167  
   168  * `auth` - (Optional) Provide authentication for a private registry (see below).
   169  
   170  * `tty` - (Optional) `true` or `false` (default). Allocate a pseudo-TTY for the
   171    container.
   172  
   173  * `interactive` - (Optional) `true` or `false` (default). Keep STDIN open on
   174    the container.
   175  
   176  * `force_pull` - (Optional) `true` or `false` (default). Always pull latest image
   177    instead of using existing local image. Should be set to `true` if repository tags
   178    are mutable.
   179  
   180  * `shm_size` - (Optional) The size (bytes) of /dev/shm for the container.
   181  
   182  * `logging` - (Optional) A key-value map of Docker logging options. The default
   183    value is `syslog`.
   184  
   185      ```hcl
   186      config {
   187        logging {
   188          type = "fluentd"
   189          config {
   190            fluentd-address = "localhost:24224"
   191            tag = "your_tag"
   192          }
   193        }
   194      }
   195      ```
   196  
   197  * `volumes` - (Optional) A list of `host_path:container_path` strings to bind
   198    host paths to container paths. Mounting host paths outside of the allocation
   199    directory can be disabled on clients by setting the `docker.volumes.enabled`
   200    option set to false. This will limit volumes to directories that exist inside
   201    the allocation directory.
   202  
   203      ```hcl
   204      config {
   205        volumes = [
   206          # Use absolute paths to mount arbitrary paths on the host
   207          "/path/on/host:/path/in/container",
   208  
   209          # Use relative paths to rebind paths already in the allocation dir
   210          "relative/to/task:/also/in/container"
   211        ]
   212      }
   213      ```
   214  
   215  * `volume_driver` - (Optional) The name of the volume driver used to mount
   216    volumes. Must be used along with `volumes`.
   217    Using a `volume_driver` also allows to use `volumes` with a named volume as
   218    well as absolute paths. If `docker.volumes.enabled` is false then volume
   219    drivers are disallowed.
   220  
   221      ```hcl
   222      config {
   223        volumes = [
   224          # Use named volume created outside nomad.
   225          "name-of-the-volume:/path/in/container"
   226        ]
   227        # Name of the Docker Volume Driver used by the container
   228        volume_driver = "flocker"
   229      }
   230      ```
   231  
   232  * `work_dir` - (Optional) The working directory inside the container.
   233  
   234  ### Container Name
   235  
   236  Nomad creates a container after pulling an image. Containers are named
   237  `{taskName}-{allocId}`. This is necessary in order to place more than one
   238  container from the same task on a host (e.g. with count > 1). This also means
   239  that each container's name is unique across the cluster.
   240  
   241  This is not configurable.
   242  
   243  ### Authentication
   244  
   245  If you want to pull from a private repo (for example on dockerhub or quay.io),
   246  you will need to specify credentials in your job via the `auth` option or by
   247  storing the credentials in a file and setting the
   248  [docker.auth.config](#auth_file) value on the client.
   249  
   250  The `auth` object supports the following keys:
   251  
   252  * `username` - (Optional) The account username.
   253  
   254  * `password` - (Optional) The account password.
   255  
   256  * `email` - (Optional) The account email.
   257  
   258  * `server_address` - (Optional) The server domain/IP without the protocol.
   259    Docker Hub is used by default.
   260  
   261  Example:
   262  
   263  ```hcl
   264  task "example" {
   265    driver = "docker"
   266  
   267    config {
   268      image = "secret/service"
   269  
   270      auth {
   271        username = "dockerhub_user"
   272        password = "dockerhub_password"
   273      }
   274    }
   275  }
   276  ```
   277  
   278  !> **Be Careful!** At this time these credentials are stored in Nomad in plain
   279  text. Secrets management will be added in a later release.
   280  
   281  ## Networking
   282  
   283  Docker supports a variety of networking configurations, including using host
   284  interfaces, SDNs, etc. Nomad uses `bridged` networking by default, like Docker.
   285  
   286  You can specify other networking options, including custom networking plugins
   287  in Docker 1.9. **You may need to perform additional configuration on the host
   288  in order to make these work.** This additional configuration is outside the
   289  scope of Nomad.
   290  
   291  ### Allocating Ports
   292  
   293  You can allocate ports to your task using the port syntax described on the
   294  [networking page](/docs/job-specification/network.html). Here is a recap:
   295  
   296  ```hcl
   297  task "example" {
   298    driver = "docker"
   299  
   300    resources {
   301      network {
   302        port "http" {}
   303        port "https" {}
   304      }
   305    }
   306  }
   307  ```
   308  
   309  ### Forwarding and Exposing Ports
   310  
   311  A Docker container typically specifies which port a service will listen on by
   312  specifying the `EXPOSE` directive in the `Dockerfile`.
   313  
   314  Because dynamic ports will not match the ports exposed in your Dockerfile,
   315  Nomad will automatically expose all of the ports it allocates to your
   316  container.
   317  
   318  These ports will be identified via environment variables. For example:
   319  
   320  ```
   321  port "http" {}
   322  ```
   323  
   324  If Nomad allocates port `23332` to your task for `http`, `23332` will be
   325  automatically exposed and forwarded to your container, and the driver will set
   326  an environment variable `NOMAD_PORT_http` with the value `23332` that you can
   327  read inside your container.
   328  
   329  This provides an easy way to use the `host` networking option for better
   330  performance.
   331  
   332  ### Using the Port Map
   333  
   334  If you prefer to use the traditional port-mapping method, you can specify the
   335  `port_map` option in your job specification. It looks like this:
   336  
   337  ```hcl
   338  task "example" {
   339    driver = "docker"
   340  
   341    config {
   342      image = "redis"
   343  
   344      port_map {
   345        redis = 6379
   346      }
   347    }
   348  
   349    resources {
   350      network {
   351        mbits = 20
   352        port "redis" {}
   353      }
   354    }
   355  }
   356  ```
   357  
   358  If Nomad allocates port `23332` to your task, the Docker driver will
   359  automatically setup the port mapping from `23332` on the host to `6379` in your
   360  container, so it will just work!
   361  
   362  Note that by default this only works with `bridged` networking mode. It may
   363  also work with custom networking plugins which implement the same API for
   364  expose and port forwarding.
   365  
   366  ### Networking Protocols
   367  
   368  The Docker driver configures ports on both the `tcp` and `udp` protocols.
   369  
   370  This is not configurable.
   371  
   372  ### Other Networking Modes
   373  
   374  Some networking modes like `container` or `none` will require coordination
   375  outside of Nomad. First-class support for these options may be improved later
   376  through Nomad plugins or dynamic job configuration.
   377  
   378  ## Client Requirements
   379  
   380  Nomad requires Docker to be installed and running on the host alongside the
   381  Nomad agent. Nomad was developed against Docker `1.8.2` and `1.9`.
   382  
   383  By default Nomad communicates with the Docker daemon using the daemon's unix
   384  socket. Nomad will need to be able to read/write to this socket. If you do not
   385  run Nomad as root, make sure you add the Nomad user to the Docker group so
   386  Nomad can communicate with the Docker daemon.
   387  
   388  For example, on Ubuntu you can use the `usermod` command to add the `vagrant`
   389  user to the `docker` group so you can run Nomad without root:
   390  
   391      sudo usermod -G docker -a vagrant
   392  
   393  For the best performance and security features you should use recent versions
   394  of the Linux Kernel and Docker daemon.
   395  
   396  ## Client Configuration
   397  
   398  The `docker` driver has the following [client configuration
   399  options](/docs/agent/configuration/client.html#options):
   400  
   401  * `docker.endpoint` - Defaults to `unix:///var/run/docker.sock`. You will need
   402    to customize this if you use a non-standard socket (HTTP or another
   403    location).
   404  
   405  * `docker.auth.config` <a id="auth_file"></a>- Allows an operator to specify a
   406    JSON file which is in the dockercfg format containing authentication
   407    information for a private registry.
   408  
   409  * `docker.tls.cert` - Path to the server's certificate file (`.pem`). Specify
   410    this along with `docker.tls.key` and `docker.tls.ca` to use a TLS client to
   411    connect to the docker daemon. `docker.endpoint` must also be specified or
   412    this setting will be ignored.
   413  
   414  * `docker.tls.key` - Path to the client's private key (`.pem`). Specify this
   415    along with `docker.tls.cert` and `docker.tls.ca` to use a TLS client to
   416    connect to the docker daemon. `docker.endpoint` must also be specified or
   417    this setting will be ignored.
   418  
   419  * `docker.tls.ca` - Path to the server's CA file (`.pem`). Specify this along
   420    with `docker.tls.cert` and `docker.tls.key` to use a TLS client to connect to
   421    the docker daemon. `docker.endpoint` must also be specified or this setting
   422    will be ignored.
   423  
   424  * `docker.cleanup.image` Defaults to `true`. Changing this to `false` will
   425    prevent Nomad from removing images from stopped tasks.
   426  
   427  * `docker.cleanup.image.delay` A time duration that defaults to `3m`. The delay
   428    controls how long Nomad will wait between an image being unused and deleting
   429    it. If a tasks is received that uses the same image within the delay, the
   430    image will be reused.
   431  
   432  * `docker.volumes.enabled`: Defaults to `true`. Allows tasks to bind host paths
   433    (`volumes`) inside their container and use volume drivers (`volume_driver`).
   434    Binding relative paths is always allowed and will be resolved relative to the
   435    allocation's directory.
   436  
   437  * `docker.volumes.selinuxlabel`: Allows the operator to set a SELinux
   438    label to the allocation and task local bind-mounts to containers. If used
   439    with `docker.volumes.enabled` set to false, the labels will still be applied
   440    to the standard binds in the container.
   441  
   442  * `docker.privileged.enabled` Defaults to `false`. Changing this to `true` will
   443    allow containers to use `privileged` mode, which gives the containers full
   444    access to the host's devices. Note that you must set a similar setting on the
   445    Docker daemon for this to work.
   446  
   447  Note: When testing or using the `-dev` flag you can use `DOCKER_HOST`,
   448  `DOCKER_TLS_VERIFY`, and `DOCKER_CERT_PATH` to customize Nomad's behavior. If
   449  `docker.endpoint` is set Nomad will **only** read client configuration from the
   450  config file.
   451  
   452  An example is given below:
   453  
   454  ```hcl
   455  client {
   456    options {
   457      "docker.cleanup.image" = "false"
   458    }
   459  }
   460  ```
   461  
   462  ## Client Attributes
   463  
   464  The `docker` driver will set the following client attributes:
   465  
   466  * `driver.docker` - This will be set to "1", indicating the driver is
   467    available.
   468  * `driver.docker.version` - This will be set to version of the docker server.
   469  
   470  Here is an example of using these properties in a job file:
   471  
   472  ```hcl
   473  job "docs" {
   474    # Require docker version higher than 1.2.
   475    constraint {
   476      attribute = "${driver.docker.version}"
   477      operator  = ">"
   478      version   = "1.2"
   479    }
   480  }
   481  ```
   482  
   483  ## Resource Isolation
   484  
   485  ### CPU
   486  
   487  Nomad limits containers' CPU based on CPU shares. CPU shares allow containers
   488  to burst past their CPU limits. CPU limits will only be imposed when there is
   489  contention for resources. When the host is under load your process may be
   490  throttled to stabilize QOS depending on how many shares it has. You can see how
   491  many CPU shares are available to your process by reading `NOMAD_CPU_LIMIT`.
   492  1000 shares are approximately equal to 1Ghz.
   493  
   494  Please keep the implications of CPU shares in mind when you load test workloads
   495  on Nomad.
   496  
   497  ### Memory
   498  
   499  Nomad limits containers' memory usage based on total virtual memory. This means
   500  that containers scheduled by Nomad cannot use swap. This is to ensure that a
   501  swappy process does not degrade performance for other workloads on the same
   502  host.
   503  
   504  Since memory is not an elastic resource, you will need to make sure your
   505  container does not exceed the amount of memory allocated to it, or it will be
   506  terminated or crash when it tries to malloc. A process can inspect its memory
   507  limit by reading `NOMAD_MEMORY_LIMIT`, but will need to track its own memory
   508  usage. Memory limit is expressed in megabytes so 1024 = 1Gb.
   509  
   510  ### IO
   511  
   512  Nomad's Docker integration does not currently provide QOS around network or
   513  filesystem IO. These will be added in a later release.
   514  
   515  ### Security
   516  
   517  Docker provides resource isolation by way of
   518  [cgroups and namespaces](https://docs.docker.com/introduction/understanding-docker/#the-underlying-technology).
   519  Containers essentially have a virtual file system all to themselves. If you
   520  need a higher degree of isolation between processes for security or other
   521  reasons, it is recommended to use full virtualization like
   522  [QEMU](/docs/drivers/qemu.html).
   523  
   524  ## Docker For Mac Caveats
   525  
   526  Docker For Mac runs docker inside a small VM and then allows access to parts of
   527  the host filesystem into that VM. At present, nomad uses a syslog server bound to
   528  a unix socket within a path that both the host and the VM can access to forward
   529  log messages back to nomad. But at present, Docker For Mac does not work for
   530  unix domain sockets (https://github.com/docker/for-mac/issues/483) in one of
   531  these shared paths.
   532  
   533  As a result, using nomad with the docker driver on OS X/macOS will work, but no
   534  logs will be available to nomad. Users must use the native docker facilities to
   535  examine the logs of any jobs running under docker.
   536  
   537  In the future, we will resolve this issue, one way or another.