github.com/adityamillind98/nomad@v0.11.8/website/pages/docs/drivers/docker.mdx (about)

     1  ---
     2  layout: docs
     3  page_title: 'Drivers: Docker'
     4  sidebar_title: Docker
     5  description: The Docker task driver is used to run Docker based tasks.
     6  ---
     7  
     8  # Docker Driver
     9  
    10  Name: `docker`
    11  
    12  The `docker` driver provides a first-class Docker workflow on Nomad. The Docker
    13  driver handles downloading containers, mapping ports, and starting, watching,
    14  and cleaning up after containers.
    15  
    16  ## Task Configuration
    17  
    18  ```hcl
    19  task "webservice" {
    20    driver = "docker"
    21  
    22    config {
    23      image = "redis:3.2"
    24      labels {
    25        group = "webservice-cache"
    26      }
    27    }
    28  }
    29  ```
    30  
    31  The `docker` driver supports the following configuration in the job spec. Only
    32  `image` is required.
    33  
    34  - `image` - The Docker image to run. The image may include a tag or custom URL
    35    and should include `https://` if required. By default it will be fetched from
    36    Docker Hub. If the tag is omitted or equal to `latest` the driver will always
    37    try to pull the image. If the image to be pulled exists in a registry that
    38    requires authentication credentials must be provided to Nomad. Please see the
    39    [Authentication section](#authentication).
    40  
    41    ```hcl
    42    config {
    43      image = "https://hub.docker.internal/redis:3.2"
    44    }
    45    ```
    46  
    47  - `args` - (Optional) A list of arguments to the optional `command`. If no
    48    `command` is specified, the arguments are passed directly to the container.
    49    References to environment variables or any [interpretable Nomad
    50    variables](/docs/runtime/interpolation) will be interpreted before
    51    launching the task. For example:
    52  
    53    ```hcl
    54    config {
    55      args = [
    56        "-bind", "${NOMAD_PORT_http}",
    57        "${nomad.datacenter}",
    58        "${MY_ENV}",
    59        "${meta.foo}",
    60      ]
    61    }
    62    ```
    63  
    64  - `auth` - (Optional) Provide authentication for a private registry (see below).
    65  
    66  - `auth_soft_fail` `(bool: false)` - Don't fail the task on an auth failure.
    67    Attempt to continue without auth.
    68  
    69  - `command` - (Optional) The command to run when starting the container.
    70  
    71    ```hcl
    72    config {
    73      command = "my-command"
    74    }
    75    ```
    76  
    77  - `dns_search_domains` - (Optional) A list of DNS search domains for the container
    78    to use.
    79  
    80  - `dns_options` - (Optional) A list of DNS options for the container to use.
    81  
    82  - `dns_servers` - (Optional) A list of DNS servers for the container to use
    83    (e.g. ["8.8.8.8", "8.8.4.4"]). Requires Docker v1.10 or greater.
    84  
    85  - `entrypoint` - (Optional) A string list overriding the image's entrypoint.
    86  
    87  - `extra_hosts` - (Optional) A list of hosts, given as host:IP, to be added to
    88    `/etc/hosts`.
    89  
    90  - `force_pull` - (Optional) `true` or `false` (default). Always pull most recent image
    91    instead of using existing local image. Should be set to `true` if repository tags
    92    are mutable. If image's tag is `latest` or omitted, the image will always be pulled
    93    regardless of this setting.
    94  
    95  - `hostname` - (Optional) The hostname to assign to the container. When
    96    launching more than one of a task (using `count`) with this option set, every
    97    container the task starts will have the same hostname.
    98  
    99  - `interactive` - (Optional) `true` or `false` (default). Keep STDIN open on
   100    the container.
   101  
   102  - `sysctl` - (Optional) A key-value map of sysctl configurations to set to the
   103    containers on start.
   104  
   105    ```hcl
   106    config {
   107      sysctl {
   108        net.core.somaxconn = "16384"
   109      }
   110    }
   111    ```
   112  
   113  - `ulimit` - (Optional) A key-value map of ulimit configurations to set to the
   114    containers on start.
   115  
   116    ```hcl
   117    config {
   118      ulimit {
   119        nproc = "4242"
   120        nofile = "2048:4096"
   121      }
   122    }
   123    ```
   124  
   125  - `privileged` - (Optional) `true` or `false` (default). Privileged mode gives
   126    the container access to devices on the host. Note that this also requires the
   127    nomad agent and docker daemon to be configured to allow privileged
   128    containers.
   129  
   130  - `ipc_mode` - (Optional) The IPC mode to be used for the container. The default
   131    is `none` for a private IPC namespace. Other values are `host` for sharing
   132    the host IPC namespace or the name or id of an existing container. Note that
   133    it is not possible to refer to Docker containers started by Nomad since their
   134    names are not known in advance. Note that setting this option also requires the
   135    Nomad agent to be configured to allow privileged containers.
   136  
   137  - `ipv4_address` - (Optional) The IPv4 address to be used for the container when
   138    using user defined networks. Requires Docker 1.13 or greater.
   139  
   140  - `ipv6_address` - (Optional) The IPv6 address to be used for the container when
   141    using user defined networks. Requires Docker 1.13 or greater.
   142  
   143  - `labels` - (Optional) A key-value map of labels to set to the containers on
   144    start.
   145  
   146    ```hcl
   147    config {
   148      labels {
   149        foo = "bar"
   150        zip = "zap"
   151      }
   152    }
   153    ```
   154  
   155  - `load` - (Optional) Load an image from a `tar` archive file instead of from a
   156    remote repository. Equivalent to the `docker load -i <filename>` command.
   157  
   158    ```hcl
   159    artifact {
   160      source = "http://path.to/redis.tar"
   161    }
   162    config {
   163      load = "redis.tar"
   164      image = "redis"
   165    }
   166    ```
   167  
   168  - `logging` - (Optional) A key-value map of Docker logging options.
   169    Defaults to `json-file` with log rotation (`max-file=2` and `max-size=2m`).
   170  
   171    ```hcl
   172    config {
   173      logging {
   174        type = "fluentd"
   175        config {
   176          fluentd-address = "localhost:24224"
   177          tag = "your_tag"
   178        }
   179      }
   180    }
   181    ```
   182  
   183  - `mac_address` - (Optional) The MAC address for the container to use (e.g.
   184    "02:68:b3:29:da:98").
   185  
   186  - `memory_hard_limit` - (Optional) The maximum allowable amount of memory used
   187    (megabytes) by the container. If set, the [`memory`](/docs/job-specification/resources#memory)
   188    parameter of the task resource configuration becomes a soft limit passed to the
   189    docker driver as [`--memory_reservation`](https://docs.docker.com/config/containers/resource_constraints/#limit-a-containers-access-to-memory),
   190    and `memory_hard_limit` is passed as the [`--memory`](https://docs.docker.com/config/containers/resource_constraints/#limit-a-containers-access-to-memory)
   191    hard limit. When the host is under memory pressure, the behavior of soft limit
   192    activation is governed by the [Kernel](https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt).
   193  
   194  - `network_aliases` - (Optional) A list of network-scoped aliases, provide a way for a
   195    container to be discovered by an alternate name by any other container within
   196    the scope of a particular network. Network-scoped alias is supported only for
   197    containers in user defined networks
   198  
   199    ```hcl
   200    config {
   201      network_mode = "user-network"
   202      network_aliases = [
   203        "${NOMAD_TASK_NAME}",
   204        "${NOMAD_TASK_NAME}-${NOMAD_ALLOC_INDEX}"
   205      ]
   206    }
   207    ```
   208  
   209  - `network_mode` - (Optional) The network mode to be used for the container. In
   210    order to support userspace networking plugins in Docker 1.9 this accepts any
   211    value. The default is `bridge` for all operating systems but Windows, which
   212    defaults to `nat`. Other networking modes may not work without additional
   213    configuration on the host (which is outside the scope of Nomad). Valid values
   214    pre-docker 1.9 are `default`, `bridge`, `host`, `none`, or `container:name`.
   215  
   216  - `pid_mode` - (Optional) `host` or not set (default). Set to `host` to share
   217    the PID namespace with the host. Note that this also requires the Nomad agent
   218    to be configured to allow privileged containers.
   219    See below for more details.
   220  
   221  - `port_map` - (Optional) A key-value map of port labels (see below).
   222  
   223  - `security_opt` - (Optional) A list of string flags to pass directly to
   224    [`--security-opt`](https://docs.docker.com/engine/reference/run/#security-configuration).
   225    For example:
   226  
   227    ```hcl
   228    config {
   229      security_opt = [
   230        "credentialspec=file://gmsaUser.json",
   231      ]
   232    }
   233    ```
   234  
   235  - `shm_size` - (Optional) The size (bytes) of /dev/shm for the container.
   236  
   237  - `storage_opt` - (Optional) A key-value map of storage options set to the containers on start.
   238    This overrides the [host dockerd configuration](https://docs.docker.com/engine/reference/commandline/dockerd/#options-per-storage-driver).
   239    For example:
   240  
   241    ```hcl
   242    config {
   243      storage_opt = {
   244        size = "40G"
   245      }
   246    }
   247    ```
   248  
   249  - `SSL` - (Optional) If this is set to true, Nomad uses SSL to talk to the
   250    repository. The default value is `true`. **Deprecated as of 0.5.3**
   251  
   252  - `tty` - (Optional) `true` or `false` (default). Allocate a pseudo-TTY for the
   253    container.
   254  
   255  - `uts_mode` - (Optional) `host` or not set (default). Set to `host` to share
   256    the UTS namespace with the host. Note that this also requires the Nomad agent
   257    to be configured to allow privileged containers.
   258  
   259  - `userns_mode` - (Optional) `host` or not set (default). Set to `host` to use
   260    the host's user namespace when user namespace remapping is enabled on the
   261    docker daemon.
   262  
   263  - `volumes` - (Optional) A list of `host_path:container_path` strings to bind
   264    host paths to container paths. Mounting host paths outside of the allocation
   265    directory can be disabled on clients by setting the `docker.volumes.enabled`
   266    option set to false. This will limit volumes to directories that exist inside
   267    the allocation directory. We recommend using [`mounts`](#mounts) if you wish
   268    to have more control over volume definitions.
   269  
   270    ```hcl
   271    config {
   272      volumes = [
   273        # Use absolute paths to mount arbitrary paths on the host
   274        "/path/on/host:/path/in/container",
   275  
   276        # Use relative paths to rebind paths already in the allocation dir
   277        "relative/to/task:/also/in/container"
   278      ]
   279    }
   280    ```
   281  
   282  - `volume_driver` - (Optional) The name of the volume driver used to mount
   283    volumes. Must be used along with `volumes`. If `volume_driver` is omitted,
   284    then relative paths will be mounted from inside the allocation dir. If a
   285    `"local"` or other driver is used, then they may be named volumes instead.
   286    If `docker.volumes.enabled` is false then volume drivers and paths outside the
   287    allocation directory are disallowed.
   288  
   289    ```hcl
   290    config {
   291      volumes = [
   292        # Use named volume created outside nomad.
   293        "name-of-the-volume:/path/in/container"
   294      ]
   295      # Name of the Docker Volume Driver used by the container
   296      volume_driver = "pxd"
   297    }
   298    ```
   299  
   300  - `work_dir` - (Optional) The working directory inside the container.
   301  
   302  - `mounts` - (Optional) A list of
   303    [mounts](https://docs.docker.com/engine/reference/commandline/service_create/#add-bind-mounts-or-volumes)
   304    to be mounted into the container. Volume, bind, and tmpfs type mounts are supported.
   305  
   306    ```hcl
   307    config {
   308      mounts = [
   309        # sample volume mount
   310        {
   311          type = "volume"
   312          target = "/path/in/container"
   313          source = "name-of-volume"
   314          readonly = false
   315          volume_options {
   316            no_copy = false
   317            labels {
   318              foo = "bar"
   319            }
   320            driver_config {
   321              name = "pxd"
   322              options = {
   323                foo = "bar"
   324              }
   325            }
   326          }
   327        },
   328        # sample bind mount
   329        {
   330          type = "bind"
   331          target = "/path/in/container"
   332          source = "/path/in/host"
   333          readonly = false
   334          bind_options {
   335            propagation = "rshared"
   336          }
   337        },
   338        # sample tmpfs mount
   339        {
   340          type = "tmpfs"
   341          target = "/path/in/container"
   342          readonly = false
   343          tmpfs_options {
   344            size = 100000 # size in bytes
   345          }
   346        }
   347      ]
   348    }
   349    ```
   350  
   351  - `devices` - (Optional) A list of
   352    [devices](https://docs.docker.com/engine/reference/commandline/run/#add-host-device-to-container-device)
   353    to be exposed the container. `host_path` is the only required field. By default, the container will be able to
   354    `read`, `write` and `mknod` these devices. Use the optional `cgroup_permissions` field to restrict permissions.
   355  
   356    ```hcl
   357    config {
   358      devices = [
   359        {
   360          host_path = "/dev/sda1"
   361          container_path = "/dev/xvdc"
   362          cgroup_permissions = "r"
   363        },
   364        {
   365          host_path = "/dev/sda2"
   366          container_path = "/dev/xvdd"
   367        }
   368      ]
   369    }
   370    ```
   371  
   372  - `cap_add` - (Optional) A list of Linux capabilities as strings to pass directly to
   373    [`--cap-add`](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities).
   374    Effective capabilities (computed from `cap_add` and `cap_drop`) have to match the configured whitelist.
   375    The whitelist can be customized using the [`allow_caps`](#plugin_caps) plugin option key in the client node's configuration.
   376    For example:
   377  
   378    ```hcl
   379    config {
   380      cap_add = [
   381        "SYS_TIME",
   382      ]
   383    }
   384    ```
   385  
   386  - `cap_drop` - (Optional) A list of Linux capabilities as strings to pass directly to
   387    [`--cap-drop`](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities).
   388    Effective capabilities (computed from `cap_add` and `cap_drop`) have to match the configured whitelist.
   389    The whitelist can be customized using the [`allow_caps`](#plugin_caps) plugin option key in the client node's configuration.
   390    For example:
   391  
   392    ```hcl
   393    config {
   394      cap_drop = [
   395        "MKNOD",
   396      ]
   397    }
   398    ```
   399  
   400  - `cpu_hard_limit` - (Optional) `true` or `false` (default). Use hard CPU
   401    limiting instead of soft limiting. By default this is `false` which means
   402    soft limiting is used and containers are able to burst above their CPU limit
   403    when there is idle capacity.
   404  
   405  - `cpu_cfs_period` - (Optional) An integer value that specifies the duration in microseconds of the period
   406    during which the CPU usage quota is measured. The default is 100000 (0.1 second) and the maximum allowed
   407    value is 1000000 (1 second). See [here](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/resource_management_guide/sec-cpu#sect-cfs)
   408    for more details.
   409  
   410  - `advertise_ipv6_address` - (Optional) `true` or `false` (default). Use the container's
   411    IPv6 address (GlobalIPv6Address in Docker) when registering services and checks.
   412    See [IPv6 Docker containers](/docs/job-specification/service#IPv6 Docker containers) for details.
   413  
   414  - `readonly_rootfs` - (Optional) `true` or `false` (default). Mount
   415    the container's filesystem as read only.
   416  
   417  - `runtime` - (Optional) A string representing a configured runtime to pass to docker.
   418    This is equivalent to the `--runtime` argument in the docker CLI
   419    For example, to use gVisor:
   420  
   421    ```hcl
   422    config {
   423      # gVisor runtime is runsc
   424      runtime = "runsc"
   425    }
   426    ```
   427  
   428  - `pids_limit` - (Optional) An integer value that specifies the pid limit for
   429    the container. Defaults to unlimited.
   430  
   431  Additionally, the docker driver supports customization of the container's user through the task's [`user` option](/docs/job-specification/task#user).
   432  
   433  ### Container Name
   434  
   435  Nomad creates a container after pulling an image. Containers are named
   436  `{taskName}-{allocId}`. This is necessary in order to place more than one
   437  container from the same task on a host (e.g. with count > 1). This also means
   438  that each container's name is unique across the cluster.
   439  
   440  This is not configurable.
   441  
   442  ### Authentication
   443  
   444  If you want to pull from a private repo (for example on dockerhub or quay.io),
   445  you will need to specify credentials in your job via:
   446  
   447  - the `auth` option in the task config.
   448  
   449  - by storing explicit repository credentials or by specifying Docker
   450    `credHelpers` in a file and setting the auth [config](#plugin_auth_file)
   451    value on the client in the plugin options.
   452  
   453  - by specifying an auth [helper](#plugin_auth_helper) on the client in the
   454    plugin options.
   455  
   456  The `auth` object supports the following keys:
   457  
   458  - `username` - (Optional) The account username.
   459  
   460  - `password` - (Optional) The account password.
   461  
   462  - `email` - (Optional) The account email.
   463  
   464  - `server_address` - (Optional) The server domain/IP without the protocol.
   465    Docker Hub is used by default.
   466  
   467  Example task-config:
   468  
   469  ```hcl
   470  task "example" {
   471    driver = "docker"
   472  
   473    config {
   474      image = "secret/service"
   475  
   476      auth {
   477        username = "dockerhub_user"
   478        password = "dockerhub_password"
   479      }
   480    }
   481  }
   482  ```
   483  
   484  Example docker-config, using two helper scripts in \$PATH,
   485  "docker-credential-ecr" and "docker-credential-vault":
   486  
   487  ```json
   488  {
   489    "auths": {
   490      "internal.repo": {
   491        "auth": "`echo -n '<username>:<password>' | base64 -w0`"
   492      }
   493    },
   494    "credHelpers": {
   495      "<XYZ>.dkr.ecr.<region>.amazonaws.com": "ecr-login"
   496    },
   497    "credsStore": "secretservice"
   498  }
   499  ```
   500  
   501  Example agent configuration, using a helper script "docker-credential-ecr" in
   502  \$PATH
   503  
   504  ```hcl
   505  client {
   506    enabled = true
   507  }
   508  
   509  plugin "docker" {
   510    config {
   511      auth {
   512        # Nomad will prepend "docker-credential-" to the helper value and call
   513        # that script name.
   514        helper = "ecr"
   515      }
   516    }
   517  }
   518  ```
   519  
   520  !> **Be Careful!** At this time these credentials are stored in Nomad in plain
   521  text. Secrets management will be added in a later release.
   522  
   523  ## Networking
   524  
   525  Docker supports a variety of networking configurations, including using host
   526  interfaces, SDNs, etc. Nomad uses `bridged` networking by default, like Docker.
   527  
   528  You can specify other networking options, including custom networking plugins
   529  in Docker 1.9. **You may need to perform additional configuration on the host
   530  in order to make these work.** This additional configuration is outside the
   531  scope of Nomad.
   532  
   533  ### Allocating Ports
   534  
   535  You can allocate ports to your task using the port syntax described on the
   536  [networking page](/docs/job-specification/network). Here is a recap:
   537  
   538  ```hcl
   539  task "example" {
   540    driver = "docker"
   541  
   542    resources {
   543      network {
   544        port "http" {}
   545        port "https" {}
   546      }
   547    }
   548  }
   549  ```
   550  
   551  ### Forwarding and Exposing Ports
   552  
   553  A Docker container typically specifies which port a service will listen on by
   554  specifying the `EXPOSE` directive in the `Dockerfile`.
   555  
   556  Because dynamic ports will not match the ports exposed in your Dockerfile,
   557  Nomad will automatically expose all of the ports it allocates to your
   558  container.
   559  
   560  These ports will be identified via environment variables. For example:
   561  
   562  ```hcl
   563  port "http" {}
   564  ```
   565  
   566  If Nomad allocates port `23332` to your task for `http`, `23332` will be
   567  automatically exposed and forwarded to your container, and the driver will set
   568  an environment variable `NOMAD_PORT_http` with the value `23332` that you can
   569  read inside your container.
   570  
   571  This provides an easy way to use the `host` networking option for better
   572  performance.
   573  
   574  ### Using the Port Map
   575  
   576  If you prefer to use the traditional port-mapping method, you can specify the
   577  `port_map` option in your job specification. It looks like this:
   578  
   579  ```hcl
   580  task "example" {
   581    driver = "docker"
   582  
   583    config {
   584      image = "redis"
   585  
   586      port_map {
   587        redis = 6379
   588      }
   589    }
   590  
   591    resources {
   592      network {
   593        mbits = 20
   594        port "redis" {}
   595      }
   596    }
   597  }
   598  ```
   599  
   600  If Nomad allocates port `23332` to your task, the Docker driver will
   601  automatically setup the port mapping from `23332` on the host to `6379` in your
   602  container, so it will just work!
   603  
   604  Note that by default this only works with `bridged` networking mode. It may
   605  also work with custom networking plugins which implement the same API for
   606  expose and port forwarding.
   607  
   608  ### Advertising Container IPs
   609  
   610  _New in Nomad 0.6._
   611  
   612  When using network plugins like `weave` that assign containers a routable IP
   613  address, that address will automatically be used in any `service`
   614  advertisements for the task. You may override what address is advertised by
   615  using the `address_mode` parameter on a `service`. See
   616  [service](/docs/job-specification/service) for details.
   617  
   618  ### Networking Protocols
   619  
   620  The Docker driver configures ports on both the `tcp` and `udp` protocols.
   621  
   622  This is not configurable.
   623  
   624  ### Other Networking Modes
   625  
   626  Some networking modes like `container` or `none` will require coordination
   627  outside of Nomad. First-class support for these options may be improved later
   628  through Nomad plugins or dynamic job configuration.
   629  
   630  ## Client Requirements
   631  
   632  Nomad requires Docker to be installed and running on the host alongside the
   633  Nomad agent. Nomad was developed against Docker `1.8.2` and `1.9`.
   634  
   635  By default Nomad communicates with the Docker daemon using the daemon's Unix
   636  socket. Nomad will need to be able to read/write to this socket. If you do not
   637  run Nomad as root, make sure you add the Nomad user to the Docker group so
   638  Nomad can communicate with the Docker daemon.
   639  
   640  For example, on Ubuntu you can use the `usermod` command to add the `vagrant`
   641  user to the `docker` group so you can run Nomad without root:
   642  
   643  ```shell-session
   644  $ sudo usermod -G docker -a vagrant
   645  ```
   646  
   647  For the best performance and security features you should use recent versions
   648  of the Linux Kernel and Docker daemon.
   649  
   650  If you would like to change any of the options related to the `docker` driver on
   651  a Nomad client, you can modify them with the [plugin stanza][plugin-stanza] syntax. Below is an example of a configuration (many of the values are the default). See the next section for more information on the options.
   652  
   653  ```hcl
   654  plugin "docker" {
   655    config {
   656      endpoint = "unix:///var/run/docker.sock"
   657  
   658      auth {
   659        config = "/etc/docker-auth.json"
   660        helper = "docker-credential-aws"
   661      }
   662  
   663      tls {
   664        cert = "/etc/nomad/nomad.pub"
   665        key  = "/etc/nomad/nomad.pem"
   666        ca   = "/etc/nomad/nomad.cert"
   667      }
   668  
   669      gc {
   670        image       = true
   671        image_delay = "3m"
   672        container   = true
   673  
   674        dangling_containers {
   675          enabled        = true
   676          dry_run        = false
   677          period         = "5m"
   678          creation_grace = "5m"
   679        }
   680      }
   681  
   682      volumes {
   683        enabled      = true
   684        selinuxlabel = "z"
   685      }
   686  
   687      allow_privileged = false
   688      allow_caps       = ["CHOWN", "NET_RAW"]
   689  
   690      # allow_caps can also be set to "ALL"
   691      # allow_caps = ["ALL"]
   692    }
   693  }
   694  ```
   695  
   696  ## Plugin Options
   697  
   698  - `endpoint` - If using a non-standard socket, HTTP or another location, or if
   699    TLS is being used, docker.endpoint must be set. If unset, Nomad will attempt
   700    to instantiate a Docker client using the DOCKER_HOST environment variable and
   701    then fall back to the default listen address for the given operating system.
   702    Defaults to unix:///var/run/docker.sock on Unix platforms and
   703    npipe:////./pipe/docker_engine for Windows.
   704  
   705  - `allow_privileged` - Defaults to `false`. Changing this to true will allow
   706    containers to use privileged mode, which gives the containers full access to
   707    the host's devices. Note that you must set a similar setting on the Docker
   708    daemon for this to work.
   709  
   710  - `pull_activity_timeout` - Defaults to `2m`. If Nomad receives no communication
   711    from the Docker engine during an image pull within this timeframe, Nomad will
   712    timeout the request that initiated the pull command. (Minimum of `1m`)
   713  
   714  - `allow_caps`<a id="plugin_caps"></a> - A list of allowed Linux capabilities.
   715    Defaults to
   716    "CHOWN,DAC_OVERRIDE,FSETID,FOWNER,MKNOD,NET_RAW,SETGID,SETUID,SETFCAP,SETPCAP,
   717    NET_BIND_SERVICE,SYS_CHROOT,KILL,AUDIT_WRITE", which is the list of
   718    capabilities allowed by docker by default, as defined here. Allows the
   719    operator to control which capabilities can be obtained by tasks using cap_add
   720    and cap_drop options. Supports the value "ALL" as a shortcut for whitelisting
   721    all capabilities.
   722  
   723  - `allow_runtimes` - defaults to `["runc", "nvidia"]` - A list of the allowed
   724    docker runtimes a task may use.
   725  
   726  - `auth` stanza:
   727  
   728    - `config`<a id="plugin_auth_file"></a> - Allows an operator to specify a
   729      JSON file which is in the dockercfg format containing authentication
   730      information for a private registry, from either (in order) `auths`,
   731      `credHelpers` or `credsStore`.
   732    - `helper`<a id="plugin_auth_helper"></a> - Allows an operator to specify a
   733      [credsStore](https://docs.docker.com/engine/reference/commandline/login/#credential-helper-protocol)
   734      -like script on \$PATH to lookup authentication information from external
   735      sources. The script's name must begin with `docker-credential-` and this
   736      option should include only the basename of the script, not the path.
   737  
   738  - `tls` stanza:
   739  
   740    - `cert` - Path to the server's certificate file (`.pem`). Specify this
   741      along with `key` and `ca` to use a TLS client to connect to the docker
   742      daemon. `endpoint` must also be specified or this setting will be ignored.
   743    - `key` - Path to the client's private key (`.pem`). Specify this along with
   744      `cert` and `ca` to use a TLS client to connect to the docker daemon.
   745      `endpoint` must also be specified or this setting will be ignored.
   746    - `ca` - Path to the server's CA file (`.pem`). Specify this along with
   747      `cert` and `key` to use a TLS client to connect to the docker daemon.
   748      `endpoint` must also be specified or this setting will be ignored.
   749  
   750  - `disable_log_collection` - Defaults to `false`. Setting this to true will
   751    disable Nomad logs collection of Docker tasks. If you don't rely on nomad log
   752    capabilities and exclusively use host based log aggregation, you may consider
   753    this option to disable nomad log collection overhead.
   754  
   755  - `gc` stanza:
   756  
   757    - `image` - Defaults to `true`. Changing this to `false` will prevent Nomad
   758      from removing images from stopped tasks.
   759    - `image_delay` - A time duration, as [defined
   760      here](https://golang.org/pkg/time/#ParseDuration), that defaults to `3m`.
   761      The delay controls how long Nomad will wait between an image being unused
   762      and deleting it. If a tasks is received that uses the same image within
   763      the delay, the image will be reused.
   764    - `container` - Defaults to `true`. This option can be used to disable Nomad
   765      from removing a container when the task exits. Under a name conflict,
   766      Nomad may still remove the dead container.
   767    - `dangling_containers` stanza for controlling dangling container detection
   768      and cleanup:
   769  
   770      - `enabled` - Defaults to `true`. Enables dangling container handling.
   771      - `dry_run` - Defaults to `false`. Only log dangling containers without
   772        cleaning them up.
   773      - `period` - Defaults to `"5m"`. A time duration that controls interval
   774        between Nomad scans for dangling containers.
   775      - `creation_grace` - Defaults to `"5m"`. Grace period after a container is
   776        created during which the GC ignores it. Only used to prevent the GC from
   777        removing newly created containers before they are registered with the
   778        GC. Should not need adjusting higher but may be adjusted lower to GC
   779        more aggressively.
   780  
   781  - `volumes` stanza:
   782  
   783    - `enabled` - Defaults to `true`. Allows tasks to bind host paths
   784      (`volumes`) inside their container and use volume drivers
   785      (`volume_driver`). Binding relative paths is always allowed and will be
   786      resolved relative to the allocation's directory.
   787    - `selinuxlabel` - Allows the operator to set a SELinux label to the
   788      allocation and task local bind-mounts to containers. If used with
   789      `docker.volumes.enabled` set to false, the labels will still be applied to
   790      the standard binds in the container.
   791  
   792  - `infra_image` - This is the Docker image to use when creating the parent
   793    container necessary when sharing network namespaces between tasks. Defaults
   794    to "gcr.io/google_containers/pause-amd64:3.0".
   795  
   796  ## Client Configuration
   797  
   798  ~> Note: client configuration options will soon be deprecated. Please use
   799  [plugin options][plugin-options] instead. See the [plugin stanza][plugin-stanza]
   800  documentation for more information.
   801  
   802  The `docker` driver has the following [client configuration
   803  options](/docs/configuration/client#options):
   804  
   805  - `docker.endpoint` - If using a non-standard socket, HTTP or another location,
   806    or if TLS is being used, `docker.endpoint` must be set. If unset, Nomad will
   807    attempt to instantiate a Docker client using the `DOCKER_HOST` environment
   808    variable and then fall back to the default listen address for the given
   809    operating system. Defaults to `unix:///var/run/docker.sock` on Unix platforms
   810    and `npipe:////./pipe/docker_engine` for Windows.
   811  
   812  - `docker.auth.config` <a id="auth_file"></a>- Allows an operator to specify a
   813    JSON file which is in the dockercfg format containing authentication
   814    information for a private registry, from either (in order) `auths`,
   815    `credHelpers` or `credsStore`.
   816  
   817  - `docker.auth.helper` <a id="auth_helper"></a>- Allows an operator to specify a
   818    [credsStore](https://docs.docker.com/engine/reference/commandline/login/#credential-helper-protocol)
   819    -like script on \$PATH to lookup authentication information from external
   820    sources. The script's name must begin with `docker-credential-` and this
   821    option should include only the basename of the script, not the path.
   822  
   823  - `docker.tls.cert` - Path to the server's certificate file (`.pem`). Specify
   824    this along with `docker.tls.key` and `docker.tls.ca` to use a TLS client to
   825    connect to the docker daemon. `docker.endpoint` must also be specified or this
   826    setting will be ignored.
   827  
   828  - `docker.tls.key` - Path to the client's private key (`.pem`). Specify this
   829    along with `docker.tls.cert` and `docker.tls.ca` to use a TLS client to
   830    connect to the docker daemon. `docker.endpoint` must also be specified or this
   831    setting will be ignored.
   832  
   833  - `docker.tls.ca` - Path to the server's CA file (`.pem`). Specify this along
   834    with `docker.tls.cert` and `docker.tls.key` to use a TLS client to connect to
   835    the docker daemon. `docker.endpoint` must also be specified or this setting
   836    will be ignored.
   837  
   838  - `docker.cleanup.image` Defaults to `true`. Changing this to `false` will
   839    prevent Nomad from removing images from stopped tasks.
   840  
   841  - `docker.cleanup.image.delay` A time duration, as [defined
   842    here](https://golang.org/pkg/time/#ParseDuration), that defaults to `3m`. The
   843    delay controls how long Nomad will wait between an image being unused and
   844    deleting it. If a tasks is received that uses the same image within the delay,
   845    the image will be reused.
   846  
   847  - `docker.volumes.enabled`: Defaults to `true`. Allows tasks to bind host paths
   848    (`volumes`) inside their container and use volume drivers (`volume_driver`).
   849    Binding relative paths is always allowed and will be resolved relative to the
   850    allocation's directory.
   851  
   852  - `docker.volumes.selinuxlabel`: Allows the operator to set a SELinux label to
   853    the allocation and task local bind-mounts to containers. If used with
   854    `docker.volumes.enabled` set to false, the labels will still be applied to the
   855    standard binds in the container.
   856  
   857  - `docker.privileged.enabled` Defaults to `false`. Changing this to `true` will
   858    allow containers to use `privileged` mode, which gives the containers full
   859    access to the host's devices. Note that you must set a similar setting on the
   860    Docker daemon for this to work.
   861  
   862  - `docker.caps.whitelist`: A list of allowed Linux capabilities. Defaults to
   863    `"CHOWN,DAC_OVERRIDE,FSETID,FOWNER,MKNOD,NET_RAW,SETGID,SETUID,SETFCAP, SETPCAP,NET_BIND_SERVICE,SYS_CHROOT,KILL,AUDIT_WRITE"`, which is the list of
   864    capabilities allowed by docker by default, as [defined
   865    here](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities).
   866    Allows the operator to control which capabilities can be obtained by tasks
   867    using `cap_add` and `cap_drop` options. Supports the value `"ALL"` as a
   868    shortcut for whitelisting all capabilities.
   869  
   870  - `docker.cleanup.container`: Defaults to `true`. This option can be used to
   871    disable Nomad from removing a container when the task exits. Under a name
   872    conflict, Nomad may still remove the dead container.
   873  
   874  - `docker.nvidia_runtime`: Defaults to `nvidia`. This option allows operators to select the runtime that should be used in order to expose Nvidia GPUs to the container.
   875  
   876  Note: When testing or using the `-dev` flag you can use `DOCKER_HOST`,
   877  `DOCKER_TLS_VERIFY`, and `DOCKER_CERT_PATH` to customize Nomad's behavior. If
   878  `docker.endpoint` is set Nomad will **only** read client configuration from the
   879  config file.
   880  
   881  An example is given below:
   882  
   883  ```hcl
   884  client {
   885    options {
   886      "docker.cleanup.image" = "false"
   887    }
   888  }
   889  ```
   890  
   891  ## Client Attributes
   892  
   893  The `docker` driver will set the following client attributes:
   894  
   895  - `driver.docker` - This will be set to "1", indicating the driver is
   896    available.
   897  - `driver.docker.bridge_ip` - The IP of the Docker bridge network if one
   898    exists.
   899  - `driver.docker.version` - This will be set to version of the docker server.
   900  
   901  Here is an example of using these properties in a job file:
   902  
   903  ```hcl
   904  job "docs" {
   905    # Require docker version higher than 1.2.
   906    constraint {
   907      attribute = "${driver.docker.version}"
   908      operator  = ">"
   909      version   = "1.2"
   910    }
   911  }
   912  ```
   913  
   914  ## Resource Isolation
   915  
   916  ### CPU
   917  
   918  Nomad limits containers' CPU based on CPU shares. CPU shares allow containers
   919  to burst past their CPU limits. CPU limits will only be imposed when there is
   920  contention for resources. When the host is under load your process may be
   921  throttled to stabilize QoS depending on how many shares it has. You can see how
   922  many CPU shares are available to your process by reading `NOMAD_CPU_LIMIT`.
   923  1000 shares are approximately equal to 1 GHz.
   924  
   925  Please keep the implications of CPU shares in mind when you load test workloads
   926  on Nomad.
   927  
   928  ### Memory
   929  
   930  Nomad limits containers' memory usage based on total virtual memory. This means
   931  that containers scheduled by Nomad cannot use swap. This is to ensure that a
   932  swappy process does not degrade performance for other workloads on the same
   933  host.
   934  
   935  Since memory is not an elastic resource, you will need to make sure your
   936  container does not exceed the amount of memory allocated to it, or it will be
   937  terminated or crash when it tries to malloc. A process can inspect its memory
   938  limit by reading `NOMAD_MEMORY_LIMIT`, but will need to track its own memory
   939  usage. Memory limit is expressed in megabytes so 1024 = 1 GB.
   940  
   941  ### IO
   942  
   943  Nomad's Docker integration does not currently provide QoS around network or
   944  filesystem IO. These will be added in a later release.
   945  
   946  ### Security
   947  
   948  Docker provides resource isolation by way of
   949  [cgroups and namespaces](https://docs.docker.com/introduction/understanding-docker/#the-underlying-technology).
   950  Containers essentially have a virtual file system all to themselves. If you
   951  need a higher degree of isolation between processes for security or other
   952  reasons, it is recommended to use full virtualization like
   953  [QEMU](/docs/drivers/qemu).
   954  
   955  ## Caveats
   956  
   957  ### Dangling Containers
   958  
   959  Nomad 0.10.2 introduces a detector and a reaper for dangling Docker containers,
   960  containers that Nomad starts yet does not manage or track. Though rare, they
   961  lead to unexpectedly running services, potentially with stale versions.
   962  
   963  When Docker daemon becomes unavailable as Nomad starts a task, it is possible
   964  for Docker to successfully start the container but return a 500 error code from
   965  the API call. In such cases, Nomad retries and eventually aims to kill such
   966  containers. However, if the Docker Engine remains unhealthy, subsequent retries
   967  and stop attempts may still fail, and the started container becomes a dangling
   968  container that Nomad no longer manages.
   969  
   970  The newly added reaper periodically scans for such containers. It only targets
   971  containers with a `com.hashicorp.nomad.allocation_id` label, or match Nomad's
   972  conventions for naming and bind-mounts (i.e. `/alloc`, `/secrets`, `local`).
   973  Containers that don't match Nomad container patterns are left untouched.
   974  
   975  Operators can run the reaper in a dry-run mode, where it only logs dangling
   976  container ids without killing them, or disable it by setting the
   977  `gc.dangling_containers` config stanza.
   978  
   979  ### Docker for Windows
   980  
   981  Docker for Windows only supports running Windows containers. Because Docker for
   982  Windows is relatively new and rapidly evolving you may want to consult the
   983  [list of relevant issues on GitHub][winissues].
   984  
   985  [winissues]: https://github.com/hashicorp/nomad/issues?q=is%3Aopen+is%3Aissue+label%3Adriver%2Fdocker+label%3Aplatform-windows
   986  [plugin-options]: #plugin-options
   987  [plugin-stanza]: /docs/configuration/plugin