github.com/cspotcode/docker-cli@v20.10.0-rc1.0.20201201121459-3faad7acc5b8+incompatible/docs/reference/commandline/dockerd.md (about)

     1  ---
     2  title: "dockerd"
     3  description: "The daemon command description and usage"
     4  keywords: "container, daemon, runtime"
     5  redirect_from:
     6  - /engine/reference/commandline/daemon/
     7  ---
     8  
     9  <!-- This file is maintained within the docker/cli GitHub
    10       repository at https://github.com/docker/cli/. Make all
    11       pull requests against that repo. If you see this file in
    12       another repository, consider it read-only there, as it will
    13       periodically be overwritten by the definitive file. Pull
    14       requests which include edits to this file in other repositories
    15       will be rejected.
    16  -->
    17  
    18  # daemon
    19  
    20  ```markdown
    21  Usage: dockerd COMMAND
    22  
    23  A self-sufficient runtime for containers.
    24  
    25  Options:
    26        --add-runtime runtime                   Register an additional OCI compatible runtime (default [])
    27        --allow-nondistributable-artifacts list Allow push of nondistributable artifacts to registry
    28        --api-cors-header string                Set CORS headers in the Engine API
    29        --authorization-plugin list             Authorization plugins to load
    30        --bip string                            Specify network bridge IP
    31    -b, --bridge string                         Attach containers to a network bridge
    32        --cgroup-parent string                  Set parent cgroup for all containers
    33        --config-file string                    Daemon configuration file (default "/etc/docker/daemon.json")
    34        --containerd string                     containerd grpc address
    35        --containerd-namespace string           Containerd namespace to use (default "moby")
    36        --containerd-plugins-namespace string   Containerd namespace to use for plugins (default "plugins.moby")
    37        --cpu-rt-period int                     Limit the CPU real-time period in microseconds for the
    38                                                parent cgroup for all containers
    39        --cpu-rt-runtime int                    Limit the CPU real-time runtime in microseconds for the
    40                                                parent cgroup for all containers
    41        --cri-containerd                        start containerd with cri
    42        --data-root string                      Root directory of persistent Docker state (default "/var/lib/docker")
    43    -D, --debug                                 Enable debug mode
    44        --default-address-pool pool-options     Default address pools for node specific local networks
    45        --default-cgroupns-mode string          Default mode for containers cgroup namespace ("host" | "private") (default "host")
    46        --default-gateway ip                    Container default gateway IPv4 address
    47        --default-gateway-v6 ip                 Container default gateway IPv6 address
    48        --default-ipc-mode string               Default mode for containers ipc ("shareable" | "private") (default "private")
    49        --default-runtime string                Default OCI runtime for containers (default "runc")
    50        --default-shm-size bytes                Default shm size for containers (default 64MiB)
    51        --default-ulimit ulimit                 Default ulimits for containers (default [])
    52        --dns list                              DNS server to use
    53        --dns-opt list                          DNS options to use
    54        --dns-search list                       DNS search domains to use
    55        --exec-opt list                         Runtime execution options
    56        --exec-root string                      Root directory for execution state files (default "/var/run/docker")
    57        --experimental                          Enable experimental features
    58        --fixed-cidr string                     IPv4 subnet for fixed IPs
    59        --fixed-cidr-v6 string                  IPv6 subnet for fixed IPs
    60    -G, --group string                          Group for the unix socket (default "docker")
    61        --help                                  Print usage
    62    -H, --host list                             Daemon socket(s) to connect to
    63        --host-gateway-ip ip                    IP address that the special 'host-gateway' string in --add-host resolves to.
    64                                                Defaults to the IP address of the default bridge
    65        --icc                                   Enable inter-container communication (default true)
    66        --init                                  Run an init in the container to forward signals and reap processes
    67        --init-path string                      Path to the docker-init binary
    68        --insecure-registry list                Enable insecure registry communication
    69        --ip ip                                 Default IP when binding container ports (default 0.0.0.0)
    70        --ip-forward                            Enable net.ipv4.ip_forward (default true)
    71        --ip-masq                               Enable IP masquerading (default true)
    72        --iptables                              Enable addition of iptables rules (default true)
    73        --ipv6                                  Enable IPv6 networking
    74        --label list                            Set key=value labels to the daemon
    75        --live-restore                          Enable live restore of docker when containers are still running
    76        --log-driver string                     Default driver for container logs (default "json-file")
    77    -l, --log-level string                      Set the logging level ("debug"|"info"|"warn"|"error"|"fatal") (default "info")
    78        --log-opt map                           Default log driver options for containers (default map[])
    79        --max-concurrent-downloads int          Set the max concurrent downloads for each pull (default 3)
    80        --max-concurrent-uploads int            Set the max concurrent uploads for each push (default 5)
    81        --max-download-attempts int             Set the max download attempts for each pull (default 5)
    82        --metrics-addr string                   Set default address and port to serve the metrics api on
    83        --mtu int                               Set the containers network MTU
    84        --network-control-plane-mtu int         Network Control plane MTU (default 1500)
    85        --no-new-privileges                     Set no-new-privileges by default for new containers
    86        --node-generic-resource list            Advertise user-defined resource
    87        --oom-score-adjust int                  Set the oom_score_adj for the daemon (default -500)
    88    -p, --pidfile string                        Path to use for daemon PID file (default "/var/run/docker.pid")
    89        --raw-logs                              Full timestamps without ANSI coloring
    90        --registry-mirror list                  Preferred Docker registry mirror
    91        --rootless                              Enable rootless mode; typically used with RootlessKit
    92        --seccomp-profile string                Path to seccomp profile
    93        --selinux-enabled                       Enable selinux support
    94        --shutdown-timeout int                  Set the default shutdown timeout (default 15)
    95    -s, --storage-driver string                 Storage driver to use
    96        --storage-opt list                      Storage driver options
    97        --swarm-default-advertise-addr string   Set default address or interface for swarm advertised address
    98        --tls                                   Use TLS; implied by --tlsverify
    99        --tlscacert string                      Trust certs signed only by this CA (default "~/.docker/ca.pem")
   100        --tlscert string                        Path to TLS certificate file (default "~/.docker/cert.pem")
   101        --tlskey string                         Path to TLS key file (default "~/.docker/key.pem")
   102        --tlsverify                             Use TLS and verify the remote
   103        --userland-proxy                        Use userland proxy for loopback traffic (default true)
   104        --userland-proxy-path string            Path to the userland proxy binary
   105        --userns-remap string                   User/Group setting for user namespaces
   106    -v, --version                               Print version information and quit
   107  ```
   108  
   109  Options with [] may be specified multiple times.
   110  
   111  ## Description
   112  
   113  `dockerd` is the persistent process that manages containers. Docker
   114  uses different binaries for the daemon and client. To run the daemon you
   115  type `dockerd`.
   116  
   117  To run the daemon with debug output, use `dockerd -D` or add `"debug": true` to
   118  the `daemon.json` file.
   119  
   120  > **Enabling experimental features**
   121  > 
   122  > Enable experimental features by starting `dockerd` with the `--experimental`
   123  > flag or adding `"experimental": true` to the `daemon.json` file.
   124  
   125  ### Environment variables
   126  
   127  For easy reference, the following list of environment variables are supported
   128  by the `dockerd` command line:
   129  
   130  * `DOCKER_DRIVER` The graph driver to use.
   131  * `DOCKER_NOWARN_KERNEL_VERSION` Prevent warnings that your Linux kernel is
   132    unsuitable for Docker.
   133  * `DOCKER_RAMDISK` If set this will disable 'pivot_root'.
   134  * `DOCKER_TMPDIR` Location for temporary Docker files.
   135  * `MOBY_DISABLE_PIGZ` Do not use [`unpigz`](https://linux.die.net/man/1/pigz) to
   136    decompress layers in parallel when pulling images, even if it is installed.
   137  
   138  ## Examples
   139  
   140  ### Daemon socket option
   141  
   142  The Docker daemon can listen for [Docker Engine API](https://docs.docker.com/engine/api/)
   143  requests via three different types of Socket: `unix`, `tcp`, and `fd`.
   144  
   145  By default, a `unix` domain socket (or IPC socket) is created at
   146  `/var/run/docker.sock`, requiring either `root` permission, or `docker` group
   147  membership.
   148  
   149  If you need to access the Docker daemon remotely, you need to enable the `tcp`
   150  Socket. Beware that the default setup provides un-encrypted and
   151  un-authenticated direct access to the Docker daemon - and should be secured
   152  either using the [built in HTTPS encrypted socket](https://docs.docker.com/engine/security/https/), or by
   153  putting a secure web proxy in front of it. You can listen on port `2375` on all
   154  network interfaces with `-H tcp://0.0.0.0:2375`, or on a particular network
   155  interface using its IP address: `-H tcp://192.168.59.103:2375`. It is
   156  conventional to use port `2375` for un-encrypted, and port `2376` for encrypted
   157  communication with the daemon.
   158  
   159  > **Note**
   160  >
   161  > If you're using an HTTPS encrypted socket, keep in mind that only
   162  > TLS1.0 and greater are supported. Protocols SSLv3 and under are not
   163  > supported anymore for security reasons.
   164  
   165  On Systemd based systems, you can communicate with the daemon via
   166  [Systemd socket activation](http://0pointer.de/blog/projects/socket-activation.html),
   167  use `dockerd -H fd://`. Using `fd://` will work perfectly for most setups but
   168  you can also specify individual sockets: `dockerd -H fd://3`. If the
   169  specified socket activated files aren't found, then Docker will exit. You can
   170  find examples of using Systemd socket activation with Docker and Systemd in the
   171  [Docker source tree](https://github.com/docker/docker/tree/master/contrib/init/systemd/).
   172  
   173  You can configure the Docker daemon to listen to multiple sockets at the same
   174  time using multiple `-H` options:
   175  
   176  ```bash
   177  # listen using the default unix socket, and on 2 specific IP addresses on this host.
   178  
   179  $ sudo dockerd -H unix:///var/run/docker.sock -H tcp://192.168.59.106 -H tcp://10.10.10.2
   180  ```
   181  
   182  The Docker client will honor the `DOCKER_HOST` environment variable to set the
   183  `-H` flag for the client. Use **one** of the following commands:
   184  
   185  ```bash
   186  $ docker -H tcp://0.0.0.0:2375 ps
   187  ```
   188  
   189  ```bash
   190  $ export DOCKER_HOST="tcp://0.0.0.0:2375"
   191  
   192  $ docker ps
   193  ```
   194  
   195  Setting the `DOCKER_TLS_VERIFY` environment variable to any value other than
   196  the empty string is equivalent to setting the `--tlsverify` flag. The following
   197  are equivalent:
   198  
   199  ```bash
   200  $ docker --tlsverify ps
   201  # or
   202  $ export DOCKER_TLS_VERIFY=1
   203  $ docker ps
   204  ```
   205  
   206  The Docker client will honor the `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY`
   207  environment variables (or the lowercase versions thereof). `HTTPS_PROXY` takes
   208  precedence over `HTTP_PROXY`.
   209  
   210  The Docker client supports connecting to a remote daemon via SSH:
   211  
   212  ```
   213  $ docker -H ssh://me@example.com:22 ps
   214  $ docker -H ssh://me@example.com ps
   215  $ docker -H ssh://example.com ps
   216  ```
   217  
   218  To use SSH connection, you need to set up `ssh` so that it can reach the
   219  remote host with public key authentication. Password authentication is not
   220  supported. If your key is protected with passphrase, you need to set up
   221  `ssh-agent`.
   222  
   223  #### Bind Docker to another host/port or a Unix socket
   224  
   225  > **Warning**
   226  >
   227  > Changing the default `docker` daemon binding to a
   228  > TCP port or Unix *docker* user group will increase your security risks
   229  > by allowing non-root users to gain *root* access on the host. Make sure
   230  > you control access to `docker`. If you are binding
   231  > to a TCP port, anyone with access to that port has full Docker access;
   232  > so it is not advisable on an open network.
   233  {: .warning :}
   234  
   235  With `-H` it is possible to make the Docker daemon to listen on a
   236  specific IP and port. By default, it will listen on
   237  `unix:///var/run/docker.sock` to allow only local connections by the
   238  *root* user. You *could* set it to `0.0.0.0:2375` or a specific host IP
   239  to give access to everybody, but that is **not recommended** because
   240  then it is trivial for someone to gain root access to the host where the
   241  daemon is running.
   242  
   243  Similarly, the Docker client can use `-H` to connect to a custom port.
   244  The Docker client will default to connecting to `unix:///var/run/docker.sock`
   245  on Linux, and `tcp://127.0.0.1:2376` on Windows.
   246  
   247  `-H` accepts host and port assignment in the following format:
   248  
   249      tcp://[host]:[port][path] or unix://path
   250  
   251  For example:
   252  
   253  -   `tcp://` -> TCP connection to `127.0.0.1` on either port `2376` when TLS encryption
   254      is on, or port `2375` when communication is in plain text.
   255  -   `tcp://host:2375` -> TCP connection on
   256      host:2375
   257  -   `tcp://host:2375/path` -> TCP connection on
   258      host:2375 and prepend path to all requests
   259  -   `unix://path/to/socket` -> Unix socket located
   260      at `path/to/socket`
   261  
   262  `-H`, when empty, will default to the same value as
   263  when no `-H` was passed in.
   264  
   265  `-H` also accepts short form for TCP bindings: `host:` or `host:port` or `:port`
   266  
   267  Run Docker in daemon mode:
   268  
   269  ```bash
   270  $ sudo <path to>/dockerd -H 0.0.0.0:5555 &
   271  ```
   272  
   273  Download an `ubuntu` image:
   274  
   275  ```bash
   276  $ docker -H :5555 pull ubuntu
   277  ```
   278  
   279  You can use multiple `-H`, for example, if you want to listen on both
   280  TCP and a Unix socket
   281  
   282  ```bash
   283  # Run docker in daemon mode
   284  $ sudo <path to>/dockerd -H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock &
   285  # Download an ubuntu image, use default Unix socket
   286  $ docker pull ubuntu
   287  # OR use the TCP port
   288  $ docker -H tcp://127.0.0.1:2375 pull ubuntu
   289  ```
   290  
   291  ### Daemon storage-driver
   292  
   293  On Linux, the Docker daemon has support for several different image layer storage
   294  drivers: `aufs`, `devicemapper`, `btrfs`, `zfs`, `overlay`, `overlay2`, and `fuse-overlayfs`.
   295  
   296  The `aufs` driver is the oldest, but is based on a Linux kernel patch-set that
   297  is unlikely to be merged into the main kernel. These are also known to cause
   298  some serious kernel crashes. However `aufs` allows containers to share
   299  executable and shared library memory, so is a useful choice when running
   300  thousands of containers with the same program or libraries.
   301  
   302  The `devicemapper` driver uses thin provisioning and Copy on Write (CoW)
   303  snapshots. For each devicemapper graph location – typically
   304  `/var/lib/docker/devicemapper` – a thin pool is created based on two block
   305  devices, one for data and one for metadata. By default, these block devices
   306  are created automatically by using loopback mounts of automatically created
   307  sparse files. Refer to [Devicemapper options](#devicemapper-options) below
   308  for a way how to customize this setup.
   309  [~jpetazzo/Resizing Docker containers with the Device Mapper plugin](http://jpetazzo.github.io/2014/01/29/docker-device-mapper-resize/)
   310  article explains how to tune your existing setup without the use of options.
   311  
   312  The `btrfs` driver is very fast for `docker build` - but like `devicemapper`
   313  does not share executable memory between devices. Use
   314  `dockerd -s btrfs -g /mnt/btrfs_partition`.
   315  
   316  The `zfs` driver is probably not as fast as `btrfs` but has a longer track record
   317  on stability. Thanks to `Single Copy ARC` shared blocks between clones will be
   318  cached only once. Use `dockerd -s zfs`. To select a different zfs filesystem
   319  set `zfs.fsname` option as described in [ZFS options](#zfs-options).
   320  
   321  The `overlay` is a very fast union filesystem. It is now merged in the main
   322  Linux kernel as of [3.18.0](https://lkml.org/lkml/2014/10/26/137). `overlay`
   323  also supports page cache sharing, this means multiple containers accessing
   324  the same file can share a single page cache entry (or entries), it makes
   325  `overlay` as efficient with memory as `aufs` driver. Call `dockerd -s overlay`
   326  to use it.
   327  
   328  The `overlay2` uses the same fast union filesystem but takes advantage of
   329  [additional features](https://lkml.org/lkml/2015/2/11/106) added in Linux
   330  kernel 4.0 to avoid excessive inode consumption. Call `dockerd -s overlay2`
   331  to use it.
   332  
   333  > **Note**
   334  >
   335  > The `overlay` storage driver can cause excessive inode consumption (especially
   336  > as the number of images grows). We recommend using the `overlay2` storage
   337  > driver instead.
   338  
   339  
   340  > **Note**
   341  >
   342  > Both `overlay` and `overlay2` are currently unsupported on `btrfs`
   343  > or any Copy on Write filesystem and should only be used over `ext4` partitions.
   344  
   345  The `fuse-overlayfs` driver is similar to `overlay2` but works in userspace.
   346  The `fuse-overlayfs` driver is expected to be used for [Rootless mode](https://docs.docker.com/engine/security/rootless/).
   347  
   348  On Windows, the Docker daemon supports a single image layer storage driver
   349  depending on the image platform: `windowsfilter` for Windows images, and
   350  `lcow` for Linux containers on Windows.
   351  
   352  ### Options per storage driver
   353  
   354  Particular storage-driver can be configured with options specified with
   355  `--storage-opt` flags. Options for `devicemapper` are prefixed with `dm`,
   356  options for `zfs` start with `zfs`, options for `btrfs` start with `btrfs`
   357  and options for `lcow` start with `lcow`.
   358  
   359  #### Devicemapper options
   360  
   361  This is an example of the configuration file for devicemapper on Linux:
   362  
   363  ```json
   364  {
   365    "storage-driver": "devicemapper",
   366    "storage-opts": [
   367      "dm.thinpooldev=/dev/mapper/thin-pool",
   368      "dm.use_deferred_deletion=true",
   369      "dm.use_deferred_removal=true"
   370    ]
   371  }
   372  ```
   373  
   374  ##### `dm.thinpooldev`
   375  
   376  Specifies a custom block storage device to use for the thin pool.
   377  
   378  If using a block device for device mapper storage, it is best to use `lvm`
   379  to create and manage the thin-pool volume. This volume is then handed to Docker
   380  to exclusively create snapshot volumes needed for images and containers.
   381  
   382  Managing the thin-pool outside of Engine makes for the most feature-rich
   383  method of having Docker utilize device mapper thin provisioning as the
   384  backing storage for Docker containers. The highlights of the lvm-based
   385  thin-pool management feature include: automatic or interactive thin-pool
   386  resize support, dynamically changing thin-pool features, automatic thinp
   387  metadata checking when lvm activates the thin-pool, etc.
   388  
   389  As a fallback if no thin pool is provided, loopback files are
   390  created. Loopback is very slow, but can be used without any
   391  pre-configuration of storage. It is strongly recommended that you do
   392  not use loopback in production. Ensure your Engine daemon has a
   393  `--storage-opt dm.thinpooldev` argument provided.
   394  
   395  ###### Example:
   396  
   397  ```bash
   398  $ sudo dockerd --storage-opt dm.thinpooldev=/dev/mapper/thin-pool
   399  ```
   400  
   401  ##### `dm.directlvm_device`
   402  
   403  As an alternative to providing a thin pool as above, Docker can setup a block
   404  device for you.
   405  
   406  ###### Example:
   407  
   408  ```bash
   409  $ sudo dockerd --storage-opt dm.directlvm_device=/dev/xvdf
   410  ```
   411  
   412  ##### `dm.thinp_percent`
   413  
   414  Sets the percentage of passed in block device to use for storage.
   415  
   416  ###### Example:
   417  
   418  ```bash
   419  $ sudo dockerd --storage-opt dm.thinp_percent=95
   420  ```
   421  
   422  ##### `dm.thinp_metapercent`
   423  
   424  Sets the percentage of the passed in block device to use for metadata storage.
   425  
   426  ###### Example:
   427  
   428  ```bash
   429  $ sudo dockerd --storage-opt dm.thinp_metapercent=1
   430  ```
   431  
   432  ##### `dm.thinp_autoextend_threshold`
   433  
   434  Sets the value of the percentage of space used before `lvm` attempts to
   435  autoextend the available space [100 = disabled]
   436  
   437  ###### Example:
   438  
   439  ```bash
   440  $ sudo dockerd --storage-opt dm.thinp_autoextend_threshold=80
   441  ```
   442  
   443  ##### `dm.thinp_autoextend_percent`
   444  
   445  Sets the value percentage value to increase the thin pool by when `lvm`
   446  attempts to autoextend the available space [100 = disabled]
   447  
   448  ###### Example:
   449  
   450  ```bash
   451  $ sudo dockerd --storage-opt dm.thinp_autoextend_percent=20
   452  ```
   453  
   454  
   455  ##### `dm.basesize`
   456  
   457  Specifies the size to use when creating the base device, which limits the
   458  size of images and containers. The default value is 10G. Note, thin devices
   459  are inherently "sparse", so a 10G device which is mostly empty doesn't use
   460  10 GB of space on the pool. However, the filesystem will use more space for
   461  the empty case the larger the device is.
   462  
   463  The base device size can be increased at daemon restart which will allow
   464  all future images and containers (based on those new images) to be of the
   465  new base device size.
   466  
   467  ###### Examples
   468  
   469  ```bash
   470  $ sudo dockerd --storage-opt dm.basesize=50G
   471  ```
   472  
   473  This will increase the base device size to 50G. The Docker daemon will throw an
   474  error if existing base device size is larger than 50G. A user can use
   475  this option to expand the base device size however shrinking is not permitted.
   476  
   477  This value affects the system-wide "base" empty filesystem
   478  that may already be initialized and inherited by pulled images. Typically,
   479  a change to this value requires additional steps to take effect:
   480  
   481   ```bash
   482  $ sudo service docker stop
   483  
   484  $ sudo rm -rf /var/lib/docker
   485  
   486  $ sudo service docker start
   487  ```
   488  
   489  
   490  ##### `dm.loopdatasize`
   491  
   492  > **Note**
   493  >
   494  > This option configures devicemapper loopback, which should not
   495  > be used in production.
   496  
   497  Specifies the size to use when creating the loopback file for the
   498  "data" device which is used for the thin pool. The default size is
   499  100G. The file is sparse, so it will not initially take up this
   500  much space.
   501  
   502  ###### Example
   503  
   504  ```bash
   505  $ sudo dockerd --storage-opt dm.loopdatasize=200G
   506  ```
   507  
   508  ##### `dm.loopmetadatasize`
   509  
   510  > **Note**
   511  >
   512  > This option configures devicemapper loopback, which should not
   513  > be used in production.
   514  
   515  Specifies the size to use when creating the loopback file for the
   516  "metadata" device which is used for the thin pool. The default size
   517  is 2G. The file is sparse, so it will not initially take up
   518  this much space.
   519  
   520  ###### Example
   521  
   522  ```bash
   523  $ sudo dockerd --storage-opt dm.loopmetadatasize=4G
   524  ```
   525  
   526  ##### `dm.fs`
   527  
   528  Specifies the filesystem type to use for the base device. The supported
   529  options are "ext4" and "xfs". The default is "xfs"
   530  
   531  ###### Example
   532  
   533  ```bash
   534  $ sudo dockerd --storage-opt dm.fs=ext4
   535  ```
   536  
   537  ##### `dm.mkfsarg`
   538  
   539  Specifies extra mkfs arguments to be used when creating the base device.
   540  
   541  ###### Example
   542  
   543  ```bash
   544  $ sudo dockerd --storage-opt "dm.mkfsarg=-O ^has_journal"
   545  ```
   546  
   547  ##### `dm.mountopt`
   548  
   549  Specifies extra mount options used when mounting the thin devices.
   550  
   551  ###### Example
   552  
   553  ```bash
   554  $ sudo dockerd --storage-opt dm.mountopt=nodiscard
   555  ```
   556  
   557  ##### `dm.datadev`
   558  
   559  (Deprecated, use `dm.thinpooldev`)
   560  
   561  Specifies a custom blockdevice to use for data for the thin pool.
   562  
   563  If using a block device for device mapper storage, ideally both `datadev` and
   564  `metadatadev` should be specified to completely avoid using the loopback
   565  device.
   566  
   567  ###### Example
   568  
   569  ```bash
   570  $ sudo dockerd \
   571        --storage-opt dm.datadev=/dev/sdb1 \
   572        --storage-opt dm.metadatadev=/dev/sdc1
   573  ```
   574  
   575  ##### `dm.metadatadev`
   576  
   577  (Deprecated, use `dm.thinpooldev`)
   578  
   579  Specifies a custom blockdevice to use for metadata for the thin pool.
   580  
   581  For best performance the metadata should be on a different spindle than the
   582  data, or even better on an SSD.
   583  
   584  If setting up a new metadata pool it is required to be valid. This can be
   585  achieved by zeroing the first 4k to indicate empty metadata, like this:
   586  
   587  ```bash
   588  $ dd if=/dev/zero of=$metadata_dev bs=4096 count=1
   589  ```
   590  
   591  ###### Example
   592  
   593  ```bash
   594  $ sudo dockerd \
   595        --storage-opt dm.datadev=/dev/sdb1 \
   596        --storage-opt dm.metadatadev=/dev/sdc1
   597  ```
   598  
   599  ##### `dm.blocksize`
   600  
   601  Specifies a custom blocksize to use for the thin pool. The default
   602  blocksize is 64K.
   603  
   604  ###### Example
   605  
   606  ```bash
   607  $ sudo dockerd --storage-opt dm.blocksize=512K
   608  ```
   609  
   610  ##### `dm.blkdiscard`
   611  
   612  Enables or disables the use of `blkdiscard` when removing devicemapper
   613  devices. This is enabled by default (only) if using loopback devices and is
   614  required to resparsify the loopback file on image/container removal.
   615  
   616  Disabling this on loopback can lead to *much* faster container removal
   617  times, but will make the space used in `/var/lib/docker` directory not be
   618  returned to the system for other use when containers are removed.
   619  
   620  ###### Examples
   621  
   622  ```bash
   623  $ sudo dockerd --storage-opt dm.blkdiscard=false
   624  ```
   625  
   626  ##### `dm.override_udev_sync_check`
   627  
   628  Overrides the `udev` synchronization checks between `devicemapper` and `udev`.
   629  `udev` is the device manager for the Linux kernel.
   630  
   631  To view the `udev` sync support of a Docker daemon that is using the
   632  `devicemapper` driver, run:
   633  
   634  ```bash
   635  $ docker info
   636  [...]
   637  Udev Sync Supported: true
   638  [...]
   639  ```
   640  
   641  When `udev` sync support is `true`, then `devicemapper` and udev can
   642  coordinate the activation and deactivation of devices for containers.
   643  
   644  When `udev` sync support is `false`, a race condition occurs between
   645  the`devicemapper` and `udev` during create and cleanup. The race condition
   646  results in errors and failures. (For information on these failures, see
   647  [docker#4036](https://github.com/docker/docker/issues/4036))
   648  
   649  To allow the `docker` daemon to start, regardless of `udev` sync not being
   650  supported, set `dm.override_udev_sync_check` to true:
   651  
   652  ```bash
   653  $ sudo dockerd --storage-opt dm.override_udev_sync_check=true
   654  ```
   655  
   656  When this value is `true`, the  `devicemapper` continues and simply warns
   657  you the errors are happening.
   658  
   659  > **Note**
   660  >
   661  > The ideal is to pursue a `docker` daemon and environment that does
   662  > support synchronizing with `udev`. For further discussion on this
   663  > topic, see [docker#4036](https://github.com/docker/docker/issues/4036).
   664  > Otherwise, set this flag for migrating existing Docker daemons to
   665  > a daemon with a supported environment.
   666  
   667  ##### `dm.use_deferred_removal`
   668  
   669  Enables use of deferred device removal if `libdm` and the kernel driver
   670  support the mechanism.
   671  
   672  Deferred device removal means that if device is busy when devices are
   673  being removed/deactivated, then a deferred removal is scheduled on
   674  device. And devices automatically go away when last user of the device
   675  exits.
   676  
   677  For example, when a container exits, its associated thin device is removed.
   678  If that device has leaked into some other mount namespace and can't be
   679  removed, the container exit still succeeds and this option causes the
   680  system to schedule the device for deferred removal. It does not wait in a
   681  loop trying to remove a busy device.
   682  
   683  ###### Example
   684  
   685  ```bash
   686  $ sudo dockerd --storage-opt dm.use_deferred_removal=true
   687  ```
   688  
   689  ##### `dm.use_deferred_deletion`
   690  
   691  Enables use of deferred device deletion for thin pool devices. By default,
   692  thin pool device deletion is synchronous. Before a container is deleted,
   693  the Docker daemon removes any associated devices. If the storage driver
   694  can not remove a device, the container deletion fails and daemon returns.
   695  
   696  ```console
   697  Error deleting container: Error response from daemon: Cannot destroy container
   698  ```
   699  
   700  To avoid this failure, enable both deferred device deletion and deferred
   701  device removal on the daemon.
   702  
   703  ```bash
   704  $ sudo dockerd \
   705        --storage-opt dm.use_deferred_deletion=true \
   706        --storage-opt dm.use_deferred_removal=true
   707  ```
   708  
   709  With these two options enabled, if a device is busy when the driver is
   710  deleting a container, the driver marks the device as deleted. Later, when
   711  the device isn't in use, the driver deletes it.
   712  
   713  In general it should be safe to enable this option by default. It will help
   714  when unintentional leaking of mount point happens across multiple mount
   715  namespaces.
   716  
   717  ##### `dm.min_free_space`
   718  
   719  Specifies the min free space percent in a thin pool require for new device
   720  creation to succeed. This check applies to both free data space as well
   721  as free metadata space. Valid values are from 0% - 99%. Value 0% disables
   722  free space checking logic. If user does not specify a value for this option,
   723  the Engine uses a default value of 10%.
   724  
   725  Whenever a new a thin pool device is created (during `docker pull` or during
   726  container creation), the Engine checks if the minimum free space is
   727  available. If sufficient space is unavailable, then device creation fails
   728  and any relevant `docker` operation fails.
   729  
   730  To recover from this error, you must create more free space in the thin pool
   731  to recover from the error. You can create free space by deleting some images
   732  and containers from the thin pool. You can also add more storage to the thin
   733  pool.
   734  
   735  To add more space to a LVM (logical volume management) thin pool, just add
   736  more storage to the volume group container thin pool; this should automatically
   737  resolve any errors. If your configuration uses loop devices, then stop the
   738  Engine daemon, grow the size of loop files and restart the daemon to resolve
   739  the issue.
   740  
   741  ###### Example
   742  
   743  ```bash
   744  $ sudo dockerd --storage-opt dm.min_free_space=10%
   745  ```
   746  
   747  ##### `dm.xfs_nospace_max_retries`
   748  
   749  Specifies the maximum number of retries XFS should attempt to complete
   750  IO when ENOSPC (no space) error is returned by underlying storage device.
   751  
   752  By default XFS retries infinitely for IO to finish and this can result
   753  in unkillable process. To change this behavior one can set
   754  xfs_nospace_max_retries to say 0 and XFS will not retry IO after getting
   755  ENOSPC and will shutdown filesystem.
   756  
   757  ###### Example
   758  
   759  ```bash
   760  $ sudo dockerd --storage-opt dm.xfs_nospace_max_retries=0
   761  ```
   762  
   763  ##### `dm.libdm_log_level`
   764  
   765  Specifies the maxmimum `libdm` log level that will be forwarded to the
   766  `dockerd` log (as specified by `--log-level`). This option is primarily
   767  intended for debugging problems involving `libdm`. Using values other than the
   768  defaults may cause false-positive warnings to be logged.
   769  
   770  Values specified must fall within the range of valid `libdm` log levels. At the
   771  time of writing, the following is the list of `libdm` log levels as well as
   772  their corresponding levels when output by `dockerd`.
   773  
   774  | `libdm` Level | Value | `--log-level` |
   775  | ------------- | -----:| ------------- |
   776  | `_LOG_FATAL`  |     2 | error         |
   777  | `_LOG_ERR`    |     3 | error         |
   778  | `_LOG_WARN`   |     4 | warn          |
   779  | `_LOG_NOTICE` |     5 | info          |
   780  | `_LOG_INFO`   |     6 | info          |
   781  | `_LOG_DEBUG`  |     7 | debug         |
   782  
   783  ###### Example
   784  
   785  ```bash
   786  $ sudo dockerd \
   787        --log-level debug \
   788        --storage-opt dm.libdm_log_level=7
   789  ```
   790  
   791  #### ZFS options
   792  
   793  ##### `zfs.fsname`
   794  
   795  Set zfs filesystem under which docker will create its own datasets.
   796  By default docker will pick up the zfs filesystem where docker graph
   797  (`/var/lib/docker`) is located.
   798  
   799  ###### Example
   800  
   801  ```bash
   802  $ sudo dockerd -s zfs --storage-opt zfs.fsname=zroot/docker
   803  ```
   804  
   805  #### Btrfs options
   806  
   807  ##### `btrfs.min_space`
   808  
   809  Specifies the minimum size to use when creating the subvolume which is used
   810  for containers. If user uses disk quota for btrfs when creating or running
   811  a container with **--storage-opt size** option, docker should ensure the
   812  **size** cannot be smaller than **btrfs.min_space**.
   813  
   814  ###### Example
   815  
   816  ```bash
   817  $ sudo dockerd -s btrfs --storage-opt btrfs.min_space=10G
   818  ```
   819  
   820  #### Overlay2 options
   821  
   822  ##### `overlay2.override_kernel_check`
   823  
   824  Overrides the Linux kernel version check allowing overlay2. Support for
   825  specifying multiple lower directories needed by overlay2 was added to the
   826  Linux kernel in 4.0.0. However, some older kernel versions may be patched
   827  to add multiple lower directory support for OverlayFS. This option should
   828  only be used after verifying this support exists in the kernel. Applying
   829  this option on a kernel without this support will cause failures on mount.
   830  
   831  ##### `overlay2.size`
   832  
   833  Sets the default max size of the container. It is supported only when the
   834  backing fs is `xfs` and mounted with `pquota` mount option. Under these
   835  conditions the user can pass any size less then the backing fs size.
   836  
   837  ###### Example
   838  
   839  ```bash
   840  $ sudo dockerd -s overlay2 --storage-opt overlay2.size=1G
   841  ```
   842  
   843  
   844  #### Windowsfilter options
   845  
   846  ##### `size`
   847  
   848  Specifies the size to use when creating the sandbox which is used for containers.
   849  Defaults to 20G.
   850  
   851  ###### Example
   852  
   853  ```powershell
   854  C:\> dockerd --storage-opt size=40G
   855  ```
   856  
   857  #### LCOW (Linux Containers on Windows) options
   858  
   859  ##### `lcow.globalmode`
   860  
   861  Specifies whether the daemon instantiates utility VM instances as required
   862  (recommended and default if omitted), or uses single global utility VM (better
   863  performance, but has security implications and not recommended for production
   864  deployments).
   865  
   866  ###### Example
   867  
   868  ```powershell
   869  C:\> dockerd --storage-opt lcow.globalmode=false
   870  ```
   871  
   872  ##### `lcow.kirdpath`
   873  
   874  Specifies the folder path to the location of a pair of kernel and initrd files
   875  used for booting a utility VM. Defaults to `%ProgramFiles%\Linux Containers`.
   876  
   877  ###### Example
   878  
   879  ```powershell
   880  C:\> dockerd --storage-opt lcow.kirdpath=c:\path\to\files
   881  ```
   882  
   883  ##### `lcow.kernel`
   884  
   885  Specifies the filename of a kernel file located in the `lcow.kirdpath` path.
   886  Defaults to `bootx64.efi`.
   887  
   888  ###### Example
   889  
   890  ```powershell
   891  C:\> dockerd --storage-opt lcow.kernel=kernel.efi
   892  ```
   893  
   894  ##### `lcow.initrd`
   895  
   896  Specifies the filename of an initrd file located in the `lcow.kirdpath` path.
   897  Defaults to `initrd.img`.
   898  
   899  ###### Example
   900  
   901  ```powershell
   902  C:\> dockerd --storage-opt lcow.initrd=myinitrd.img
   903  ```
   904  
   905  ##### `lcow.bootparameters`
   906  
   907  Specifies additional boot parameters for booting utility VMs when in kernel/
   908  initrd mode. Ignored if the utility VM is booting from VHD. These settings
   909  are kernel specific.
   910  
   911  ###### Example
   912  
   913  ```powershell
   914  C:\> dockerd --storage-opt "lcow.bootparameters='option=value'"
   915  ```
   916  
   917  ##### `lcow.vhdx`
   918  
   919  Specifies a custom VHDX to boot a utility VM, as an alternate to kernel
   920  and initrd booting. Defaults to `uvm.vhdx` under `lcow.kirdpath`.
   921  
   922  ###### Example
   923  
   924  ```powershell
   925  C:\> dockerd --storage-opt lcow.vhdx=custom.vhdx
   926  ```
   927  
   928  ##### `lcow.timeout`
   929  
   930  Specifies the timeout for utility VM operations in seconds. Defaults
   931  to 300.
   932  
   933  ###### Example
   934  
   935  ```powershell
   936  C:\> dockerd --storage-opt lcow.timeout=240
   937  ```
   938  
   939  ##### `lcow.sandboxsize`
   940  
   941  Specifies the size in GB to use when creating the sandbox which is used for
   942  containers. Defaults to 20. Cannot be less than 20.
   943  
   944  ###### Example
   945  
   946  ```powershell
   947  C:\> dockerd --storage-opt lcow.sandboxsize=40
   948  ```
   949  
   950  ### Docker runtime execution options
   951  
   952  The Docker daemon relies on a
   953  [OCI](https://github.com/opencontainers/runtime-spec) compliant runtime
   954  (invoked via the `containerd` daemon) as its interface to the Linux
   955  kernel `namespaces`, `cgroups`, and `SELinux`.
   956  
   957  By default, the Docker daemon automatically starts `containerd`. If you want to
   958  control `containerd` startup, manually start `containerd` and pass the path to
   959  the `containerd` socket using the `--containerd` flag. For example:
   960  
   961  ```bash
   962  $ sudo dockerd --containerd /var/run/dev/docker-containerd.sock
   963  ```
   964  
   965  Runtimes can be registered with the daemon either via the
   966  configuration file or using the `--add-runtime` command line argument.
   967  
   968  The following is an example adding 2 runtimes via the configuration:
   969  
   970  ```json
   971  {
   972    "default-runtime": "runc",
   973    "runtimes": {
   974      "custom": {
   975        "path": "/usr/local/bin/my-runc-replacement",
   976        "runtimeArgs": [
   977          "--debug"
   978        ]
   979      },
   980      "runc": {
   981        "path": "runc"
   982      }
   983    }
   984  }
   985  ```
   986  
   987  This is the same example via the command line:
   988  
   989  ```bash
   990  $ sudo dockerd --add-runtime runc=runc --add-runtime custom=/usr/local/bin/my-runc-replacement
   991  ```
   992  
   993  > **Note**
   994  >
   995  > Defining runtime arguments via the command line is not supported.
   996  
   997  #### Options for the runtime
   998  
   999  You can configure the runtime using options specified
  1000  with the `--exec-opt` flag. All the flag's options have the `native` prefix. A
  1001  single `native.cgroupdriver` option is available.
  1002  
  1003  The `native.cgroupdriver` option specifies the management of the container's
  1004  cgroups. You can only specify `cgroupfs` or `systemd`. If you specify
  1005  `systemd` and it is not available, the system errors out. If you omit the
  1006  `native.cgroupdriver` option,` cgroupfs` is used on cgroup v1 hosts, `systemd`
  1007  is used on cgroup v2 hosts with systemd available.
  1008  
  1009  This example sets the `cgroupdriver` to `systemd`:
  1010  
  1011  ```bash
  1012  $ sudo dockerd --exec-opt native.cgroupdriver=systemd
  1013  ```
  1014  
  1015  Setting this option applies to all containers the daemon launches.
  1016  
  1017  Also Windows Container makes use of `--exec-opt` for special purpose. Docker user
  1018  can specify default container isolation technology with this, for example:
  1019  
  1020  ```console
  1021  > dockerd --exec-opt isolation=hyperv
  1022  ```
  1023  
  1024  Will make `hyperv` the default isolation technology on Windows. If no isolation
  1025  value is specified on daemon start, on Windows client, the default is
  1026  `hyperv`, and on Windows server, the default is `process`.
  1027  
  1028  ### Daemon DNS options
  1029  
  1030  To set the DNS server for all Docker containers, use:
  1031  
  1032  ```bash
  1033  $ sudo dockerd --dns 8.8.8.8
  1034  ```
  1035  
  1036  To set the DNS search domain for all Docker containers, use:
  1037  
  1038  ```bash
  1039  $ sudo dockerd --dns-search example.com
  1040  ```
  1041  
  1042  ### Allow push of nondistributable artifacts
  1043  
  1044  Some images (e.g., Windows base images) contain artifacts whose distribution is
  1045  restricted by license. When these images are pushed to a registry, restricted
  1046  artifacts are not included.
  1047  
  1048  To override this behavior for specific registries, use the
  1049  `--allow-nondistributable-artifacts` option in one of the following forms:
  1050  
  1051  * `--allow-nondistributable-artifacts myregistry:5000` tells the Docker daemon
  1052    to push nondistributable artifacts to myregistry:5000.
  1053  * `--allow-nondistributable-artifacts 10.1.0.0/16` tells the Docker daemon to
  1054    push nondistributable artifacts to all registries whose resolved IP address
  1055    is within the subnet described by the CIDR syntax.
  1056  
  1057  This option can be used multiple times.
  1058  
  1059  This option is useful when pushing images containing nondistributable artifacts
  1060  to a registry on an air-gapped network so hosts on that network can pull the
  1061  images without connecting to another server.
  1062  
  1063  > **Warning**: Nondistributable artifacts typically have restrictions on how
  1064  > and where they can be distributed and shared. Only use this feature to push
  1065  > artifacts to private registries and ensure that you are in compliance with
  1066  > any terms that cover redistributing nondistributable artifacts.
  1067  
  1068  ### Insecure registries
  1069  
  1070  Docker considers a private registry either secure or insecure. In the rest of
  1071  this section, *registry* is used for *private registry*, and `myregistry:5000`
  1072  is a placeholder example for a private registry.
  1073  
  1074  A secure registry uses TLS and a copy of its CA certificate is placed on the
  1075  Docker host at `/etc/docker/certs.d/myregistry:5000/ca.crt`. An insecure
  1076  registry is either not using TLS (i.e., listening on plain text HTTP), or is
  1077  using TLS with a CA certificate not known by the Docker daemon. The latter can
  1078  happen when the certificate was not found under
  1079  `/etc/docker/certs.d/myregistry:5000/`, or if the certificate verification
  1080  failed (i.e., wrong CA).
  1081  
  1082  By default, Docker assumes all, but local (see local registries below),
  1083  registries are secure. Communicating with an insecure registry is not possible
  1084  if Docker assumes that registry is secure. In order to communicate with an
  1085  insecure registry, the Docker daemon requires `--insecure-registry` in one of
  1086  the following two forms:
  1087  
  1088  * `--insecure-registry myregistry:5000` tells the Docker daemon that
  1089    myregistry:5000 should be considered insecure.
  1090  * `--insecure-registry 10.1.0.0/16` tells the Docker daemon that all registries
  1091    whose domain resolve to an IP address is part of the subnet described by the
  1092    CIDR syntax, should be considered insecure.
  1093  
  1094  The flag can be used multiple times to allow multiple registries to be marked
  1095  as insecure.
  1096  
  1097  If an insecure registry is not marked as insecure, `docker pull`,
  1098  `docker push`, and `docker search` will result in an error message prompting
  1099  the user to either secure or pass the `--insecure-registry` flag to the Docker
  1100  daemon as described above.
  1101  
  1102  Local registries, whose IP address falls in the 127.0.0.0/8 range, are
  1103  automatically marked as insecure as of Docker 1.3.2. It is not recommended to
  1104  rely on this, as it may change in the future.
  1105  
  1106  Enabling `--insecure-registry`, i.e., allowing un-encrypted and/or untrusted
  1107  communication, can be useful when running a local registry.  However,
  1108  because its use creates security vulnerabilities it should ONLY be enabled for
  1109  testing purposes.  For increased security, users should add their CA to their
  1110  system's list of trusted CAs instead of enabling `--insecure-registry`.
  1111  
  1112  #### Legacy Registries
  1113  
  1114  Operations against registries supporting only the legacy v1 protocol are no longer
  1115  supported. Specifically, the daemon will not attempt `push`, `pull` and `login`
  1116  to v1 registries. The exception to this is `search` which can still be performed
  1117  on v1 registries.
  1118  
  1119  
  1120  ### Running a Docker daemon behind an HTTPS_PROXY
  1121  
  1122  When running inside a LAN that uses an `HTTPS` proxy, the Docker Hub
  1123  certificates will be replaced by the proxy's certificates. These certificates
  1124  need to be added to your Docker host's configuration:
  1125  
  1126  1. Install the `ca-certificates` package for your distribution
  1127  2. Ask your network admin for the proxy's CA certificate and append them to
  1128     `/etc/pki/tls/certs/ca-bundle.crt`
  1129  3. Then start your Docker daemon with `HTTPS_PROXY=http://username:password@proxy:port/ dockerd`.
  1130     The `username:` and `password@` are optional - and are only needed if your
  1131     proxy is set up to require authentication.
  1132  
  1133  This will only add the proxy and authentication to the Docker daemon's requests -
  1134  your `docker build`s and running containers will need extra configuration to
  1135  use the proxy
  1136  
  1137  ### Default `ulimit` settings
  1138  
  1139  `--default-ulimit` allows you to set the default `ulimit` options to use for
  1140  all containers. It takes the same options as `--ulimit` for `docker run`. If
  1141  these defaults are not set, `ulimit` settings will be inherited, if not set on
  1142  `docker run`, from the Docker daemon. Any `--ulimit` options passed to
  1143  `docker run` will overwrite these defaults.
  1144  
  1145  Be careful setting `nproc` with the `ulimit` flag as `nproc` is designed by Linux to
  1146  set the maximum number of processes available to a user, not to a container. For details
  1147  please check the [run](run.md) reference.
  1148  
  1149  ### Node discovery
  1150  
  1151  The `--cluster-advertise` option specifies the `host:port` or `interface:port`
  1152  combination that this particular daemon instance should use when advertising
  1153  itself to the cluster. The daemon is reached by remote hosts through this value.
  1154  If you  specify an interface, make sure it includes the IP address of the actual
  1155  Docker host. For Engine installation created through `docker-machine`, the
  1156  interface is typically `eth1`.
  1157  
  1158  The daemon uses [libkv](https://github.com/docker/libkv/) to advertise
  1159  the node within the cluster. Some key-value backends support mutual
  1160  TLS. To configure the client TLS settings used by the daemon can be configured
  1161  using the `--cluster-store-opt` flag, specifying the paths to PEM encoded
  1162  files. For example:
  1163  
  1164  ```bash
  1165  $ sudo dockerd \
  1166      --cluster-advertise 192.168.1.2:2376 \
  1167      --cluster-store etcd://192.168.1.2:2379 \
  1168      --cluster-store-opt kv.cacertfile=/path/to/ca.pem \
  1169      --cluster-store-opt kv.certfile=/path/to/cert.pem \
  1170      --cluster-store-opt kv.keyfile=/path/to/key.pem
  1171  ```
  1172  
  1173  The currently supported cluster store options are:
  1174  
  1175  | Option                | Description                                                                                                                                                                                                                   |
  1176  |:----------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  1177  | `discovery.heartbeat` | Specifies the heartbeat timer in seconds which is used by the daemon as a `keepalive` mechanism to make sure discovery module treats the node as alive in the cluster. If not configured, the default value is 20 seconds.    |
  1178  | `discovery.ttl`       | Specifies the TTL (time-to-live) in seconds which is used by the discovery module to timeout a node if a valid heartbeat is not received within the configured ttl value. If not configured, the default value is 60 seconds. |
  1179  | `kv.cacertfile`       | Specifies the path to a local file with PEM encoded CA certificates to trust.                                                                                                                                                 |
  1180  | `kv.certfile`         | Specifies the path to a local file with a PEM encoded certificate. This certificate is used as the client cert for communication with the Key/Value store.                                                                    |
  1181  | `kv.keyfile`          | Specifies the path to a local file with a PEM encoded private key. This private key is used as the client key for communication with the Key/Value store.                                                                     |
  1182  | `kv.path`             | Specifies the path in the Key/Value store. If not configured, the default value is 'docker/nodes'.                                                                                                                            |
  1183  
  1184  ### Access authorization
  1185  
  1186  Docker's access authorization can be extended by authorization plugins that your
  1187  organization can purchase or build themselves. You can install one or more
  1188  authorization plugins when you start the Docker `daemon` using the
  1189  `--authorization-plugin=PLUGIN_ID` option.
  1190  
  1191  ```bash
  1192  $ sudo dockerd --authorization-plugin=plugin1 --authorization-plugin=plugin2,...
  1193  ```
  1194  
  1195  The `PLUGIN_ID` value is either the plugin's name or a path to its specification
  1196  file. The plugin's implementation determines whether you can specify a name or
  1197  path. Consult with your Docker administrator to get information about the
  1198  plugins available to you.
  1199  
  1200  Once a plugin is installed, requests made to the `daemon` through the
  1201  command line or Docker's Engine API are allowed or denied by the plugin.
  1202  If you have multiple plugins installed, each plugin, in order, must
  1203  allow the request for it to complete.
  1204  
  1205  For information about how to create an authorization plugin, see [authorization
  1206  plugin](../../extend/plugins_authorization.md) section in the Docker extend section of this documentation.
  1207  
  1208  
  1209  ### Daemon user namespace options
  1210  
  1211  The Linux kernel
  1212  [user namespace support](http://man7.org/linux/man-pages/man7/user_namespaces.7.html)
  1213  provides additional security by enabling a process, and therefore a container,
  1214  to have a unique range of user and group IDs which are outside the traditional
  1215  user and group range utilized by the host system. Potentially the most important
  1216  security improvement is that, by default, container processes running as the
  1217  `root` user will have expected administrative privilege (with some restrictions)
  1218  inside the container but will effectively be mapped to an unprivileged `uid` on
  1219  the host.
  1220  
  1221  For details about how to use this feature, as well as limitations, see
  1222  [Isolate containers with a user namespace](https://docs.docker.com/engine/security/userns-remap/).
  1223  
  1224  ### Miscellaneous options
  1225  
  1226  IP masquerading uses address translation to allow containers without a public
  1227  IP to talk to other machines on the Internet. This may interfere with some
  1228  network topologies and can be disabled with `--ip-masq=false`.
  1229  
  1230  Docker supports softlinks for the Docker data directory (`/var/lib/docker`) and
  1231  for `/var/lib/docker/tmp`. The `DOCKER_TMPDIR` and the data directory can be
  1232  set like this:
  1233  
  1234      DOCKER_TMPDIR=/mnt/disk2/tmp /usr/local/bin/dockerd -D -g /var/lib/docker -H unix:// > /var/lib/docker-machine/docker.log 2>&1
  1235      # or
  1236      export DOCKER_TMPDIR=/mnt/disk2/tmp
  1237      /usr/local/bin/dockerd -D -g /var/lib/docker -H unix:// > /var/lib/docker-machine/docker.log 2>&1
  1238  
  1239  #### Default cgroup parent
  1240  
  1241  The `--cgroup-parent` option allows you to set the default cgroup parent
  1242  to use for containers. If this option is not set, it defaults to `/docker` for
  1243  fs cgroup driver and `system.slice` for systemd cgroup driver.
  1244  
  1245  If the cgroup has a leading forward slash (`/`), the cgroup is created
  1246  under the root cgroup, otherwise the cgroup is created under the daemon
  1247  cgroup.
  1248  
  1249  Assuming the daemon is running in cgroup `daemoncgroup`,
  1250  `--cgroup-parent=/foobar` creates a cgroup in
  1251  `/sys/fs/cgroup/memory/foobar`, whereas using `--cgroup-parent=foobar`
  1252  creates the cgroup in `/sys/fs/cgroup/memory/daemoncgroup/foobar`
  1253  
  1254  The systemd cgroup driver has different rules for `--cgroup-parent`. Systemd
  1255  represents hierarchy by slice and the name of the slice encodes the location in
  1256  the tree. So `--cgroup-parent` for systemd cgroups should be a slice name. A
  1257  name can consist of a dash-separated series of names, which describes the path
  1258  to the slice from the root slice. For example, `--cgroup-parent=user-a-b.slice`
  1259  means the memory cgroup for the container is created in
  1260  `/sys/fs/cgroup/memory/user.slice/user-a.slice/user-a-b.slice/docker-<id>.scope`.
  1261  
  1262  This setting can also be set per container, using the `--cgroup-parent`
  1263  option on `docker create` and `docker run`, and takes precedence over
  1264  the `--cgroup-parent` option on the daemon.
  1265  
  1266  #### Daemon metrics
  1267  
  1268  The `--metrics-addr` option takes a tcp address to serve the metrics API.
  1269  This feature is still experimental, therefore, the daemon must be running in experimental
  1270  mode for this feature to work.
  1271  
  1272  To serve the metrics API on `localhost:9323` you would specify `--metrics-addr 127.0.0.1:9323`,
  1273  allowing you to make requests on the API at `127.0.0.1:9323/metrics` to receive metrics in the
  1274  [prometheus](https://prometheus.io/docs/instrumenting/exposition_formats/) format.
  1275  
  1276  Port `9323` is the [default port associated with Docker
  1277  metrics](https://github.com/prometheus/prometheus/wiki/Default-port-allocations)
  1278  to avoid collisions with other prometheus exporters and services.
  1279  
  1280  If you are running a prometheus server you can add this address to your scrape configs
  1281  to have prometheus collect metrics on Docker.  For more information
  1282  on prometheus refer to the [prometheus website](https://prometheus.io/).
  1283  
  1284  ```yaml
  1285  scrape_configs:
  1286    - job_name: 'docker'
  1287      static_configs:
  1288        - targets: ['127.0.0.1:9323']
  1289  ```
  1290  
  1291  Please note that this feature is still marked as experimental as metrics and metric
  1292  names could change while this feature is still in experimental.  Please provide
  1293  feedback on what you would like to see collected in the API.
  1294  
  1295  #### Node Generic Resources
  1296  
  1297  The `--node-generic-resources` option takes a list of key-value
  1298  pair (`key=value`) that allows you to advertise user defined resources
  1299  in a swarm cluster.
  1300  
  1301  The current expected use case is to advertise NVIDIA GPUs so that services
  1302  requesting `NVIDIA-GPU=[0-16]` can land on a node that has enough GPUs for
  1303  the task to run.
  1304  
  1305  Example of usage:
  1306  ```json
  1307  {
  1308    "node-generic-resources": [
  1309      "NVIDIA-GPU=UUID1",
  1310      "NVIDIA-GPU=UUID2"
  1311    ]
  1312  }
  1313  ```
  1314  
  1315  ### Daemon configuration file
  1316  
  1317  The `--config-file` option allows you to set any configuration option
  1318  for the daemon in a JSON format. This file uses the same flag names as keys,
  1319  except for flags that allow several entries, where it uses the plural
  1320  of the flag name, e.g., `labels` for the `label` flag.
  1321  
  1322  The options set in the configuration file must not conflict with options set
  1323  via flags. The docker daemon fails to start if an option is duplicated between
  1324  the file and the flags, regardless their value. We do this to avoid
  1325  silently ignore changes introduced in configuration reloads.
  1326  For example, the daemon fails to start if you set daemon labels
  1327  in the configuration file and also set daemon labels via the `--label` flag.
  1328  Options that are not present in the file are ignored when the daemon starts.
  1329  
  1330  ##### On Linux
  1331  
  1332  The default location of the configuration file on Linux is
  1333  `/etc/docker/daemon.json`. The `--config-file` flag can be used to specify a
  1334   non-default location.
  1335  
  1336  This is a full example of the allowed configuration options on Linux:
  1337  
  1338  ```json
  1339  {
  1340    "allow-nondistributable-artifacts": [],
  1341    "api-cors-header": "",
  1342    "authorization-plugins": [],
  1343    "bip": "",
  1344    "bridge": "",
  1345    "cgroup-parent": "",
  1346    "cluster-advertise": "",
  1347    "cluster-store": "",
  1348    "cluster-store-opts": {},
  1349    "containerd": "/run/containerd/containerd.sock",
  1350    "containerd-namespace": "docker",
  1351    "containerd-plugin-namespace": "docker-plugins",
  1352    "data-root": "",
  1353    "debug": true,
  1354    "default-address-pools": [
  1355      {
  1356        "base": "172.80.0.0/16",
  1357        "size": 24
  1358      },
  1359      {
  1360        "base": "172.90.0.0/16",
  1361        "size": 24
  1362      }
  1363    ],
  1364    "default-cgroupns-mode": "private",
  1365    "default-gateway": "",
  1366    "default-gateway-v6": "",
  1367    "default-runtime": "runc",
  1368    "default-shm-size": "64M",
  1369    "default-ulimits": {
  1370      "nofile": {
  1371        "Hard": 64000,
  1372        "Name": "nofile",
  1373        "Soft": 64000
  1374      }
  1375    },
  1376    "dns": [],
  1377    "dns-opts": [],
  1378    "dns-search": [],
  1379    "exec-opts": [],
  1380    "exec-root": "",
  1381    "experimental": false,
  1382    "features": {},
  1383    "fixed-cidr": "",
  1384    "fixed-cidr-v6": "",
  1385    "group": "",
  1386    "hosts": [],
  1387    "icc": false,
  1388    "init": false,
  1389    "init-path": "/usr/libexec/docker-init",
  1390    "insecure-registries": [],
  1391    "ip": "0.0.0.0",
  1392    "ip-forward": false,
  1393    "ip-masq": false,
  1394    "iptables": false,
  1395    "ipv6": false,
  1396    "labels": [],
  1397    "live-restore": true,
  1398    "log-driver": "json-file",
  1399    "log-level": "",
  1400    "log-opts": {
  1401      "env": "os,customer",
  1402      "labels": "somelabel",
  1403      "max-file": "5",
  1404      "max-size": "10m"
  1405    },
  1406    "max-concurrent-downloads": 3,
  1407    "max-concurrent-uploads": 5,
  1408    "max-download-attempts": 5,
  1409    "mtu": 0,
  1410    "no-new-privileges": false,
  1411    "node-generic-resources": [
  1412      "NVIDIA-GPU=UUID1",
  1413      "NVIDIA-GPU=UUID2"
  1414    ],
  1415    "oom-score-adjust": -500,
  1416    "pidfile": "",
  1417    "raw-logs": false,
  1418    "registry-mirrors": [],
  1419    "runtimes": {
  1420      "cc-runtime": {
  1421        "path": "/usr/bin/cc-runtime"
  1422      },
  1423      "custom": {
  1424        "path": "/usr/local/bin/my-runc-replacement",
  1425        "runtimeArgs": [
  1426          "--debug"
  1427        ]
  1428      }
  1429    },
  1430    "seccomp-profile": "",
  1431    "selinux-enabled": false,
  1432    "shutdown-timeout": 15,
  1433    "storage-driver": "",
  1434    "storage-opts": [],
  1435    "swarm-default-advertise-addr": "",
  1436    "tls": true,
  1437    "tlscacert": "",
  1438    "tlscert": "",
  1439    "tlskey": "",
  1440    "tlsverify": true,
  1441    "userland-proxy": false,
  1442    "userland-proxy-path": "/usr/libexec/docker-proxy",
  1443    "userns-remap": ""
  1444  }
  1445  ```
  1446  
  1447  > **Note:**
  1448  >
  1449  > You cannot set options in `daemon.json` that have already been set on
  1450  > daemon startup as a flag.
  1451  > On systems that use `systemd` to start the Docker daemon, `-H` is already set, so
  1452  > you cannot use the `hosts` key in `daemon.json` to add listening addresses.
  1453  > See https://docs.docker.com/engine/admin/systemd/#custom-docker-daemon-options for how
  1454  > to accomplish this task with a systemd drop-in file.
  1455  
  1456  ##### On Windows
  1457  
  1458  The default location of the configuration file on Windows is
  1459   `%programdata%\docker\config\daemon.json`. The `--config-file` flag can be
  1460   used to specify a non-default location.
  1461  
  1462  This is a full example of the allowed configuration options on Windows:
  1463  
  1464  ```json
  1465  {
  1466    "allow-nondistributable-artifacts": [],
  1467    "authorization-plugins": [],
  1468    "bridge": "",
  1469    "cluster-advertise": "",
  1470    "cluster-store": "",
  1471    "containerd": "\\\\.\\pipe\\containerd-containerd",
  1472    "containerd-namespace": "docker",
  1473    "containerd-plugin-namespace": "docker-plugins",
  1474    "data-root": "",
  1475    "debug": true,
  1476    "default-ulimits": {},
  1477    "dns": [],
  1478    "dns-opts": [],
  1479    "dns-search": [],
  1480    "exec-opts": [],
  1481    "experimental": false,
  1482    "features": {},
  1483    "fixed-cidr": "",
  1484    "group": "",
  1485    "hosts": [],
  1486    "insecure-registries": [],
  1487    "labels": [],
  1488    "log-driver": "",
  1489    "log-level": "",
  1490    "max-concurrent-downloads": 3,
  1491    "max-concurrent-uploads": 5,
  1492    "max-download-attempts": 5,
  1493    "mtu": 0,
  1494    "pidfile": "",
  1495    "raw-logs": false,
  1496    "registry-mirrors": [],
  1497    "shutdown-timeout": 15,
  1498    "storage-driver": "",
  1499    "storage-opts": [],
  1500    "swarm-default-advertise-addr": "",
  1501    "tlscacert": "",
  1502    "tlscert": "",
  1503    "tlskey": "",
  1504    "tlsverify": true
  1505  }
  1506  ```
  1507  
  1508  #### Feature options
  1509  The optional field `features` in `daemon.json` allows users to enable or disable specific
  1510  daemon features. For example, `{"features":{"buildkit": true}}` enables `buildkit` as the
  1511  default docker image builder.
  1512  
  1513  The list of currently supported feature options:
  1514  - `buildkit`: It enables `buildkit` as default builder when set to `true` or disables it by
  1515  `false`. Note that if this option is not explicitly set in the daemon config file, then it
  1516  is up to the cli to determine which builder to invoke.
  1517  
  1518  #### Configuration reload behavior
  1519  
  1520  Some options can be reconfigured when the daemon is running without requiring
  1521  to restart the process. We use the `SIGHUP` signal in Linux to reload, and a global event
  1522  in Windows with the key `Global\docker-daemon-config-$PID`. The options can
  1523  be modified in the configuration file but still will check for conflicts with
  1524  the provided flags. The daemon fails to reconfigure itself
  1525  if there are conflicts, but it won't stop execution.
  1526  
  1527  The list of currently supported options that can be reconfigured is this:
  1528  
  1529  - `debug`: it changes the daemon to debug mode when set to true.
  1530  - `cluster-store`: it reloads the discovery store with the new address.
  1531  - `cluster-store-opts`: it uses the new options to reload the discovery store.
  1532  - `cluster-advertise`: it modifies the address advertised after reloading.
  1533  - `labels`: it replaces the daemon labels with a new set of labels.
  1534  - `live-restore`: Enables [keeping containers alive during daemon downtime](https://docs.docker.com/config/containers/live-restore/).
  1535  - `max-concurrent-downloads`: it updates the max concurrent downloads for each pull.
  1536  - `max-concurrent-uploads`: it updates the max concurrent uploads for each push.
  1537  - `max-download-attempts`: it updates the max download attempts for each pull.
  1538  - `default-runtime`: it updates the runtime to be used if not is
  1539    specified at container creation. It defaults to "default" which is
  1540    the runtime shipped with the official docker packages.
  1541  - `runtimes`: it updates the list of available OCI runtimes that can
  1542    be used to run containers.
  1543  - `authorization-plugin`: it specifies the authorization plugins to use.
  1544  - `allow-nondistributable-artifacts`: Replaces the set of registries to which the daemon will push nondistributable artifacts with a new set of registries.
  1545  - `insecure-registries`: it replaces the daemon insecure registries with a new set of insecure registries. If some existing insecure registries in daemon's configuration are not in newly reloaded insecure resgitries, these existing ones will be removed from daemon's config.
  1546  - `registry-mirrors`: it replaces the daemon registry mirrors with a new set of registry mirrors. If some existing registry mirrors in daemon's configuration are not in newly reloaded registry mirrors, these existing ones will be removed from daemon's config.
  1547  - `shutdown-timeout`: it replaces the daemon's existing configuration timeout with a new timeout for shutting down all containers.
  1548  - `features`: it explicitly enables or disables specific features.
  1549  
  1550  Updating and reloading the cluster configurations such as `--cluster-store`,
  1551  `--cluster-advertise` and `--cluster-store-opts` will take effect only if
  1552  these configurations were not previously configured. If `--cluster-store`
  1553  has been provided in flags and `cluster-advertise` not, `cluster-advertise`
  1554  can be added in the configuration file without accompanied by `--cluster-store`.
  1555  Configuration reload will log a warning message if it detects a change in
  1556  previously configured cluster configurations.
  1557  
  1558  
  1559  ### Run multiple daemons
  1560  
  1561  > **Note:** Running multiple daemons on a single host is considered as "experimental". The user should be aware of
  1562  > unsolved problems. This solution may not work properly in some cases. Solutions are currently under development
  1563  > and will be delivered in the near future.
  1564  
  1565  This section describes how to run multiple Docker daemons on a single host. To
  1566  run multiple daemons, you must configure each daemon so that it does not
  1567  conflict with other daemons on the same host. You can set these options either
  1568  by providing them as flags, or by using a [daemon configuration file](#daemon-configuration-file).
  1569  
  1570  The following daemon options must be configured for each daemon:
  1571  
  1572  ```console
  1573  -b, --bridge=                          Attach containers to a network bridge
  1574  --exec-root=/var/run/docker            Root of the Docker execdriver
  1575  --data-root=/var/lib/docker            Root of persisted Docker data
  1576  -p, --pidfile=/var/run/docker.pid      Path to use for daemon PID file
  1577  -H, --host=[]                          Daemon socket(s) to connect to
  1578  --iptables=true                        Enable addition of iptables rules
  1579  --config-file=/etc/docker/daemon.json  Daemon configuration file
  1580  --tlscacert="~/.docker/ca.pem"         Trust certs signed only by this CA
  1581  --tlscert="~/.docker/cert.pem"         Path to TLS certificate file
  1582  --tlskey="~/.docker/key.pem"           Path to TLS key file
  1583  ```
  1584  
  1585  When your daemons use different values for these flags, you can run them on the same host without any problems.
  1586  It is very important to properly understand the meaning of those options and to use them correctly.
  1587  
  1588  - The `-b, --bridge=` flag is set to `docker0` as default bridge network. It is created automatically when you install Docker.
  1589  If you are not using the default, you must create and configure the bridge manually or just set it to 'none': `--bridge=none`
  1590  - `--exec-root` is the path where the container state is stored. The default value is `/var/run/docker`. Specify the path for
  1591  your running daemon here.
  1592  - `--data-root` is the path where persisted data such as images, volumes, and
  1593  cluster state are stored. The default value is `/var/lib/docker`. To avoid any
  1594  conflict with other daemons, set this parameter separately for each daemon.
  1595  - `-p, --pidfile=/var/run/docker.pid` is the path where the process ID of the daemon is stored. Specify the path for your
  1596  pid file here.
  1597  - `--host=[]` specifies where the Docker daemon will listen for client connections. If unspecified, it defaults to `/var/run/docker.sock`.
  1598  -  `--iptables=false` prevents the Docker daemon from adding iptables rules. If
  1599  multiple daemons manage iptables rules, they may overwrite rules set by another
  1600  daemon. Be aware that disabling this option requires you to manually add
  1601  iptables rules to expose container ports. If you prevent Docker from adding
  1602  iptables rules, Docker will also not add IP masquerading rules, even if you set
  1603  `--ip-masq` to `true`. Without IP masquerading rules, Docker containers will not be
  1604  able to connect to external hosts or the internet when using network other than
  1605  default bridge.
  1606  - `--config-file=/etc/docker/daemon.json` is the path where configuration file is stored. You can use it instead of
  1607  daemon flags. Specify the path for each daemon.
  1608  - `--tls*` Docker daemon supports `--tlsverify` mode that enforces encrypted and authenticated remote connections.
  1609  The `--tls*` options enable use of specific certificates for individual daemons.
  1610  
  1611  Example script for a separate “bootstrap” instance of the Docker daemon without network:
  1612  
  1613  ```bash
  1614  $ sudo dockerd \
  1615          -H unix:///var/run/docker-bootstrap.sock \
  1616          -p /var/run/docker-bootstrap.pid \
  1617          --iptables=false \
  1618          --ip-masq=false \
  1619          --bridge=none \
  1620          --data-root=/var/lib/docker-bootstrap \
  1621          --exec-root=/var/run/docker-bootstrap
  1622  ```