github.com/fabiokung/docker@v0.11.2-0.20170222101415-4534dcd49497/docs/reference/commandline/dockerd.md (about)

     1  ---
     2  title: "dockerd"
     3  aliases: ["/engine/reference/commandline/daemon/"]
     4  description: "The daemon command description and usage"
     5  keywords: "container, daemon, runtime"
     6  ---
     7  
     8  <!-- This file is maintained within the docker/docker Github
     9       repository at https://github.com/docker/docker/. Make all
    10       pull requests against that repo. If you see this file in
    11       another repository, consider it read-only there, as it will
    12       periodically be overwritten by the definitive file. Pull
    13       requests which include edits to this file in other repositories
    14       will be rejected.
    15  -->
    16  
    17  # daemon
    18  
    19  ```markdown
    20  Usage: dockerd [OPTIONS]
    21  
    22  A self-sufficient runtime for containers.
    23  
    24  Options:
    25        --add-runtime value                     Register an additional OCI compatible runtime (default [])
    26        --api-cors-header string                Set CORS headers in the Engine API
    27        --authorization-plugin value            Authorization plugins to load (default [])
    28        --bip string                            Specify network bridge IP
    29    -b, --bridge string                         Attach containers to a network bridge
    30        --cgroup-parent string                  Set parent cgroup for all containers
    31        --cluster-advertise string              Address or interface name to advertise
    32        --cluster-store string                  URL of the distributed storage backend
    33        --cluster-store-opt value               Set cluster store options (default map[])
    34        --config-file string                    Daemon configuration file (default "/etc/docker/daemon.json")
    35        --containerd string                     Path to containerd socket
    36    -D, --debug                                 Enable debug mode
    37        --default-gateway value                 Container default gateway IPv4 address
    38        --default-gateway-v6 value              Container default gateway IPv6 address
    39        --default-runtime string                Default OCI runtime for containers (default "runc")
    40        --default-shm-size bytes                Set the default shm size for containers (default 64 MiB)
    41        --default-ulimit value                  Default ulimits for containers (default [])
    42        --disable-legacy-registry               Disable contacting legacy registries
    43        --dns value                             DNS server to use (default [])
    44        --dns-opt value                         DNS options to use (default [])
    45        --dns-search value                      DNS search domains to use (default [])
    46        --exec-opt value                        Runtime execution options (default [])
    47        --exec-root string                      Root directory for execution state files (default "/var/run/docker")
    48        --experimental                          Enable experimental features
    49        --fixed-cidr string                     IPv4 subnet for fixed IPs
    50        --fixed-cidr-v6 string                  IPv6 subnet for fixed IPs
    51    -g, --graph string                          Root of the Docker runtime (default "/var/lib/docker")
    52    -G, --group string                          Group for the unix socket (default "docker")
    53        --help                                  Print usage
    54    -H, --host value                            Daemon socket(s) to connect to (default [])
    55        --icc                                   Enable inter-container communication (default true)
    56        --init                                  Run an init in the container to forward signals and reap processes
    57        --init-path string                      Path to the docker-init binary
    58        --insecure-registry value               Enable insecure registry communication (default [])
    59        --ip value                              Default IP when binding container ports (default 0.0.0.0)
    60        --ip-forward                            Enable net.ipv4.ip_forward (default true)
    61        --ip-masq                               Enable IP masquerading (default true)
    62        --iptables                              Enable addition of iptables rules (default true)
    63        --ipv6                                  Enable IPv6 networking
    64        --label value                           Set key=value labels to the daemon (default [])
    65        --live-restore                          Enable live restore of docker when containers are still running (Linux only)
    66        --log-driver string                     Default driver for container logs (default "json-file")
    67    -l, --log-level string                      Set the logging level ("debug"|"info"|"warn"|"error"|"fatal") (default "info")
    68        --log-opt value                         Default log driver options for containers (default map[])
    69        --max-concurrent-downloads int          Set the max concurrent downloads for each pull (default 3)
    70        --max-concurrent-uploads int            Set the max concurrent uploads for each push (default 5)
    71        --metrics-addr string                   Set address and port to serve the metrics api (default "")
    72        --mtu int                               Set the containers network MTU
    73        --no-new-privileges                     Disable container processes from gaining new privileges
    74        --oom-score-adjust int                  Set the oom_score_adj for the daemon (default -500)
    75    -p, --pidfile string                        Path to use for daemon PID file (default "/var/run/docker.pid")
    76        --raw-logs                              Full timestamps without ANSI coloring
    77        --registry-mirror value                 Preferred Docker registry mirror (default [])
    78        --seccomp-profile value                 Path to seccomp profile
    79        --selinux-enabled                       Enable selinux support
    80        --shutdown-timeout=15                   Set the shutdown timeout value in seconds
    81    -s, --storage-driver string                 Storage driver to use
    82        --storage-opt value                     Storage driver options (default [])
    83        --swarm-default-advertise-addr string   Set default address or interface for swarm advertised address
    84        --tls                                   Use TLS; implied by --tlsverify
    85        --tlscacert string                      Trust certs signed only by this CA (default "/root/.docker/ca.pem")
    86        --tlscert string                        Path to TLS certificate file (default "/root/.docker/cert.pem")
    87        --tlskey string                         Path to TLS key file (default "/root/.docker/key.pem")
    88        --tlsverify                             Use TLS and verify the remote
    89        --userland-proxy                        Use userland proxy for loopback traffic (default true)
    90        --userland-proxy-path string            Path to the userland proxy binary
    91        --userns-remap string                   User/Group setting for user namespaces
    92    -v, --version                               Print version information and quit
    93  ```
    94  
    95  Options with [] may be specified multiple times.
    96  
    97  ## Description
    98  
    99  `dockerd` is the persistent process that manages containers. Docker
   100  uses different binaries for the daemon and client. To run the daemon you
   101  type `dockerd`.
   102  
   103  To run the daemon with debug output, use `dockerd -D`.
   104  
   105  ## Examples
   106  
   107  ### Daemon socket option
   108  
   109  The Docker daemon can listen for [Docker Engine API](../api/)
   110  requests via three different types of Socket: `unix`, `tcp`, and `fd`.
   111  
   112  By default, a `unix` domain socket (or IPC socket) is created at
   113  `/var/run/docker.sock`, requiring either `root` permission, or `docker` group
   114  membership.
   115  
   116  If you need to access the Docker daemon remotely, you need to enable the `tcp`
   117  Socket. Beware that the default setup provides un-encrypted and
   118  un-authenticated direct access to the Docker daemon - and should be secured
   119  either using the [built in HTTPS encrypted socket](https://docs.docker.com/engine/security/https/), or by
   120  putting a secure web proxy in front of it. You can listen on port `2375` on all
   121  network interfaces with `-H tcp://0.0.0.0:2375`, or on a particular network
   122  interface using its IP address: `-H tcp://192.168.59.103:2375`. It is
   123  conventional to use port `2375` for un-encrypted, and port `2376` for encrypted
   124  communication with the daemon.
   125  
   126  > **Note**: If you're using an HTTPS encrypted socket, keep in mind that only
   127  > TLS1.0 and greater are supported. Protocols SSLv3 and under are not
   128  > supported anymore for security reasons.
   129  
   130  On Systemd based systems, you can communicate with the daemon via
   131  [Systemd socket activation](http://0pointer.de/blog/projects/socket-activation.html),
   132  use `dockerd -H fd://`. Using `fd://` will work perfectly for most setups but
   133  you can also specify individual sockets: `dockerd -H fd://3`. If the
   134  specified socket activated files aren't found, then Docker will exit. You can
   135  find examples of using Systemd socket activation with Docker and Systemd in the
   136  [Docker source tree](https://github.com/docker/docker/tree/master/contrib/init/systemd/).
   137  
   138  You can configure the Docker daemon to listen to multiple sockets at the same
   139  time using multiple `-H` options:
   140  
   141  ```bash
   142  # listen using the default unix socket, and on 2 specific IP addresses on this host.
   143  
   144  $ sudo dockerd -H unix:///var/run/docker.sock -H tcp://192.168.59.106 -H tcp://10.10.10.2
   145  ```
   146  
   147  The Docker client will honor the `DOCKER_HOST` environment variable to set the
   148  `-H` flag for the client. Use **one** of the following commands:
   149  
   150  ```bash
   151  $ docker -H tcp://0.0.0.0:2375 ps
   152  ```
   153  
   154  ```bash
   155  $ export DOCKER_HOST="tcp://0.0.0.0:2375"
   156  
   157  $ docker ps
   158  ```
   159  
   160  Setting the `DOCKER_TLS_VERIFY` environment variable to any value other than
   161  the empty string is equivalent to setting the `--tlsverify` flag. The following
   162  are equivalent:
   163  
   164  ```bash
   165  $ docker --tlsverify ps
   166  # or
   167  $ export DOCKER_TLS_VERIFY=1
   168  $ docker ps
   169  ```
   170  
   171  The Docker client will honor the `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY`
   172  environment variables (or the lowercase versions thereof). `HTTPS_PROXY` takes
   173  precedence over `HTTP_PROXY`.
   174  
   175  #### Bind Docker to another host/port or a Unix socket
   176  
   177  > **Warning**:
   178  > Changing the default `docker` daemon binding to a
   179  > TCP port or Unix *docker* user group will increase your security risks
   180  > by allowing non-root users to gain *root* access on the host. Make sure
   181  > you control access to `docker`. If you are binding
   182  > to a TCP port, anyone with access to that port has full Docker access;
   183  > so it is not advisable on an open network.
   184  
   185  With `-H` it is possible to make the Docker daemon to listen on a
   186  specific IP and port. By default, it will listen on
   187  `unix:///var/run/docker.sock` to allow only local connections by the
   188  *root* user. You *could* set it to `0.0.0.0:2375` or a specific host IP
   189  to give access to everybody, but that is **not recommended** because
   190  then it is trivial for someone to gain root access to the host where the
   191  daemon is running.
   192  
   193  Similarly, the Docker client can use `-H` to connect to a custom port.
   194  The Docker client will default to connecting to `unix:///var/run/docker.sock`
   195  on Linux, and `tcp://127.0.0.1:2376` on Windows.
   196  
   197  `-H` accepts host and port assignment in the following format:
   198  
   199      tcp://[host]:[port][path] or unix://path
   200  
   201  For example:
   202  
   203  -   `tcp://` -> TCP connection to `127.0.0.1` on either port `2376` when TLS encryption
   204      is on, or port `2375` when communication is in plain text.
   205  -   `tcp://host:2375` -> TCP connection on
   206      host:2375
   207  -   `tcp://host:2375/path` -> TCP connection on
   208      host:2375 and prepend path to all requests
   209  -   `unix://path/to/socket` -> Unix socket located
   210      at `path/to/socket`
   211  
   212  `-H`, when empty, will default to the same value as
   213  when no `-H` was passed in.
   214  
   215  `-H` also accepts short form for TCP bindings: `host:` or `host:port` or `:port`
   216  
   217  Run Docker in daemon mode:
   218  
   219  ```bash
   220  $ sudo <path to>/dockerd -H 0.0.0.0:5555 &
   221  ```
   222  
   223  Download an `ubuntu` image:
   224  
   225  ```bash
   226  $ docker -H :5555 pull ubuntu
   227  ```
   228  
   229  You can use multiple `-H`, for example, if you want to listen on both
   230  TCP and a Unix socket
   231  
   232  ```bash
   233  # Run docker in daemon mode
   234  $ sudo <path to>/dockerd -H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock &
   235  # Download an ubuntu image, use default Unix socket
   236  $ docker pull ubuntu
   237  # OR use the TCP port
   238  $ docker -H tcp://127.0.0.1:2375 pull ubuntu
   239  ```
   240  
   241  ### Daemon storage-driver
   242  
   243  The Docker daemon has support for several different image layer storage
   244  drivers: `aufs`, `devicemapper`, `btrfs`, `zfs`, `overlay` and `overlay2`.
   245  
   246  The `aufs` driver is the oldest, but is based on a Linux kernel patch-set that
   247  is unlikely to be merged into the main kernel. These are also known to cause
   248  some serious kernel crashes. However `aufs` allows containers to share
   249  executable and shared library memory, so is a useful choice when running
   250  thousands of containers with the same program or libraries.
   251  
   252  The `devicemapper` driver uses thin provisioning and Copy on Write (CoW)
   253  snapshots. For each devicemapper graph location – typically
   254  `/var/lib/docker/devicemapper` – a thin pool is created based on two block
   255  devices, one for data and one for metadata. By default, these block devices
   256  are created automatically by using loopback mounts of automatically created
   257  sparse files. Refer to [Storage driver options](#storage-driver-options) below
   258  for a way how to customize this setup.
   259  [~jpetazzo/Resizing Docker containers with the Device Mapper plugin](http://jpetazzo.github.io/2014/01/29/docker-device-mapper-resize/)
   260  article explains how to tune your existing setup without the use of options.
   261  
   262  The `btrfs` driver is very fast for `docker build` - but like `devicemapper`
   263  does not share executable memory between devices. Use
   264  `dockerd -s btrfs -g /mnt/btrfs_partition`.
   265  
   266  The `zfs` driver is probably not as fast as `btrfs` but has a longer track record
   267  on stability. Thanks to `Single Copy ARC` shared blocks between clones will be
   268  cached only once. Use `dockerd -s zfs`. To select a different zfs filesystem
   269  set `zfs.fsname` option as described in [Storage driver options](#storage-driver-options).
   270  
   271  The `overlay` is a very fast union filesystem. It is now merged in the main
   272  Linux kernel as of [3.18.0](https://lkml.org/lkml/2014/10/26/137). `overlay`
   273  also supports page cache sharing, this means multiple containers accessing
   274  the same file can share a single page cache entry (or entries), it makes
   275  `overlay` as efficient with memory as `aufs` driver. Call
   276  `dockerd -s overlay` to use it.
   277  
   278  > **Note**: As promising as `overlay` is, the feature is still quite young and
   279  > should not be used in production. Most notably, using `overlay` can cause
   280  > excessive inode consumption (especially as the number of images grows), as
   281  > well as > being incompatible with the use of RPMs.
   282  
   283  The `overlay2` uses the same fast union filesystem but takes advantage of
   284  [additional features](https://lkml.org/lkml/2015/2/11/106) added in Linux
   285  kernel 4.0 to avoid excessive inode consumption. Call `dockerd -s overlay2`
   286  to use it.
   287  
   288  > **Note**: Both `overlay` and `overlay2` are currently unsupported on `btrfs`
   289  > or any Copy on Write filesystem and should only be used over `ext4` partitions.
   290  
   291  ### Options per storage driver
   292  
   293  Particular storage-driver can be configured with options specified with
   294  `--storage-opt` flags. Options for `devicemapper` are prefixed with `dm`,
   295  options for `zfs` start with `zfs` and options for `btrfs` start with `btrfs`.
   296  
   297  #### Devicemapper options
   298  
   299  ##### `dm.thinpooldev`
   300  
   301  Specifies a custom block storage device to use for the thin pool.
   302  
   303  If using a block device for device mapper storage, it is best to use `lvm`
   304  to create and manage the thin-pool volume. This volume is then handed to Docker
   305  to exclusively create snapshot volumes needed for images and containers.
   306  
   307  Managing the thin-pool outside of Engine makes for the most feature-rich
   308  method of having Docker utilize device mapper thin provisioning as the
   309  backing storage for Docker containers. The highlights of the lvm-based
   310  thin-pool management feature include: automatic or interactive thin-pool
   311  resize support, dynamically changing thin-pool features, automatic thinp
   312  metadata checking when lvm activates the thin-pool, etc.
   313  
   314  As a fallback if no thin pool is provided, loopback files are
   315  created. Loopback is very slow, but can be used without any
   316  pre-configuration of storage. It is strongly recommended that you do
   317  not use loopback in production. Ensure your Engine daemon has a
   318  `--storage-opt dm.thinpooldev` argument provided.
   319  
   320  ###### Example:
   321  
   322  ```bash
   323  $ sudo dockerd --storage-opt dm.thinpooldev=/dev/mapper/thin-pool
   324  ```
   325  
   326  ##### `dm.basesize`
   327  
   328  Specifies the size to use when creating the base device, which limits the
   329  size of images and containers. The default value is 10G. Note, thin devices
   330  are inherently "sparse", so a 10G device which is mostly empty doesn't use
   331  10 GB of space on the pool. However, the filesystem will use more space for
   332  the empty case the larger the device is.
   333  
   334  The base device size can be increased at daemon restart which will allow
   335  all future images and containers (based on those new images) to be of the
   336  new base device size.
   337  
   338  ###### Examples
   339  
   340  ```bash
   341  $ sudo dockerd --storage-opt dm.basesize=50G
   342  ```
   343  
   344  This will increase the base device size to 50G. The Docker daemon will throw an
   345  error if existing base device size is larger than 50G. A user can use
   346  this option to expand the base device size however shrinking is not permitted.
   347  
   348  This value affects the system-wide "base" empty filesystem
   349  that may already be initialized and inherited by pulled images. Typically,
   350  a change to this value requires additional steps to take effect:
   351  
   352   ```bash
   353  $ sudo service docker stop
   354  
   355  $ sudo rm -rf /var/lib/docker
   356  
   357  $ sudo service docker start
   358  ```
   359  
   360  
   361  ##### `dm.loopdatasize`
   362  
   363  > **Note**: This option configures devicemapper loopback, which should not
   364  > be used in production.
   365  
   366  Specifies the size to use when creating the loopback file for the
   367  "data" device which is used for the thin pool. The default size is
   368  100G. The file is sparse, so it will not initially take up this
   369  much space.
   370  
   371  ###### Example
   372  
   373  ```bash
   374  $ sudo dockerd --storage-opt dm.loopdatasize=200G
   375  ```
   376  
   377  ##### `dm.loopmetadatasize`
   378  
   379  > **Note**: This option configures devicemapper loopback, which should not
   380  > be used in production.
   381  
   382  Specifies the size to use when creating the loopback file for the
   383  "metadata" device which is used for the thin pool. The default size
   384  is 2G. The file is sparse, so it will not initially take up
   385  this much space.
   386  
   387  ###### Example
   388  
   389  ```bash
   390  $ sudo dockerd --storage-opt dm.loopmetadatasize=4G
   391  ```
   392  
   393  ##### `dm.fs`
   394  
   395  Specifies the filesystem type to use for the base device. The supported
   396  options are "ext4" and "xfs". The default is "xfs"
   397  
   398  ###### Example
   399  
   400  ```bash
   401  $ sudo dockerd --storage-opt dm.fs=ext4
   402  ```
   403  
   404  ##### `dm.mkfsarg`
   405  
   406  Specifies extra mkfs arguments to be used when creating the base device.
   407  
   408  ###### Example
   409  
   410  ```bash
   411  $ sudo dockerd --storage-opt "dm.mkfsarg=-O ^has_journal"
   412  ```
   413  
   414  ##### `dm.mountopt`
   415  
   416  Specifies extra mount options used when mounting the thin devices.
   417  
   418  ###### Example
   419  
   420  ```bash
   421  $ sudo dockerd --storage-opt dm.mountopt=nodiscard
   422  ```
   423  
   424  ##### `dm.datadev`
   425  
   426  (Deprecated, use `dm.thinpooldev`)
   427  
   428  Specifies a custom blockdevice to use for data for the thin pool.
   429  
   430  If using a block device for device mapper storage, ideally both `datadev` and
   431  `metadatadev` should be specified to completely avoid using the loopback
   432  device.
   433  
   434  ###### Example
   435  
   436  ```bash
   437  $ sudo dockerd \
   438        --storage-opt dm.datadev=/dev/sdb1 \
   439        --storage-opt dm.metadatadev=/dev/sdc1
   440  ```
   441  
   442  ##### `dm.metadatadev`
   443  
   444  (Deprecated, use `dm.thinpooldev`)
   445  
   446  Specifies a custom blockdevice to use for metadata for the thin pool.
   447  
   448  For best performance the metadata should be on a different spindle than the
   449  data, or even better on an SSD.
   450  
   451  If setting up a new metadata pool it is required to be valid. This can be
   452  achieved by zeroing the first 4k to indicate empty metadata, like this:
   453  
   454  ```bash
   455  $ dd if=/dev/zero of=$metadata_dev bs=4096 count=1
   456  ```
   457  
   458  ###### Example
   459  
   460  ```bash
   461  $ sudo dockerd \
   462        --storage-opt dm.datadev=/dev/sdb1 \
   463        --storage-opt dm.metadatadev=/dev/sdc1
   464  ```
   465  
   466  ##### `dm.blocksize`
   467  
   468  Specifies a custom blocksize to use for the thin pool. The default
   469  blocksize is 64K.
   470  
   471  ###### Example
   472  
   473  ```bash
   474  $ sudo dockerd --storage-opt dm.blocksize=512K
   475  ```
   476  
   477  ##### `dm.blkdiscard`
   478  
   479  Enables or disables the use of `blkdiscard` when removing devicemapper
   480  devices. This is enabled by default (only) if using loopback devices and is
   481  required to resparsify the loopback file on image/container removal.
   482  
   483  Disabling this on loopback can lead to *much* faster container removal
   484  times, but will make the space used in `/var/lib/docker` directory not be
   485  returned to the system for other use when containers are removed.
   486  
   487  ###### Examples
   488  
   489  ```bash
   490  $ sudo dockerd --storage-opt dm.blkdiscard=false
   491  ```
   492  
   493  ##### `dm.override_udev_sync_check`
   494  
   495  Overrides the `udev` synchronization checks between `devicemapper` and `udev`.
   496  `udev` is the device manager for the Linux kernel.
   497  
   498  To view the `udev` sync support of a Docker daemon that is using the
   499  `devicemapper` driver, run:
   500  
   501  ```bash
   502  $ docker info
   503  [...]
   504  Udev Sync Supported: true
   505  [...]
   506  ```
   507  
   508  When `udev` sync support is `true`, then `devicemapper` and udev can
   509  coordinate the activation and deactivation of devices for containers.
   510  
   511  When `udev` sync support is `false`, a race condition occurs between
   512  the`devicemapper` and `udev` during create and cleanup. The race condition
   513  results in errors and failures. (For information on these failures, see
   514  [docker#4036](https://github.com/docker/docker/issues/4036))
   515  
   516  To allow the `docker` daemon to start, regardless of `udev` sync not being
   517  supported, set `dm.override_udev_sync_check` to true:
   518  
   519  ```bash
   520  $ sudo dockerd --storage-opt dm.override_udev_sync_check=true
   521  ```
   522  
   523  When this value is `true`, the  `devicemapper` continues and simply warns
   524  you the errors are happening.
   525  
   526  > **Note**: The ideal is to pursue a `docker` daemon and environment that does
   527  > support synchronizing with `udev`. For further discussion on this
   528  > topic, see [docker#4036](https://github.com/docker/docker/issues/4036).
   529  > Otherwise, set this flag for migrating existing Docker daemons to
   530  > a daemon with a supported environment.
   531  
   532  ##### `dm.use_deferred_removal`
   533  
   534  Enables use of deferred device removal if `libdm` and the kernel driver
   535  support the mechanism.
   536  
   537  Deferred device removal means that if device is busy when devices are
   538  being removed/deactivated, then a deferred removal is scheduled on
   539  device. And devices automatically go away when last user of the device
   540  exits.
   541  
   542  For example, when a container exits, its associated thin device is removed.
   543  If that device has leaked into some other mount namespace and can't be
   544  removed, the container exit still succeeds and this option causes the
   545  system to schedule the device for deferred removal. It does not wait in a
   546  loop trying to remove a busy device.
   547  
   548  ###### Example
   549  
   550  ```bash
   551  $ sudo dockerd --storage-opt dm.use_deferred_removal=true
   552  ```
   553  
   554  ##### `dm.use_deferred_deletion`
   555  
   556  Enables use of deferred device deletion for thin pool devices. By default,
   557  thin pool device deletion is synchronous. Before a container is deleted,
   558  the Docker daemon removes any associated devices. If the storage driver
   559  can not remove a device, the container deletion fails and daemon returns.
   560  
   561  ```none
   562  Error deleting container: Error response from daemon: Cannot destroy container
   563  ```
   564  
   565  To avoid this failure, enable both deferred device deletion and deferred
   566  device removal on the daemon.
   567  
   568  ```bash
   569  $ sudo dockerd \
   570        --storage-opt dm.use_deferred_deletion=true \
   571        --storage-opt dm.use_deferred_removal=true
   572  ```
   573  
   574  With these two options enabled, if a device is busy when the driver is
   575  deleting a container, the driver marks the device as deleted. Later, when
   576  the device isn't in use, the driver deletes it.
   577  
   578  In general it should be safe to enable this option by default. It will help
   579  when unintentional leaking of mount point happens across multiple mount
   580  namespaces.
   581  
   582  ##### `dm.min_free_space`
   583  
   584  Specifies the min free space percent in a thin pool require for new device
   585  creation to succeed. This check applies to both free data space as well
   586  as free metadata space. Valid values are from 0% - 99%. Value 0% disables
   587  free space checking logic. If user does not specify a value for this option,
   588  the Engine uses a default value of 10%.
   589  
   590  Whenever a new a thin pool device is created (during `docker pull` or during
   591  container creation), the Engine checks if the minimum free space is
   592  available. If sufficient space is unavailable, then device creation fails
   593  and any relevant `docker` operation fails.
   594  
   595  To recover from this error, you must create more free space in the thin pool
   596  to recover from the error. You can create free space by deleting some images
   597  and containers from the thin pool. You can also add more storage to the thin
   598  pool.
   599  
   600  To add more space to a LVM (logical volume management) thin pool, just add
   601  more storage to the volume group container thin pool; this should automatically
   602  resolve any errors. If your configuration uses loop devices, then stop the
   603  Engine daemon, grow the size of loop files and restart the daemon to resolve
   604  the issue.
   605  
   606  ###### Example
   607  
   608  ```bash
   609  $ sudo dockerd --storage-opt dm.min_free_space=10%
   610  ```
   611  
   612  ##### `dm.xfs_nospace_max_retries`
   613  
   614  Specifies the maximum number of retries XFS should attempt to complete
   615  IO when ENOSPC (no space) error is returned by underlying storage device.
   616  
   617  By default XFS retries infinitely for IO to finish and this can result
   618  in unkillable process. To change this behavior one can set
   619  xfs_nospace_max_retries to say 0 and XFS will not retry IO after getting
   620  ENOSPC and will shutdown filesystem.
   621  
   622  ###### Example
   623  
   624  ```bash
   625  $ sudo dockerd --storage-opt dm.xfs_nospace_max_retries=0
   626  ```
   627  
   628  #### ZFS options
   629  
   630  ##### `zfs.fsname`
   631  
   632  Set zfs filesystem under which docker will create its own datasets.
   633  By default docker will pick up the zfs filesystem where docker graph
   634  (`/var/lib/docker`) is located.
   635  
   636  ###### Example
   637  
   638  ```bash
   639  $ sudo dockerd -s zfs --storage-opt zfs.fsname=zroot/docker
   640  ```
   641  
   642  #### Btrfs options
   643  
   644  ##### `btrfs.min_space`
   645  
   646  Specifies the minimum size to use when creating the subvolume which is used
   647  for containers. If user uses disk quota for btrfs when creating or running
   648  a container with **--storage-opt size** option, docker should ensure the
   649  **size** cannot be smaller than **btrfs.min_space**.
   650  
   651  ###### Example
   652  
   653  ```bash
   654  $ sudo dockerd -s btrfs --storage-opt btrfs.min_space=10G
   655  ```
   656  
   657  #### Overlay2 options
   658  
   659  ##### `overlay2.override_kernel_check`
   660  
   661  Overrides the Linux kernel version check allowing overlay2. Support for
   662  specifying multiple lower directories needed by overlay2 was added to the
   663  Linux kernel in 4.0.0. However, some older kernel versions may be patched
   664  to add multiple lower directory support for OverlayFS. This option should
   665  only be used after verifying this support exists in the kernel. Applying
   666  this option on a kernel without this support will cause failures on mount.
   667  
   668  ### Docker runtime execution options
   669  
   670  The Docker daemon relies on a
   671  [OCI](https://github.com/opencontainers/runtime-spec) compliant runtime
   672  (invoked via the `containerd` daemon) as its interface to the Linux
   673  kernel `namespaces`, `cgroups`, and `SELinux`.
   674  
   675  By default, the Docker daemon automatically starts `containerd`. If you want to
   676  control `containerd` startup, manually start `containerd` and pass the path to
   677  the `containerd` socket using the `--containerd` flag. For example:
   678  
   679  ```bash
   680  $ sudo dockerd --containerd /var/run/dev/docker-containerd.sock
   681  ```
   682  
   683  Runtimes can be registered with the daemon either via the
   684  configuration file or using the `--add-runtime` command line argument.
   685  
   686  The following is an example adding 2 runtimes via the configuration:
   687  
   688  ```json
   689  {
   690  	"default-runtime": "runc",
   691  	"runtimes": {
   692  		"runc": {
   693  			"path": "runc"
   694  		},
   695  		"custom": {
   696  			"path": "/usr/local/bin/my-runc-replacement",
   697  			"runtimeArgs": [
   698  				"--debug"
   699  			]
   700  		}
   701  	}
   702  }
   703  ```
   704  
   705  This is the same example via the command line:
   706  
   707  ```bash
   708  $ sudo dockerd --add-runtime runc=runc --add-runtime custom=/usr/local/bin/my-runc-replacement
   709  ```
   710  
   711  > **Note**: Defining runtime arguments via the command line is not supported.
   712  
   713  #### Options for the runtime
   714  
   715  You can configure the runtime using options specified
   716  with the `--exec-opt` flag. All the flag's options have the `native` prefix. A
   717  single `native.cgroupdriver` option is available.
   718  
   719  The `native.cgroupdriver` option specifies the management of the container's
   720  cgroups. You can only specify `cgroupfs` or `systemd`. If you specify
   721  `systemd` and it is not available, the system errors out. If you omit the
   722  `native.cgroupdriver` option,` cgroupfs` is used.
   723  
   724  This example sets the `cgroupdriver` to `systemd`:
   725  
   726  ```bash
   727  $ sudo dockerd --exec-opt native.cgroupdriver=systemd
   728  ```
   729  
   730  Setting this option applies to all containers the daemon launches.
   731  
   732  Also Windows Container makes use of `--exec-opt` for special purpose. Docker user
   733  can specify default container isolation technology with this, for example:
   734  
   735  ```console
   736  > dockerd --exec-opt isolation=hyperv
   737  ```
   738  
   739  Will make `hyperv` the default isolation technology on Windows. If no isolation
   740  value is specified on daemon start, on Windows client, the default is
   741  `hyperv`, and on Windows server, the default is `process`.
   742  
   743  #### Daemon DNS options
   744  
   745  To set the DNS server for all Docker containers, use:
   746  
   747  ```bash
   748  $ sudo dockerd --dns 8.8.8.8
   749  ```
   750  
   751  To set the DNS search domain for all Docker containers, use:
   752  
   753  ```bash
   754  $ sudo dockerd --dns-search example.com
   755  ```
   756  
   757  #### Insecure registries
   758  
   759  Docker considers a private registry either secure or insecure. In the rest of
   760  this section, *registry* is used for *private registry*, and `myregistry:5000`
   761  is a placeholder example for a private registry.
   762  
   763  A secure registry uses TLS and a copy of its CA certificate is placed on the
   764  Docker host at `/etc/docker/certs.d/myregistry:5000/ca.crt`. An insecure
   765  registry is either not using TLS (i.e., listening on plain text HTTP), or is
   766  using TLS with a CA certificate not known by the Docker daemon. The latter can
   767  happen when the certificate was not found under
   768  `/etc/docker/certs.d/myregistry:5000/`, or if the certificate verification
   769  failed (i.e., wrong CA).
   770  
   771  By default, Docker assumes all, but local (see local registries below),
   772  registries are secure. Communicating with an insecure registry is not possible
   773  if Docker assumes that registry is secure. In order to communicate with an
   774  insecure registry, the Docker daemon requires `--insecure-registry` in one of
   775  the following two forms:
   776  
   777  * `--insecure-registry myregistry:5000` tells the Docker daemon that
   778    myregistry:5000 should be considered insecure.
   779  * `--insecure-registry 10.1.0.0/16` tells the Docker daemon that all registries
   780    whose domain resolve to an IP address is part of the subnet described by the
   781    CIDR syntax, should be considered insecure.
   782  
   783  The flag can be used multiple times to allow multiple registries to be marked
   784  as insecure.
   785  
   786  If an insecure registry is not marked as insecure, `docker pull`,
   787  `docker push`, and `docker search` will result in an error message prompting
   788  the user to either secure or pass the `--insecure-registry` flag to the Docker
   789  daemon as described above.
   790  
   791  Local registries, whose IP address falls in the 127.0.0.0/8 range, are
   792  automatically marked as insecure as of Docker 1.3.2. It is not recommended to
   793  rely on this, as it may change in the future.
   794  
   795  Enabling `--insecure-registry`, i.e., allowing un-encrypted and/or untrusted
   796  communication, can be useful when running a local registry.  However,
   797  because its use creates security vulnerabilities it should ONLY be enabled for
   798  testing purposes.  For increased security, users should add their CA to their
   799  system's list of trusted CAs instead of enabling `--insecure-registry`.
   800  
   801  ##### Legacy Registries
   802  
   803  Enabling `--disable-legacy-registry` forces a docker daemon to only interact with registries which support the V2 protocol.  Specifically, the daemon will not attempt `push`, `pull` and `login` to v1 registries.  The exception to this is `search` which can still be performed on v1 registries.
   804  
   805  #### Running a Docker daemon behind an HTTPS_PROXY
   806  
   807  When running inside a LAN that uses an `HTTPS` proxy, the Docker Hub
   808  certificates will be replaced by the proxy's certificates. These certificates
   809  need to be added to your Docker host's configuration:
   810  
   811  1. Install the `ca-certificates` package for your distribution
   812  2. Ask your network admin for the proxy's CA certificate and append them to
   813     `/etc/pki/tls/certs/ca-bundle.crt`
   814  3. Then start your Docker daemon with `HTTPS_PROXY=http://username:password@proxy:port/ dockerd`.
   815     The `username:` and `password@` are optional - and are only needed if your
   816     proxy is set up to require authentication.
   817  
   818  This will only add the proxy and authentication to the Docker daemon's requests -
   819  your `docker build`s and running containers will need extra configuration to
   820  use the proxy
   821  
   822  #### Default `ulimit` settings
   823  
   824  `--default-ulimit` allows you to set the default `ulimit` options to use for
   825  all containers. It takes the same options as `--ulimit` for `docker run`. If
   826  these defaults are not set, `ulimit` settings will be inherited, if not set on
   827  `docker run`, from the Docker daemon. Any `--ulimit` options passed to
   828  `docker run` will overwrite these defaults.
   829  
   830  Be careful setting `nproc` with the `ulimit` flag as `nproc` is designed by Linux to
   831  set the maximum number of processes available to a user, not to a container. For details
   832  please check the [run](run.md) reference.
   833  
   834  #### Node discovery
   835  
   836  The `--cluster-advertise` option specifies the `host:port` or `interface:port`
   837  combination that this particular daemon instance should use when advertising
   838  itself to the cluster. The daemon is reached by remote hosts through this value.
   839  If you  specify an interface, make sure it includes the IP address of the actual
   840  Docker host. For Engine installation created through `docker-machine`, the
   841  interface is typically `eth1`.
   842  
   843  The daemon uses [libkv](https://github.com/docker/libkv/) to advertise
   844  the node within the cluster. Some key-value backends support mutual
   845  TLS. To configure the client TLS settings used by the daemon can be configured
   846  using the `--cluster-store-opt` flag, specifying the paths to PEM encoded
   847  files. For example:
   848  
   849  ```bash
   850  $ sudo dockerd \
   851      --cluster-advertise 192.168.1.2:2376 \
   852      --cluster-store etcd://192.168.1.2:2379 \
   853      --cluster-store-opt kv.cacertfile=/path/to/ca.pem \
   854      --cluster-store-opt kv.certfile=/path/to/cert.pem \
   855      --cluster-store-opt kv.keyfile=/path/to/key.pem
   856  ```
   857  
   858  The currently supported cluster store options are:
   859  
   860  | Option                | Description |
   861  |-----------------------|-------------|
   862  | `discovery.heartbeat` | Specifies the heartbeat timer in seconds which is used by the daemon as a `keepalive` mechanism to make sure discovery module treats the node as alive in the cluster. If not configured, the default value is 20 seconds. |
   863  | `discovery.ttl`       | Specifies the TTL (time-to-live) in seconds which is used by the discovery module to timeout a node if a valid heartbeat is not received within the configured ttl value. If not configured, the default value is 60 seconds. |
   864  | `kv.cacertfile`       | Specifies the path to a local file with PEM encoded CA certificates to trust. |
   865  | `kv.certfile`         | Specifies the path to a local file with a PEM encoded certificate. This certificate is used as the client cert for communication with the Key/Value store. |
   866  | `kv.keyfile`          | Specifies the path to a local file with a PEM encoded private key. This private key is used as the client key for communication with the Key/Value store. |
   867  | `kv.path`             | Specifies the path in the Key/Value store. If not configured, the default value is 'docker/nodes'. |
   868  
   869  #### Access authorization
   870  
   871  Docker's access authorization can be extended by authorization plugins that your
   872  organization can purchase or build themselves. You can install one or more
   873  authorization plugins when you start the Docker `daemon` using the
   874  `--authorization-plugin=PLUGIN_ID` option.
   875  
   876  ```bash
   877  $ sudo dockerd --authorization-plugin=plugin1 --authorization-plugin=plugin2,...
   878  ```
   879  
   880  The `PLUGIN_ID` value is either the plugin's name or a path to its specification
   881  file. The plugin's implementation determines whether you can specify a name or
   882  path. Consult with your Docker administrator to get information about the
   883  plugins available to you.
   884  
   885  Once a plugin is installed, requests made to the `daemon` through the command
   886  line or Docker's Engine API are allowed or denied by the plugin.  If you have
   887  multiple plugins installed, at least one must allow the request for it to
   888  complete.
   889  
   890  For information about how to create an authorization plugin, see [authorization
   891  plugin](../../extend/plugins_authorization.md) section in the Docker extend section of this documentation.
   892  
   893  
   894  #### Daemon user namespace options
   895  
   896  The Linux kernel [user namespace support](http://man7.org/linux/man-pages/man7/user_namespaces.7.html) provides additional security by enabling
   897  a process, and therefore a container, to have a unique range of user and
   898  group IDs which are outside the traditional user and group range utilized by
   899  the host system. Potentially the most important security improvement is that,
   900  by default, container processes running as the `root` user will have expected
   901  administrative privilege (with some restrictions) inside the container but will
   902  effectively be mapped to an unprivileged `uid` on the host.
   903  
   904  When user namespace support is enabled, Docker creates a single daemon-wide mapping
   905  for all containers running on the same engine instance. The mappings will
   906  utilize the existing subordinate user and group ID feature available on all modern
   907  Linux distributions.
   908  The [`/etc/subuid`](http://man7.org/linux/man-pages/man5/subuid.5.html) and
   909  [`/etc/subgid`](http://man7.org/linux/man-pages/man5/subgid.5.html) files will be
   910  read for the user, and optional group, specified to the `--userns-remap`
   911  parameter.  If you do not wish to specify your own user and/or group, you can
   912  provide `default` as the value to this flag, and a user will be created on your behalf
   913  and provided subordinate uid and gid ranges. This default user will be named
   914  `dockremap`, and entries will be created for it in `/etc/passwd` and
   915  `/etc/group` using your distro's standard user and group creation tools.
   916  
   917  > **Note**: The single mapping per-daemon restriction is in place for now
   918  > because Docker shares image layers from its local cache across all
   919  > containers running on the engine instance.  Since file ownership must be
   920  > the same for all containers sharing the same layer content, the decision
   921  > was made to map the file ownership on `docker pull` to the daemon's user and
   922  > group mappings so that there is no delay for running containers once the
   923  > content is downloaded. This design preserves the same performance for `docker
   924  > pull`, `docker push`, and container startup as users expect with
   925  > user namespaces disabled.
   926  
   927  ##### Start the daemon with user namespaces enabled
   928  
   929  To enable user namespace support, start the daemon with the
   930  `--userns-remap` flag, which accepts values in the following formats:
   931  
   932   - uid
   933   - uid:gid
   934   - username
   935   - username:groupname
   936  
   937  If numeric IDs are provided, translation back to valid user or group names
   938  will occur so that the subordinate uid and gid information can be read, given
   939  these resources are name-based, not id-based.  If the numeric ID information
   940  provided does not exist as entries in `/etc/passwd` or `/etc/group`, daemon
   941  startup will fail with an error message.
   942  
   943  **Example: starting with default Docker user management:**
   944  
   945  ```bash
   946  $ sudo dockerd --userns-remap=default
   947  ```
   948  
   949  When `default` is provided, Docker will create - or find the existing - user and group
   950  named `dockremap`. If the user is created, and the Linux distribution has
   951  appropriate support, the `/etc/subuid` and `/etc/subgid` files will be populated
   952  with a contiguous 65536 length range of subordinate user and group IDs, starting
   953  at an offset based on prior entries in those files.  For example, Ubuntu will
   954  create the following range, based on an existing user named `user1` already owning
   955  the first 65536 range:
   956  
   957  ```bash
   958  $ cat /etc/subuid
   959  user1:100000:65536
   960  dockremap:165536:65536
   961  ```
   962  
   963  If you have a preferred/self-managed user with subordinate ID mappings already
   964  configured, you can provide that username or uid to the `--userns-remap` flag.
   965  If you have a group that doesn't match the username, you may provide the `gid`
   966  or group name as well; otherwise the username will be used as the group name
   967  when querying the system for the subordinate group ID range.
   968  
   969  The output of `docker info` can be used to determine if the daemon is running
   970  with user namespaces enabled or not. If the daemon is configured with user
   971  namespaces, the Security Options entry in the response will list "userns" as
   972  one of the enabled security features.
   973  
   974  ##### Behavior differences when user namespaces are enabled
   975  
   976  When you start the Docker daemon with `--userns-remap`, Docker segregates the graph directory
   977  where the images are stored by adding an extra directory with a name corresponding to the
   978  remapped UID and GID. For example, if the remapped UID and GID begin with `165536`, all
   979  images and containers running with that remap setting are located in `/var/lib/docker/165536.165536`
   980  instead of `/var/lib/docker/`.
   981  
   982  In addition, the files and directories within the new directory, which correspond to
   983  images and container layers, are also owned by the new UID and GID. To set the ownership
   984  correctly, you need to re-pull the images and restart the containers after starting the
   985  daemon with `--userns-remap`.
   986  
   987  ##### Detailed information on `subuid`/`subgid` ranges
   988  
   989  Given potential advanced use of the subordinate ID ranges by power users, the
   990  following paragraphs define how the Docker daemon currently uses the range entries
   991  found within the subordinate range files.
   992  
   993  The simplest case is that only one contiguous range is defined for the
   994  provided user or group. In this case, Docker will use that entire contiguous
   995  range for the mapping of host uids and gids to the container process.  This
   996  means that the first ID in the range will be the remapped root user, and the
   997  IDs above that initial ID will map host ID 1 through the end of the range.
   998  
   999  From the example `/etc/subuid` content shown above, the remapped root
  1000  user would be uid 165536.
  1001  
  1002  If the system administrator has set up multiple ranges for a single user or
  1003  group, the Docker daemon will read all the available ranges and use the
  1004  following algorithm to create the mapping ranges:
  1005  
  1006  1. The range segments found for the particular user will be sorted by *start ID* ascending.
  1007  2. Map segments will be created from each range in increasing value with a length matching the length of each segment. Therefore the range segment with the lowest numeric starting value will be equal to the remapped root, and continue up through host uid/gid equal to the range segment length. As an example, if the lowest segment starts at ID 1000 and has a length of 100, then a map of 1000 -> 0 (the remapped root) up through 1100 -> 100 will be created from this segment. If the next segment starts at ID 10000, then the next map will start with mapping 10000 -> 101 up to the length of this second segment. This will continue until no more segments are found in the subordinate files for this user.
  1008  3. If more than five range segments exist for a single user, only the first five will be utilized, matching the kernel's limitation of only five entries in `/proc/self/uid_map` and `proc/self/gid_map`.
  1009  
  1010  ##### Disable user namespace for a container
  1011  
  1012  If you enable user namespaces on the daemon, all containers are started
  1013  with user namespaces enabled. In some situations you might want to disable
  1014  this feature for a container, for example, to start a privileged container (see
  1015  [user namespace known restrictions](#user-namespace-known-restrictions)).
  1016  To enable those advanced features for a specific container use `--userns=host`
  1017  in the `run/exec/create` command.
  1018  This option will completely disable user namespace mapping for the container's user.
  1019  
  1020  ##### User namespace known restrictions
  1021  
  1022  The following standard Docker features are currently incompatible when
  1023  running a Docker daemon with user namespaces enabled:
  1024  
  1025   - sharing PID or NET namespaces with the host (`--pid=host` or `--net=host`)
  1026   - Using `--privileged` mode flag on `docker run` (unless also specifying `--userns=host`)
  1027  
  1028  In general, user namespaces are an advanced feature and will require
  1029  coordination with other capabilities. For example, if volumes are mounted from
  1030  the host, file ownership will have to be pre-arranged if the user or
  1031  administrator wishes the containers to have expected access to the volume
  1032  contents. Note that when using external volume or graph driver plugins, those
  1033  external software programs must be made aware of user and group mapping ranges
  1034  if they are to work seamlessly with user namespace support.
  1035  
  1036  Finally, while the `root` user inside a user namespaced container process has
  1037  many of the expected admin privileges that go along with being the superuser, the
  1038  Linux kernel has restrictions based on internal knowledge that this is a user namespaced
  1039  process. The most notable restriction that we are aware of at this time is the
  1040  inability to use `mknod`. Permission will be denied for device creation even as
  1041  container `root` inside a user namespace.
  1042  
  1043  ### Miscellaneous options
  1044  
  1045  IP masquerading uses address translation to allow containers without a public
  1046  IP to talk to other machines on the Internet. This may interfere with some
  1047  network topologies and can be disabled with `--ip-masq=false`.
  1048  
  1049  Docker supports softlinks for the Docker data directory (`/var/lib/docker`) and
  1050  for `/var/lib/docker/tmp`. The `DOCKER_TMPDIR` and the data directory can be
  1051  set like this:
  1052  
  1053      DOCKER_TMPDIR=/mnt/disk2/tmp /usr/local/bin/dockerd -D -g /var/lib/docker -H unix:// > /var/lib/docker-machine/docker.log 2>&1
  1054      # or
  1055      export DOCKER_TMPDIR=/mnt/disk2/tmp
  1056      /usr/local/bin/dockerd -D -g /var/lib/docker -H unix:// > /var/lib/docker-machine/docker.log 2>&1
  1057  
  1058  #### Default cgroup parent
  1059  
  1060  The `--cgroup-parent` option allows you to set the default cgroup parent
  1061  to use for containers. If this option is not set, it defaults to `/docker` for
  1062  fs cgroup driver and `system.slice` for systemd cgroup driver.
  1063  
  1064  If the cgroup has a leading forward slash (`/`), the cgroup is created
  1065  under the root cgroup, otherwise the cgroup is created under the daemon
  1066  cgroup.
  1067  
  1068  Assuming the daemon is running in cgroup `daemoncgroup`,
  1069  `--cgroup-parent=/foobar` creates a cgroup in
  1070  `/sys/fs/cgroup/memory/foobar`, whereas using `--cgroup-parent=foobar`
  1071  creates the cgroup in `/sys/fs/cgroup/memory/daemoncgroup/foobar`
  1072  
  1073  The systemd cgroup driver has different rules for `--cgroup-parent`. Systemd
  1074  represents hierarchy by slice and the name of the slice encodes the location in
  1075  the tree. So `--cgroup-parent` for systemd cgroups should be a slice name. A
  1076  name can consist of a dash-separated series of names, which describes the path
  1077  to the slice from the root slice. For example, `--cgroup-parent=user-a-b.slice`
  1078  means the memory cgroup for the container is created in
  1079  `/sys/fs/cgroup/memory/user.slice/user-a.slice/user-a-b.slice/docker-<id>.scope`.
  1080  
  1081  This setting can also be set per container, using the `--cgroup-parent`
  1082  option on `docker create` and `docker run`, and takes precedence over
  1083  the `--cgroup-parent` option on the daemon.
  1084  
  1085  #### Daemon metrics
  1086  
  1087  The `--metrics-addr` option takes a tcp address to serve the metrics API.
  1088  This feature is still experimental, therefore, the daemon must be running in experimental
  1089  mode for this feature to work.
  1090  
  1091  To serve the metrics API on localhost:1337 you would specify `--metrics-addr 127.0.0.1:1337`
  1092  allowing you to make requests on the API at `127.0.0.1:1337/metrics` to receive metrics in the
  1093  [prometheus](https://prometheus.io/docs/instrumenting/exposition_formats/) format.
  1094  
  1095  If you are running a prometheus server you can add this address to your scrape configs
  1096  to have prometheus collect metrics on Docker.  For more information
  1097  on prometheus you can view the website [here](https://prometheus.io/).
  1098  
  1099  ```none
  1100  scrape_configs:
  1101    - job_name: 'docker'
  1102      static_configs:
  1103        - targets: ['127.0.0.1:1337']
  1104  ```
  1105  
  1106  Please note that this feature is still marked as experimental as metrics and metric
  1107  names could change while this feature is still in experimental.  Please provide
  1108  feedback on what you would like to see collected in the API.
  1109  
  1110  #### Daemon configuration file
  1111  
  1112  The `--config-file` option allows you to set any configuration option
  1113  for the daemon in a JSON format. This file uses the same flag names as keys,
  1114  except for flags that allow several entries, where it uses the plural
  1115  of the flag name, e.g., `labels` for the `label` flag.
  1116  
  1117  The options set in the configuration file must not conflict with options set
  1118  via flags. The docker daemon fails to start if an option is duplicated between
  1119  the file and the flags, regardless their value. We do this to avoid
  1120  silently ignore changes introduced in configuration reloads.
  1121  For example, the daemon fails to start if you set daemon labels
  1122  in the configuration file and also set daemon labels via the `--label` flag.
  1123  Options that are not present in the file are ignored when the daemon starts.
  1124  
  1125  ##### On Linux
  1126  
  1127  The default location of the configuration file on Linux is
  1128  `/etc/docker/daemon.json`. The `--config-file` flag can be used to specify a
  1129   non-default location.
  1130  
  1131  This is a full example of the allowed configuration options on Linux:
  1132  
  1133  ```json
  1134  {
  1135  	"authorization-plugins": [],
  1136  	"dns": [],
  1137  	"dns-opts": [],
  1138  	"dns-search": [],
  1139  	"exec-opts": [],
  1140  	"exec-root": "",
  1141  	"experimental": false,
  1142  	"storage-driver": "",
  1143  	"storage-opts": [],
  1144  	"labels": [],
  1145  	"live-restore": true,
  1146  	"log-driver": "",
  1147  	"log-opts": {},
  1148  	"mtu": 0,
  1149  	"pidfile": "",
  1150  	"graph": "",
  1151  	"cluster-store": "",
  1152  	"cluster-store-opts": {},
  1153  	"cluster-advertise": "",
  1154  	"max-concurrent-downloads": 3,
  1155  	"max-concurrent-uploads": 5,
  1156  	"default-shm-size": "64M",
  1157  	"shutdown-timeout": 15,
  1158  	"debug": true,
  1159  	"hosts": [],
  1160  	"log-level": "",
  1161  	"tls": true,
  1162  	"tlsverify": true,
  1163  	"tlscacert": "",
  1164  	"tlscert": "",
  1165  	"tlskey": "",
  1166  	"swarm-default-advertise-addr": "",
  1167  	"api-cors-header": "",
  1168  	"selinux-enabled": false,
  1169  	"userns-remap": "",
  1170  	"group": "",
  1171  	"cgroup-parent": "",
  1172  	"default-ulimits": {},
  1173  	"init": false,
  1174  	"init-path": "/usr/libexec/docker-init",
  1175  	"ipv6": false,
  1176  	"iptables": false,
  1177  	"ip-forward": false,
  1178  	"ip-masq": false,
  1179  	"userland-proxy": false,
  1180  	"userland-proxy-path": "/usr/libexec/docker-proxy",
  1181  	"ip": "0.0.0.0",
  1182  	"bridge": "",
  1183  	"bip": "",
  1184  	"fixed-cidr": "",
  1185  	"fixed-cidr-v6": "",
  1186  	"default-gateway": "",
  1187  	"default-gateway-v6": "",
  1188  	"icc": false,
  1189  	"raw-logs": false,
  1190  	"registry-mirrors": [],
  1191  	"seccomp-profile": "",
  1192  	"insecure-registries": [],
  1193  	"disable-legacy-registry": false,
  1194  	"no-new-privileges": false,
  1195  	"default-runtime": "runc",
  1196  	"oom-score-adjust": -500,
  1197  	"runtimes": {
  1198  		"runc": {
  1199  			"path": "runc"
  1200  		},
  1201  		"custom": {
  1202  			"path": "/usr/local/bin/my-runc-replacement",
  1203  			"runtimeArgs": [
  1204  				"--debug"
  1205  			]
  1206  		}
  1207  	}
  1208  }
  1209  ```
  1210  
  1211  ##### On Windows
  1212  
  1213  The default location of the configuration file on Windows is
  1214   `%programdata%\docker\config\daemon.json`. The `--config-file` flag can be
  1215   used to specify a non-default location.
  1216  
  1217  This is a full example of the allowed configuration options on Windows:
  1218  
  1219  ```json
  1220  {
  1221      "authorization-plugins": [],
  1222      "dns": [],
  1223      "dns-opts": [],
  1224      "dns-search": [],
  1225      "exec-opts": [],
  1226      "experimental": false,
  1227      "storage-driver": "",
  1228      "storage-opts": [],
  1229      "labels": [],
  1230      "log-driver": "",
  1231      "mtu": 0,
  1232      "pidfile": "",
  1233      "graph": "",
  1234      "cluster-store": "",
  1235      "cluster-advertise": "",
  1236      "max-concurrent-downloads": 3,
  1237      "max-concurrent-uploads": 5,
  1238      "shutdown-timeout": 15,
  1239      "debug": true,
  1240      "hosts": [],
  1241      "log-level": "",
  1242      "tlsverify": true,
  1243      "tlscacert": "",
  1244      "tlscert": "",
  1245      "tlskey": "",
  1246      "swarm-default-advertise-addr": "",
  1247      "group": "",
  1248      "default-ulimits": {},
  1249      "bridge": "",
  1250      "fixed-cidr": "",
  1251      "raw-logs": false,
  1252      "registry-mirrors": [],
  1253      "insecure-registries": [],
  1254      "disable-legacy-registry": false
  1255  }
  1256  ```
  1257  
  1258  #### Configuration reload behavior
  1259  
  1260  Some options can be reconfigured when the daemon is running without requiring
  1261  to restart the process. We use the `SIGHUP` signal in Linux to reload, and a global event
  1262  in Windows with the key `Global\docker-daemon-config-$PID`. The options can
  1263  be modified in the configuration file but still will check for conflicts with
  1264  the provided flags. The daemon fails to reconfigure itself
  1265  if there are conflicts, but it won't stop execution.
  1266  
  1267  The list of currently supported options that can be reconfigured is this:
  1268  
  1269  - `debug`: it changes the daemon to debug mode when set to true.
  1270  - `cluster-store`: it reloads the discovery store with the new address.
  1271  - `cluster-store-opts`: it uses the new options to reload the discovery store.
  1272  - `cluster-advertise`: it modifies the address advertised after reloading.
  1273  - `labels`: it replaces the daemon labels with a new set of labels.
  1274  - `live-restore`: Enables [keeping containers alive during daemon downtime](https://docs.docker.com/engine/admin/live-restore/).
  1275  - `max-concurrent-downloads`: it updates the max concurrent downloads for each pull.
  1276  - `max-concurrent-uploads`: it updates the max concurrent uploads for each push.
  1277  - `default-runtime`: it updates the runtime to be used if not is
  1278    specified at container creation. It defaults to "default" which is
  1279    the runtime shipped with the official docker packages.
  1280  - `runtimes`: it updates the list of available OCI runtimes that can
  1281    be used to run containers
  1282  - `authorization-plugin`: specifies the authorization plugins to use.
  1283  - `insecure-registries`: it replaces the daemon insecure registries with a new set of insecure registries. If some existing insecure registries in daemon's configuration are not in newly reloaded insecure resgitries, these existing ones will be removed from daemon's config.
  1284  - `registry-mirrors`: it replaces the daemon registry mirrors with a new set of registry mirrors. If some existing registry mirrors in daemon's configuration are not in newly reloaded registry mirrors, these existing ones will be removed from daemon's config.
  1285  
  1286  Updating and reloading the cluster configurations such as `--cluster-store`,
  1287  `--cluster-advertise` and `--cluster-store-opts` will take effect only if
  1288  these configurations were not previously configured. If `--cluster-store`
  1289  has been provided in flags and `cluster-advertise` not, `cluster-advertise`
  1290  can be added in the configuration file without accompanied by `--cluster-store`.
  1291  Configuration reload will log a warning message if it detects a change in
  1292  previously configured cluster configurations.
  1293  
  1294  
  1295  ### Run multiple daemons
  1296  
  1297  > **Note:** Running multiple daemons on a single host is considered as "experimental". The user should be aware of
  1298  > unsolved problems. This solution may not work properly in some cases. Solutions are currently under development
  1299  > and will be delivered in the near future.
  1300  
  1301  This section describes how to run multiple Docker daemons on a single host. To
  1302  run multiple daemons, you must configure each daemon so that it does not
  1303  conflict with other daemons on the same host. You can set these options either
  1304  by providing them as flags, or by using a [daemon configuration file](#daemon-configuration-file).
  1305  
  1306  The following daemon options must be configured for each daemon:
  1307  
  1308  ```none
  1309  -b, --bridge=                          Attach containers to a network bridge
  1310  --exec-root=/var/run/docker            Root of the Docker execdriver
  1311  -g, --graph=/var/lib/docker            Root of the Docker runtime
  1312  -p, --pidfile=/var/run/docker.pid      Path to use for daemon PID file
  1313  -H, --host=[]                          Daemon socket(s) to connect to
  1314  --iptables=true                        Enable addition of iptables rules
  1315  --config-file=/etc/docker/daemon.json  Daemon configuration file
  1316  --tlscacert="~/.docker/ca.pem"         Trust certs signed only by this CA
  1317  --tlscert="~/.docker/cert.pem"         Path to TLS certificate file
  1318  --tlskey="~/.docker/key.pem"           Path to TLS key file
  1319  ```
  1320  
  1321  When your daemons use different values for these flags, you can run them on the same host without any problems.
  1322  It is very important to properly understand the meaning of those options and to use them correctly.
  1323  
  1324  - The `-b, --bridge=` flag is set to `docker0` as default bridge network. It is created automatically when you install Docker.
  1325  If you are not using the default, you must create and configure the bridge manually or just set it to 'none': `--bridge=none`
  1326  - `--exec-root` is the path where the container state is stored. The default value is `/var/run/docker`. Specify the path for
  1327  your running daemon here.
  1328  - `--graph` is the path where images are stored. The default value is `/var/lib/docker`. To avoid any conflict with other daemons
  1329  set this parameter separately for each daemon.
  1330  - `-p, --pidfile=/var/run/docker.pid` is the path where the process ID of the daemon is stored. Specify the path for your
  1331  pid file here.
  1332  - `--host=[]` specifies where the Docker daemon will listen for client connections. If unspecified, it defaults to `/var/run/docker.sock`.
  1333  -  `--iptables=false` prevents the Docker daemon from adding iptables rules. If
  1334  multiple daemons manage iptables rules, they may overwrite rules set by another
  1335  daemon. Be aware that disabling this option requires you to manually add
  1336  iptables rules to expose container ports. If you prevent Docker from adding
  1337  iptables rules, Docker will also not add IP masquerading rules, even if you set
  1338  `--ip-masq` to `true`. Without IP masquerading rules, Docker containers will not be
  1339  able to connect to external hosts or the internet when using network other than
  1340  default bridge.
  1341  - `--config-file=/etc/docker/daemon.json` is the path where configuration file is stored. You can use it instead of
  1342  daemon flags. Specify the path for each daemon.
  1343  - `--tls*` Docker daemon supports `--tlsverify` mode that enforces encrypted and authenticated remote connections.
  1344  The `--tls*` options enable use of specific certificates for individual daemons.
  1345  
  1346  Example script for a separate “bootstrap” instance of the Docker daemon without network:
  1347  
  1348  ```bash
  1349  $ sudo dockerd \
  1350          -H unix:///var/run/docker-bootstrap.sock \
  1351          -p /var/run/docker-bootstrap.pid \
  1352          --iptables=false \
  1353          --ip-masq=false \
  1354          --bridge=none \
  1355          --graph=/var/lib/docker-bootstrap \
  1356          --exec-root=/var/run/docker-bootstrap
  1357  ```