github.com/kim0/docker@v0.6.2-0.20161130212042-4addda3f07e7/docs/reference/commandline/dockerd.md (about)

     1  ---
     2  title: "dockerd"
     3  aliases: ["/engine/reference/commandline/daemon/"]
     4  description: "The daemon command description and usage"
     5  keywords: ["container, daemon, runtime"]
     6  ---
     7  
     8  <!-- This file is maintained within the docker/docker Github
     9       repository at https://github.com/docker/docker/. Make all
    10       pull requests against that repo. If you see this file in
    11       another repository, consider it read-only there, as it will
    12       periodically be overwritten by the definitive file. Pull
    13       requests which include edits to this file in other repositories
    14       will be rejected.
    15  -->
    16  
    17  # daemon
    18  
    19  ```markdown
    20  Usage: dockerd [OPTIONS]
    21  
    22  A self-sufficient runtime for containers.
    23  
    24  Options:
    25  
    26        --add-runtime value                     Register an additional OCI compatible runtime (default [])
    27        --api-cors-header string                Set CORS headers in the remote API
    28        --authorization-plugin value            Authorization plugins to load (default [])
    29        --bip string                            Specify network bridge IP
    30    -b, --bridge string                         Attach containers to a network bridge
    31        --cgroup-parent string                  Set parent cgroup for all containers
    32        --cluster-advertise string              Address or interface name to advertise
    33        --cluster-store string                  URL of the distributed storage backend
    34        --cluster-store-opt value               Set cluster store options (default map[])
    35        --config-file string                    Daemon configuration file (default "/etc/docker/daemon.json")
    36        --containerd string                     Path to containerd socket
    37    -D, --debug                                 Enable debug mode
    38        --default-gateway value                 Container default gateway IPv4 address
    39        --default-gateway-v6 value              Container default gateway IPv6 address
    40        --default-runtime string                Default OCI runtime for containers (default "runc")
    41        --default-ulimit value                  Default ulimits for containers (default [])
    42        --disable-legacy-registry               Disable contacting legacy registries
    43        --dns value                             DNS server to use (default [])
    44        --dns-opt value                         DNS options to use (default [])
    45        --dns-search value                      DNS search domains to use (default [])
    46        --exec-opt value                        Runtime execution options (default [])
    47        --exec-root string                      Root directory for execution state files (default "/var/run/docker")
    48        --experimental                          Enable experimental features
    49        --fixed-cidr string                     IPv4 subnet for fixed IPs
    50        --fixed-cidr-v6 string                  IPv6 subnet for fixed IPs
    51    -g, --graph string                          Root of the Docker runtime (default "/var/lib/docker")
    52    -G, --group string                          Group for the unix socket (default "docker")
    53        --help                                  Print usage
    54    -H, --host value                            Daemon socket(s) to connect to (default [])
    55        --icc                                   Enable inter-container communication (default true)
    56        --init                                  Run an init in the container to forward signals and reap processes
    57        --init-path string                      Path to the docker-init binary
    58        --insecure-registry value               Enable insecure registry communication (default [])
    59        --ip value                              Default IP when binding container ports (default 0.0.0.0)
    60        --ip-forward                            Enable net.ipv4.ip_forward (default true)
    61        --ip-masq                               Enable IP masquerading (default true)
    62        --iptables                              Enable addition of iptables rules (default true)
    63        --ipv6                                  Enable IPv6 networking
    64        --label value                           Set key=value labels to the daemon (default [])
    65        --live-restore                          Enable live restore of docker when containers are still running
    66        --log-driver string                     Default driver for container logs (default "json-file")
    67    -l, --log-level string                      Set the logging level (debug, info, warn, error, fatal) (default "info")
    68        --log-opt value                         Default log driver options for containers (default map[])
    69        --max-concurrent-downloads int          Set the max concurrent downloads for each pull (default 3)
    70        --max-concurrent-uploads int            Set the max concurrent uploads for each push (default 5)
    71        --mtu int                               Set the containers network MTU
    72        --oom-score-adjust int                  Set the oom_score_adj for the daemon (default -500)
    73    -p, --pidfile string                        Path to use for daemon PID file (default "/var/run/docker.pid")
    74        --raw-logs                              Full timestamps without ANSI coloring
    75        --registry-mirror value                 Preferred Docker registry mirror (default [])
    76        --selinux-enabled                       Enable selinux support
    77        --shutdown-timeout=15                   Set the shutdown timeout value in seconds
    78    -s, --storage-driver string                 Storage driver to use
    79        --storage-opt value                     Storage driver options (default [])
    80        --swarm-default-advertise-addr string   Set default address or interface for swarm advertised address
    81        --tls                                   Use TLS; implied by --tlsverify
    82        --tlscacert string                      Trust certs signed only by this CA (default "/root/.docker/ca.pem")
    83        --tlscert string                        Path to TLS certificate file (default "/root/.docker/cert.pem")
    84        --tlskey string                         Path to TLS key file (default "/root/.docker/key.pem")
    85        --tlsverify                             Use TLS and verify the remote
    86        --userland-proxy                        Use userland proxy for loopback traffic (default true)
    87        --userland-proxy-path string            Path to the userland proxy binary
    88        --userns-remap string                   User/Group setting for user namespaces
    89    -v, --version                               Print version information and quit
    90  ```
    91  
    92  Options with [] may be specified multiple times.
    93  
    94  dockerd is the persistent process that manages containers. Docker
    95  uses different binaries for the daemon and client. To run the daemon you
    96  type `dockerd`.
    97  
    98  To run the daemon with debug output, use `dockerd -D`.
    99  
   100  ## Daemon socket option
   101  
   102  The Docker daemon can listen for [Docker Remote API](../api/docker_remote_api.md)
   103  requests via three different types of Socket: `unix`, `tcp`, and `fd`.
   104  
   105  By default, a `unix` domain socket (or IPC socket) is created at
   106  `/var/run/docker.sock`, requiring either `root` permission, or `docker` group
   107  membership.
   108  
   109  If you need to access the Docker daemon remotely, you need to enable the `tcp`
   110  Socket. Beware that the default setup provides un-encrypted and
   111  un-authenticated direct access to the Docker daemon - and should be secured
   112  either using the [built in HTTPS encrypted socket](https://docs.docker.com/engine/security/https/), or by
   113  putting a secure web proxy in front of it. You can listen on port `2375` on all
   114  network interfaces with `-H tcp://0.0.0.0:2375`, or on a particular network
   115  interface using its IP address: `-H tcp://192.168.59.103:2375`. It is
   116  conventional to use port `2375` for un-encrypted, and port `2376` for encrypted
   117  communication with the daemon.
   118  
   119  > **Note:**
   120  > If you're using an HTTPS encrypted socket, keep in mind that only
   121  > TLS1.0 and greater are supported. Protocols SSLv3 and under are not
   122  > supported anymore for security reasons.
   123  
   124  On Systemd based systems, you can communicate with the daemon via
   125  [Systemd socket activation](http://0pointer.de/blog/projects/socket-activation.html),
   126  use `dockerd -H fd://`. Using `fd://` will work perfectly for most setups but
   127  you can also specify individual sockets: `dockerd -H fd://3`. If the
   128  specified socket activated files aren't found, then Docker will exit. You can
   129  find examples of using Systemd socket activation with Docker and Systemd in the
   130  [Docker source tree](https://github.com/docker/docker/tree/master/contrib/init/systemd/).
   131  
   132  You can configure the Docker daemon to listen to multiple sockets at the same
   133  time using multiple `-H` options:
   134  
   135  ```bash
   136  # listen using the default unix socket, and on 2 specific IP addresses on this host.
   137  $ sudo dockerd -H unix:///var/run/docker.sock -H tcp://192.168.59.106 -H tcp://10.10.10.2
   138  ```
   139  
   140  The Docker client will honor the `DOCKER_HOST` environment variable to set the
   141  `-H` flag for the client.
   142  
   143  ```bash
   144  $ docker -H tcp://0.0.0.0:2375 ps
   145  # or
   146  $ export DOCKER_HOST="tcp://0.0.0.0:2375"
   147  $ docker ps
   148  # both are equal
   149  ```
   150  
   151  Setting the `DOCKER_TLS_VERIFY` environment variable to any value other than
   152  the empty string is equivalent to setting the `--tlsverify` flag. The following
   153  are equivalent:
   154  
   155  ```bash
   156  $ docker --tlsverify ps
   157  # or
   158  $ export DOCKER_TLS_VERIFY=1
   159  $ docker ps
   160  ```
   161  
   162  The Docker client will honor the `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY`
   163  environment variables (or the lowercase versions thereof). `HTTPS_PROXY` takes
   164  precedence over `HTTP_PROXY`.
   165  
   166  ### Bind Docker to another host/port or a Unix socket
   167  
   168  > **Warning**:
   169  > Changing the default `docker` daemon binding to a
   170  > TCP port or Unix *docker* user group will increase your security risks
   171  > by allowing non-root users to gain *root* access on the host. Make sure
   172  > you control access to `docker`. If you are binding
   173  > to a TCP port, anyone with access to that port has full Docker access;
   174  > so it is not advisable on an open network.
   175  
   176  With `-H` it is possible to make the Docker daemon to listen on a
   177  specific IP and port. By default, it will listen on
   178  `unix:///var/run/docker.sock` to allow only local connections by the
   179  *root* user. You *could* set it to `0.0.0.0:2375` or a specific host IP
   180  to give access to everybody, but that is **not recommended** because
   181  then it is trivial for someone to gain root access to the host where the
   182  daemon is running.
   183  
   184  Similarly, the Docker client can use `-H` to connect to a custom port.
   185  The Docker client will default to connecting to `unix:///var/run/docker.sock`
   186  on Linux, and `tcp://127.0.0.1:2376` on Windows.
   187  
   188  `-H` accepts host and port assignment in the following format:
   189  
   190      tcp://[host]:[port][path] or unix://path
   191  
   192  For example:
   193  
   194  -   `tcp://` -> TCP connection to `127.0.0.1` on either port `2376` when TLS encryption
   195      is on, or port `2375` when communication is in plain text.
   196  -   `tcp://host:2375` -> TCP connection on
   197      host:2375
   198  -   `tcp://host:2375/path` -> TCP connection on
   199      host:2375 and prepend path to all requests
   200  -   `unix://path/to/socket` -> Unix socket located
   201      at `path/to/socket`
   202  
   203  `-H`, when empty, will default to the same value as
   204  when no `-H` was passed in.
   205  
   206  `-H` also accepts short form for TCP bindings: `host:` or `host:port` or `:port`
   207  
   208  Run Docker in daemon mode:
   209  
   210  ```bash
   211  $ sudo <path to>/dockerd -H 0.0.0.0:5555 &
   212  ```
   213  
   214  Download an `ubuntu` image:
   215  
   216  ```bash
   217  $ docker -H :5555 pull ubuntu
   218  ```
   219  
   220  You can use multiple `-H`, for example, if you want to listen on both
   221  TCP and a Unix socket
   222  
   223  ```bash
   224  # Run docker in daemon mode
   225  $ sudo <path to>/dockerd -H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock &
   226  # Download an ubuntu image, use default Unix socket
   227  $ docker pull ubuntu
   228  # OR use the TCP port
   229  $ docker -H tcp://127.0.0.1:2375 pull ubuntu
   230  ```
   231  
   232  ### Daemon storage-driver option
   233  
   234  The Docker daemon has support for several different image layer storage
   235  drivers: `aufs`, `devicemapper`, `btrfs`, `zfs`, `overlay` and `overlay2`.
   236  
   237  The `aufs` driver is the oldest, but is based on a Linux kernel patch-set that
   238  is unlikely to be merged into the main kernel. These are also known to cause
   239  some serious kernel crashes. However, `aufs` allows containers to share
   240  executable and shared library memory, so is a useful choice when running
   241  thousands of containers with the same program or libraries.
   242  
   243  The `devicemapper` driver uses thin provisioning and Copy on Write (CoW)
   244  snapshots. For each devicemapper graph location – typically
   245  `/var/lib/docker/devicemapper` – a thin pool is created based on two block
   246  devices, one for data and one for metadata. By default, these block devices
   247  are created automatically by using loopback mounts of automatically created
   248  sparse files. Refer to [Storage driver options](#storage-driver-options) below
   249  for a way how to customize this setup.
   250  [~jpetazzo/Resizing Docker containers with the Device Mapper plugin](http://jpetazzo.github.io/2014/01/29/docker-device-mapper-resize/)
   251  article explains how to tune your existing setup without the use of options.
   252  
   253  The `btrfs` driver is very fast for `docker build` - but like `devicemapper`
   254  does not share executable memory between devices. Use
   255  `dockerd -s btrfs -g /mnt/btrfs_partition`.
   256  
   257  The `zfs` driver is probably not as fast as `btrfs` but has a longer track record
   258  on stability. Thanks to `Single Copy ARC` shared blocks between clones will be
   259  cached only once. Use `dockerd -s zfs`. To select a different zfs filesystem
   260  set `zfs.fsname` option as described in [Storage driver options](#storage-driver-options).
   261  
   262  The `overlay` is a very fast union filesystem. It is now merged in the main
   263  Linux kernel as of [3.18.0](https://lkml.org/lkml/2014/10/26/137). `overlay`
   264  also supports page cache sharing, this means multiple containers accessing
   265  the same file can share a single page cache entry (or entries), it makes
   266  `overlay` as efficient with memory as `aufs` driver. Call
   267  `dockerd -s overlay` to use it.
   268  
   269  > **Note:**
   270  > As promising as `overlay` is, the feature is still quite young and should not
   271  > be used in production. Most notably, using `overlay` can cause excessive
   272  > inode consumption (especially as the number of images grows), as well as
   273  > being incompatible with the use of RPMs.
   274  
   275  The `overlay2` uses the same fast union filesystem but takes advantage of
   276  [additional features](https://lkml.org/lkml/2015/2/11/106) added in Linux
   277  kernel 4.0 to avoid excessive inode consumption. Call `dockerd -s overlay2`
   278  to use it.
   279  
   280  > **Note:**
   281  > Both `overlay` and `overlay2` are currently unsupported on `btrfs` or any
   282  > Copy on Write filesystem and should only be used over `ext4` partitions.
   283  
   284  ### Storage driver options
   285  
   286  Particular storage-driver can be configured with options specified with
   287  `--storage-opt` flags. Options for `devicemapper` are prefixed with `dm`,
   288  options for `zfs` start with `zfs` and options for `btrfs` start with `btrfs`.
   289  
   290  #### Devicemapper options
   291  
   292  *   `dm.thinpooldev`
   293  
   294      Specifies a custom block storage device to use for the thin pool.
   295  
   296      If using a block device for device mapper storage, it is best to use `lvm`
   297      to create and manage the thin-pool volume. This volume is then handed to Docker
   298      to exclusively create snapshot volumes needed for images and containers.
   299  
   300      Managing the thin-pool outside of Engine makes for the most feature-rich
   301      method of having Docker utilize device mapper thin provisioning as the
   302      backing storage for Docker containers. The highlights of the lvm-based
   303      thin-pool management feature include: automatic or interactive thin-pool
   304      resize support, dynamically changing thin-pool features, automatic thinp
   305      metadata checking when lvm activates the thin-pool, etc.
   306  
   307      As a fallback if no thin pool is provided, loopback files are
   308      created. Loopback is very slow, but can be used without any
   309      pre-configuration of storage. It is strongly recommended that you do
   310      not use loopback in production. Ensure your Engine daemon has a
   311      `--storage-opt dm.thinpooldev` argument provided.
   312  
   313      Example use:
   314  
   315      ```bash
   316      $ sudo dockerd --storage-opt dm.thinpooldev=/dev/mapper/thin-pool
   317      ```
   318  
   319  *   `dm.basesize`
   320  
   321      Specifies the size to use when creating the base device, which limits the
   322      size of images and containers. The default value is 10G. Note, thin devices
   323      are inherently "sparse", so a 10G device which is mostly empty doesn't use
   324      10 GB of space on the pool. However, the filesystem will use more space for
   325      the empty case the larger the device is.
   326  
   327      The base device size can be increased at daemon restart which will allow
   328      all future images and containers (based on those new images) to be of the
   329      new base device size.
   330  
   331      Example use:
   332  
   333      ```bash
   334      $ sudo dockerd --storage-opt dm.basesize=50G
   335      ```
   336  
   337      This will increase the base device size to 50G. The Docker daemon will throw an
   338      error if existing base device size is larger than 50G. A user can use
   339      this option to expand the base device size however shrinking is not permitted.
   340  
   341      This value affects the system-wide "base" empty filesystem
   342      that may already be initialized and inherited by pulled images. Typically,
   343      a change to this value requires additional steps to take effect:
   344  
   345       ```bash
   346      $ sudo service docker stop
   347      $ sudo rm -rf /var/lib/docker
   348      $ sudo service docker start
   349      ```
   350  
   351      Example use:
   352  
   353      ```bash
   354      $ sudo dockerd --storage-opt dm.basesize=20G
   355      ```
   356  
   357  *   `dm.loopdatasize`
   358  
   359      > **Note**:
   360      > This option configures devicemapper loopback, which should not
   361      > be used in production.
   362  
   363      Specifies the size to use when creating the loopback file for the
   364      "data" device which is used for the thin pool. The default size is
   365      100G. The file is sparse, so it will not initially take up this
   366      much space.
   367  
   368      Example use:
   369  
   370      ```bash
   371      $ sudo dockerd --storage-opt dm.loopdatasize=200G
   372      ```
   373  
   374  *   `dm.loopmetadatasize`
   375  
   376      > **Note**:
   377      > This option configures devicemapper loopback, which should not
   378      > be used in production.
   379  
   380      Specifies the size to use when creating the loopback file for the
   381      "metadata" device which is used for the thin pool. The default size
   382      is 2G. The file is sparse, so it will not initially take up
   383      this much space.
   384  
   385      Example use:
   386  
   387      ```bash
   388      $ sudo dockerd --storage-opt dm.loopmetadatasize=4G
   389      ```
   390  
   391  *   `dm.fs`
   392  
   393      Specifies the filesystem type to use for the base device. The supported
   394      options are "ext4" and "xfs". The default is "xfs"
   395  
   396      Example use:
   397  
   398      ```bash
   399      $ sudo dockerd --storage-opt dm.fs=ext4
   400      ```
   401  
   402  *   `dm.mkfsarg`
   403  
   404      Specifies extra mkfs arguments to be used when creating the base device.
   405  
   406      Example use:
   407  
   408      ```bash
   409      $ sudo dockerd --storage-opt "dm.mkfsarg=-O ^has_journal"
   410      ```
   411  
   412  *   `dm.mountopt`
   413  
   414      Specifies extra mount options used when mounting the thin devices.
   415  
   416      Example use:
   417  
   418      ```bash
   419      $ sudo dockerd --storage-opt dm.mountopt=nodiscard
   420      ```
   421  
   422  *   `dm.datadev`
   423  
   424      (Deprecated, use `dm.thinpooldev`)
   425  
   426      Specifies a custom blockdevice to use for data for the thin pool.
   427  
   428      If using a block device for device mapper storage, ideally both datadev and
   429      metadatadev should be specified to completely avoid using the loopback
   430      device.
   431  
   432      Example use:
   433  
   434      ```bash
   435      $ sudo dockerd \
   436            --storage-opt dm.datadev=/dev/sdb1 \
   437            --storage-opt dm.metadatadev=/dev/sdc1
   438      ```
   439  
   440  *   `dm.metadatadev`
   441  
   442      (Deprecated, use `dm.thinpooldev`)
   443  
   444      Specifies a custom blockdevice to use for metadata for the thin pool.
   445  
   446      For best performance the metadata should be on a different spindle than the
   447      data, or even better on an SSD.
   448  
   449      If setting up a new metadata pool it is required to be valid. This can be
   450      achieved by zeroing the first 4k to indicate empty metadata, like this:
   451  
   452      ```bash
   453      $ dd if=/dev/zero of=$metadata_dev bs=4096 count=1
   454      ```
   455  
   456      Example use:
   457  
   458      ```bash
   459      $ sudo dockerd \
   460            --storage-opt dm.datadev=/dev/sdb1 \
   461            --storage-opt dm.metadatadev=/dev/sdc1
   462      ```
   463  
   464  *   `dm.blocksize`
   465  
   466      Specifies a custom blocksize to use for the thin pool. The default
   467      blocksize is 64K.
   468  
   469      Example use:
   470  
   471      ```bash
   472      $ sudo dockerd --storage-opt dm.blocksize=512K
   473      ```
   474  
   475  *   `dm.blkdiscard`
   476  
   477      Enables or disables the use of blkdiscard when removing devicemapper
   478      devices. This is enabled by default (only) if using loopback devices and is
   479      required to resparsify the loopback file on image/container removal.
   480  
   481      Disabling this on loopback can lead to *much* faster container removal
   482      times, but will make the space used in `/var/lib/docker` directory not be
   483      returned to the system for other use when containers are removed.
   484  
   485      Example use:
   486  
   487      ```bash
   488      $ sudo dockerd --storage-opt dm.blkdiscard=false
   489      ```
   490  
   491  *   `dm.override_udev_sync_check`
   492  
   493      Overrides the `udev` synchronization checks between `devicemapper` and `udev`.
   494      `udev` is the device manager for the Linux kernel.
   495  
   496      To view the `udev` sync support of a Docker daemon that is using the
   497      `devicemapper` driver, run:
   498  
   499      ```bash
   500      $ docker info
   501      [...]
   502      Udev Sync Supported: true
   503      [...]
   504      ```
   505  
   506      When `udev` sync support is `true`, then `devicemapper` and udev can
   507      coordinate the activation and deactivation of devices for containers.
   508  
   509      When `udev` sync support is `false`, a race condition occurs between
   510      the`devicemapper` and `udev` during create and cleanup. The race condition
   511      results in errors and failures. (For information on these failures, see
   512      [docker#4036](https://github.com/docker/docker/issues/4036))
   513  
   514      To allow the `docker` daemon to start, regardless of `udev` sync not being
   515      supported, set `dm.override_udev_sync_check` to true:
   516  
   517      ```bash
   518      $ sudo dockerd --storage-opt dm.override_udev_sync_check=true
   519      ```
   520  
   521      When this value is `true`, the  `devicemapper` continues and simply warns
   522      you the errors are happening.
   523  
   524      > **Note:**
   525      > The ideal is to pursue a `docker` daemon and environment that does
   526      > support synchronizing with `udev`. For further discussion on this
   527      > topic, see [docker#4036](https://github.com/docker/docker/issues/4036).
   528      > Otherwise, set this flag for migrating existing Docker daemons to
   529      > a daemon with a supported environment.
   530  
   531  *   `dm.use_deferred_removal`
   532  
   533      Enables use of deferred device removal if `libdm` and the kernel driver
   534      support the mechanism.
   535  
   536      Deferred device removal means that if device is busy when devices are
   537      being removed/deactivated, then a deferred removal is scheduled on
   538      device. And devices automatically go away when last user of the device
   539      exits.
   540  
   541      For example, when a container exits, its associated thin device is removed.
   542      If that device has leaked into some other mount namespace and can't be
   543      removed, the container exit still succeeds and this option causes the
   544      system to schedule the device for deferred removal. It does not wait in a
   545      loop trying to remove a busy device.
   546  
   547      Example use:
   548  
   549      ```bash
   550      $ sudo dockerd --storage-opt dm.use_deferred_removal=true
   551      ```
   552  
   553  *   `dm.use_deferred_deletion`
   554  
   555      Enables use of deferred device deletion for thin pool devices. By default,
   556      thin pool device deletion is synchronous. Before a container is deleted,
   557      the Docker daemon removes any associated devices. If the storage driver
   558      can not remove a device, the container deletion fails and daemon returns.
   559  
   560          Error deleting container: Error response from daemon: Cannot destroy container
   561  
   562      To avoid this failure, enable both deferred device deletion and deferred
   563      device removal on the daemon.
   564  
   565      ```bash
   566      $ sudo dockerd \
   567            --storage-opt dm.use_deferred_deletion=true \
   568            --storage-opt dm.use_deferred_removal=true
   569      ```
   570  
   571      With these two options enabled, if a device is busy when the driver is
   572      deleting a container, the driver marks the device as deleted. Later, when
   573      the device isn't in use, the driver deletes it.
   574  
   575      In general it should be safe to enable this option by default. It will help
   576      when unintentional leaking of mount point happens across multiple mount
   577      namespaces.
   578  
   579  *   `dm.min_free_space`
   580  
   581      Specifies the min free space percent in a thin pool require for new device
   582      creation to succeed. This check applies to both free data space as well
   583      as free metadata space. Valid values are from 0% - 99%. Value 0% disables
   584      free space checking logic. If user does not specify a value for this option,
   585      the Engine uses a default value of 10%.
   586  
   587      Whenever a new a thin pool device is created (during `docker pull` or during
   588      container creation), the Engine checks if the minimum free space is
   589      available. If sufficient space is unavailable, then device creation fails
   590      and any relevant `docker` operation fails.
   591  
   592      To recover from this error, you must create more free space in the thin pool
   593      to recover from the error. You can create free space by deleting some images
   594      and containers from the thin pool. You can also add more storage to the thin
   595      pool.
   596  
   597      To add more space to a LVM (logical volume management) thin pool, just add
   598      more storage to the volume group container thin pool; this should automatically
   599      resolve any errors. If your configuration uses loop devices, then stop the
   600      Engine daemon, grow the size of loop files and restart the daemon to resolve
   601      the issue.
   602  
   603      Example use:
   604  
   605      ```bash
   606      $ sudo dockerd --storage-opt dm.min_free_space=10%
   607      ```
   608  
   609  *  `dm.xfs_nospace_max_retries`
   610  
   611      Specifies the maximum number of retries XFS should attempt to complete
   612      IO when ENOSPC (no space) error is returned by underlying storage device.
   613  
   614      By default XFS retries infinitely for IO to finish and this can result
   615      in unkillable process. To change this behavior one can set
   616      xfs_nospace_max_retries to say 0 and XFS will not retry IO after getting
   617      ENOSPC and will shutdown filesystem.
   618  
   619      Example use:
   620  
   621      ```bash
   622      $ sudo dockerd --storage-opt dm.xfs_nospace_max_retries=0
   623      ```
   624  
   625  #### ZFS options
   626  
   627  *   `zfs.fsname`
   628  
   629      Set zfs filesystem under which docker will create its own datasets.
   630      By default docker will pick up the zfs filesystem where docker graph
   631      (`/var/lib/docker`) is located.
   632  
   633      Example use:
   634  
   635      ```bash
   636      $ sudo dockerd -s zfs --storage-opt zfs.fsname=zroot/docker
   637      ```
   638  
   639  #### Btrfs options
   640  
   641  *   `btrfs.min_space`
   642  
   643      Specifies the minimum size to use when creating the subvolume which is used
   644      for containers. If user uses disk quota for btrfs when creating or running
   645      a container with **--storage-opt size** option, docker should ensure the
   646      **size** cannot be smaller than **btrfs.min_space**.
   647  
   648      Example use:
   649  
   650      ```bash
   651      $ sudo dockerd -s btrfs --storage-opt btrfs.min_space=10G
   652      ```
   653  
   654  #### Overlay2 options
   655  
   656  *   `overlay2.override_kernel_check`
   657  
   658      Overrides the Linux kernel version check allowing overlay2. Support for
   659      specifying multiple lower directories needed by overlay2 was added to the
   660      Linux kernel in 4.0.0. However some older kernel versions may be patched
   661      to add multiple lower directory support for OverlayFS. This option should
   662      only be used after verifying this support exists in the kernel. Applying
   663      this option on a kernel without this support will cause failures on mount.
   664  
   665  ## Docker runtime execution options
   666  
   667  The Docker daemon relies on a
   668  [OCI](https://github.com/opencontainers/runtime-spec) compliant runtime
   669  (invoked via the `containerd` daemon) as its interface to the Linux
   670  kernel `namespaces`, `cgroups`, and `SELinux`.
   671  
   672  By default, the Docker daemon automatically starts `containerd`. If you want to
   673  control `containerd` startup, manually start `containerd` and pass the path to
   674  the `containerd` socket using the `--containerd` flag. For example:
   675  
   676  ```bash
   677  $ sudo dockerd --containerd /var/run/dev/docker-containerd.sock
   678  ```
   679  
   680  Runtimes can be registered with the daemon either via the
   681  configuration file or using the `--add-runtime` command line argument.
   682  
   683  The following is an example adding 2 runtimes via the configuration:
   684  
   685  ```json
   686  "default-runtime": "runc",
   687  "runtimes": {
   688  	"runc": {
   689  		"path": "runc"
   690  	},
   691  	"custom": {
   692  		"path": "/usr/local/bin/my-runc-replacement",
   693  		"runtimeArgs": [
   694  			"--debug"
   695  		]
   696  	}
   697  }
   698  ```
   699  
   700  This is the same example via the command line:
   701  
   702  ```bash
   703  $ sudo dockerd --add-runtime runc=runc --add-runtime custom=/usr/local/bin/my-runc-replacement
   704  ```
   705  
   706  > **Note**: defining runtime arguments via the command line is not supported.
   707  
   708  ## Options for the runtime
   709  
   710  You can configure the runtime using options specified
   711  with the `--exec-opt` flag. All the flag's options have the `native` prefix. A
   712  single `native.cgroupdriver` option is available.
   713  
   714  The `native.cgroupdriver` option specifies the management of the container's
   715  cgroups. You can specify only specify `cgroupfs` or `systemd`. If you specify
   716  `systemd` and it is not available, the system errors out. If you omit the
   717  `native.cgroupdriver` option,` cgroupfs` is used.
   718  
   719  This example sets the `cgroupdriver` to `systemd`:
   720  
   721  ```bash
   722  $ sudo dockerd --exec-opt native.cgroupdriver=systemd
   723  ```
   724  
   725  Setting this option applies to all containers the daemon launches.
   726  
   727  Also Windows Container makes use of `--exec-opt` for special purpose. Docker user
   728  can specify default container isolation technology with this, for example:
   729  
   730  ```bash
   731  $ sudo dockerd --exec-opt isolation=hyperv
   732  ```
   733  
   734  Will make `hyperv` the default isolation technology on Windows. If no isolation
   735  value is specified on daemon start, on Windows client, the default is
   736  `hyperv`, and on Windows server, the default is `process`.
   737  
   738  ## Daemon DNS options
   739  
   740  To set the DNS server for all Docker containers, use:
   741  
   742  ```bash
   743  $ sudo dockerd --dns 8.8.8.8
   744  ```
   745  
   746  
   747  To set the DNS search domain for all Docker containers, use:
   748  
   749  ```bash
   750  $ sudo dockerd --dns-search example.com
   751  ```
   752  
   753  
   754  ## Insecure registries
   755  
   756  Docker considers a private registry either secure or insecure. In the rest of
   757  this section, *registry* is used for *private registry*, and `myregistry:5000`
   758  is a placeholder example for a private registry.
   759  
   760  A secure registry uses TLS and a copy of its CA certificate is placed on the
   761  Docker host at `/etc/docker/certs.d/myregistry:5000/ca.crt`. An insecure
   762  registry is either not using TLS (i.e., listening on plain text HTTP), or is
   763  using TLS with a CA certificate not known by the Docker daemon. The latter can
   764  happen when the certificate was not found under
   765  `/etc/docker/certs.d/myregistry:5000/`, or if the certificate verification
   766  failed (i.e., wrong CA).
   767  
   768  By default, Docker assumes all, but local (see local registries below),
   769  registries are secure. Communicating with an insecure registry is not possible
   770  if Docker assumes that registry is secure. In order to communicate with an
   771  insecure registry, the Docker daemon requires `--insecure-registry` in one of
   772  the following two forms:
   773  
   774  * `--insecure-registry myregistry:5000` tells the Docker daemon that
   775    myregistry:5000 should be considered insecure.
   776  * `--insecure-registry 10.1.0.0/16` tells the Docker daemon that all registries
   777    whose domain resolve to an IP address is part of the subnet described by the
   778    CIDR syntax, should be considered insecure.
   779  
   780  The flag can be used multiple times to allow multiple registries to be marked
   781  as insecure.
   782  
   783  If an insecure registry is not marked as insecure, `docker pull`,
   784  `docker push`, and `docker search` will result in an error message prompting
   785  the user to either secure or pass the `--insecure-registry` flag to the Docker
   786  daemon as described above.
   787  
   788  Local registries, whose IP address falls in the 127.0.0.0/8 range, are
   789  automatically marked as insecure as of Docker 1.3.2. It is not recommended to
   790  rely on this, as it may change in the future.
   791  
   792  Enabling `--insecure-registry`, i.e., allowing un-encrypted and/or untrusted
   793  communication, can be useful when running a local registry.  However,
   794  because its use creates security vulnerabilities it should ONLY be enabled for
   795  testing purposes.  For increased security, users should add their CA to their
   796  system's list of trusted CAs instead of enabling `--insecure-registry`.
   797  
   798  ## Legacy Registries
   799  
   800  Enabling `--disable-legacy-registry` forces a docker daemon to only interact with registries which support the V2 protocol.  Specifically, the daemon will not attempt `push`, `pull` and `login` to v1 registries.  The exception to this is `search` which can still be performed on v1 registries.
   801  
   802  ## Running a Docker daemon behind an HTTPS_PROXY
   803  
   804  When running inside a LAN that uses an `HTTPS` proxy, the Docker Hub
   805  certificates will be replaced by the proxy's certificates. These certificates
   806  need to be added to your Docker host's configuration:
   807  
   808  1. Install the `ca-certificates` package for your distribution
   809  2. Ask your network admin for the proxy's CA certificate and append them to
   810     `/etc/pki/tls/certs/ca-bundle.crt`
   811  3. Then start your Docker daemon with `HTTPS_PROXY=http://username:password@proxy:port/ dockerd`.
   812     The `username:` and `password@` are optional - and are only needed if your
   813     proxy is set up to require authentication.
   814  
   815  This will only add the proxy and authentication to the Docker daemon's requests -
   816  your `docker build`s and running containers will need extra configuration to
   817  use the proxy
   818  
   819  ## Default Ulimits
   820  
   821  `--default-ulimit` allows you to set the default `ulimit` options to use for
   822  all containers. It takes the same options as `--ulimit` for `docker run`. If
   823  these defaults are not set, `ulimit` settings will be inherited, if not set on
   824  `docker run`, from the Docker daemon. Any `--ulimit` options passed to
   825  `docker run` will overwrite these defaults.
   826  
   827  Be careful setting `nproc` with the `ulimit` flag as `nproc` is designed by Linux to
   828  set the maximum number of processes available to a user, not to a container. For details
   829  please check the [run](run.md) reference.
   830  
   831  ## Nodes discovery
   832  
   833  The `--cluster-advertise` option specifies the `host:port` or `interface:port`
   834  combination that this particular daemon instance should use when advertising
   835  itself to the cluster. The daemon is reached by remote hosts through this value.
   836  If you  specify an interface, make sure it includes the IP address of the actual
   837  Docker host. For Engine installation created through `docker-machine`, the
   838  interface is typically `eth1`.
   839  
   840  The daemon uses [libkv](https://github.com/docker/libkv/) to advertise
   841  the node within the cluster. Some key-value backends support mutual
   842  TLS. To configure the client TLS settings used by the daemon can be configured
   843  using the `--cluster-store-opt` flag, specifying the paths to PEM encoded
   844  files. For example:
   845  
   846  ```bash
   847  $ sudo dockerd \
   848      --cluster-advertise 192.168.1.2:2376 \
   849      --cluster-store etcd://192.168.1.2:2379 \
   850      --cluster-store-opt kv.cacertfile=/path/to/ca.pem \
   851      --cluster-store-opt kv.certfile=/path/to/cert.pem \
   852      --cluster-store-opt kv.keyfile=/path/to/key.pem
   853  ```
   854  
   855  The currently supported cluster store options are:
   856  
   857  *   `discovery.heartbeat`
   858  
   859      Specifies the heartbeat timer in seconds which is used by the daemon as a
   860      keepalive mechanism to make sure discovery module treats the node as alive
   861      in the cluster. If not configured, the default value is 20 seconds.
   862  
   863  *   `discovery.ttl`
   864  
   865      Specifies the ttl (time-to-live) in seconds which is used by the discovery
   866      module to timeout a node if a valid heartbeat is not received within the
   867      configured ttl value. If not configured, the default value is 60 seconds.
   868  
   869  *   `kv.cacertfile`
   870  
   871      Specifies the path to a local file with PEM encoded CA certificates to trust
   872  
   873  *   `kv.certfile`
   874  
   875      Specifies the path to a local file with a PEM encoded certificate.  This
   876      certificate is used as the client cert for communication with the
   877      Key/Value store.
   878  
   879  *   `kv.keyfile`
   880  
   881      Specifies the path to a local file with a PEM encoded private key.  This
   882      private key is used as the client key for communication with the
   883      Key/Value store.
   884  
   885  *   `kv.path`
   886  
   887      Specifies the path in the Key/Value store. If not configured, the default value is 'docker/nodes'.
   888  
   889  ## Access authorization
   890  
   891  Docker's access authorization can be extended by authorization plugins that your
   892  organization can purchase or build themselves. You can install one or more
   893  authorization plugins when you start the Docker `daemon` using the
   894  `--authorization-plugin=PLUGIN_ID` option.
   895  
   896  ```bash
   897  $ sudo dockerd --authorization-plugin=plugin1 --authorization-plugin=plugin2,...
   898  ```
   899  
   900  The `PLUGIN_ID` value is either the plugin's name or a path to its specification
   901  file. The plugin's implementation determines whether you can specify a name or
   902  path. Consult with your Docker administrator to get information about the
   903  plugins available to you.
   904  
   905  Once a plugin is installed, requests made to the `daemon` through the command
   906  line or Docker's remote API are allowed or denied by the plugin.  If you have
   907  multiple plugins installed, at least one must allow the request for it to
   908  complete.
   909  
   910  For information about how to create an authorization plugin, see [authorization
   911  plugin](../../extend/plugins_authorization.md) section in the Docker extend section of this documentation.
   912  
   913  
   914  ## Daemon user namespace options
   915  
   916  The Linux kernel [user namespace support](http://man7.org/linux/man-pages/man7/user_namespaces.7.html) provides additional security by enabling
   917  a process, and therefore a container, to have a unique range of user and
   918  group IDs which are outside the traditional user and group range utilized by
   919  the host system. Potentially the most important security improvement is that,
   920  by default, container processes running as the `root` user will have expected
   921  administrative privilege (with some restrictions) inside the container but will
   922  effectively be mapped to an unprivileged `uid` on the host.
   923  
   924  When user namespace support is enabled, Docker creates a single daemon-wide mapping
   925  for all containers running on the same engine instance. The mappings will
   926  utilize the existing subordinate user and group ID feature available on all modern
   927  Linux distributions.
   928  The [`/etc/subuid`](http://man7.org/linux/man-pages/man5/subuid.5.html) and
   929  [`/etc/subgid`](http://man7.org/linux/man-pages/man5/subgid.5.html) files will be
   930  read for the user, and optional group, specified to the `--userns-remap`
   931  parameter.  If you do not wish to specify your own user and/or group, you can
   932  provide `default` as the value to this flag, and a user will be created on your behalf
   933  and provided subordinate uid and gid ranges. This default user will be named
   934  `dockremap`, and entries will be created for it in `/etc/passwd` and
   935  `/etc/group` using your distro's standard user and group creation tools.
   936  
   937  > **Note**: The single mapping per-daemon restriction is in place for now
   938  > because Docker shares image layers from its local cache across all
   939  > containers running on the engine instance.  Since file ownership must be
   940  > the same for all containers sharing the same layer content, the decision
   941  > was made to map the file ownership on `docker pull` to the daemon's user and
   942  > group mappings so that there is no delay for running containers once the
   943  > content is downloaded. This design preserves the same performance for `docker
   944  > pull`, `docker push`, and container startup as users expect with
   945  > user namespaces disabled.
   946  
   947  ### Starting the daemon with user namespaces enabled
   948  
   949  To enable user namespace support, start the daemon with the
   950  `--userns-remap` flag, which accepts values in the following formats:
   951  
   952   - uid
   953   - uid:gid
   954   - username
   955   - username:groupname
   956  
   957  If numeric IDs are provided, translation back to valid user or group names
   958  will occur so that the subordinate uid and gid information can be read, given
   959  these resources are name-based, not id-based.  If the numeric ID information
   960  provided does not exist as entries in `/etc/passwd` or `/etc/group`, daemon
   961  startup will fail with an error message.
   962  
   963  **Example: starting with default Docker user management:**
   964  
   965  ```bash
   966  $ sudo dockerd --userns-remap=default
   967  ```
   968  
   969  When `default` is provided, Docker will create - or find the existing - user and group
   970  named `dockremap`. If the user is created, and the Linux distribution has
   971  appropriate support, the `/etc/subuid` and `/etc/subgid` files will be populated
   972  with a contiguous 65536 length range of subordinate user and group IDs, starting
   973  at an offset based on prior entries in those files.  For example, Ubuntu will
   974  create the following range, based on an existing user named `user1` already owning
   975  the first 65536 range:
   976  
   977  ```bash
   978  $ cat /etc/subuid
   979  user1:100000:65536
   980  dockremap:165536:65536
   981  ```
   982  
   983  If you have a preferred/self-managed user with subordinate ID mappings already
   984  configured, you can provide that username or uid to the `--userns-remap` flag.
   985  If you have a group that doesn't match the username, you may provide the `gid`
   986  or group name as well; otherwise the username will be used as the group name
   987  when querying the system for the subordinate group ID range.
   988  
   989  The output of `docker info` can be used to determine if the daemon is running
   990  with user namespaces enabled or not. If the daemon is configured with user
   991  namespaces, the Security Options entry in the response will list "userns" as
   992  one of the enabled security features.
   993  
   994  ### Detailed information on `subuid`/`subgid` ranges
   995  
   996  Given potential advanced use of the subordinate ID ranges by power users, the
   997  following paragraphs define how the Docker daemon currently uses the range entries
   998  found within the subordinate range files.
   999  
  1000  The simplest case is that only one contiguous range is defined for the
  1001  provided user or group. In this case, Docker will use that entire contiguous
  1002  range for the mapping of host uids and gids to the container process.  This
  1003  means that the first ID in the range will be the remapped root user, and the
  1004  IDs above that initial ID will map host ID 1 through the end of the range.
  1005  
  1006  From the example `/etc/subuid` content shown above, the remapped root
  1007  user would be uid 165536.
  1008  
  1009  If the system administrator has set up multiple ranges for a single user or
  1010  group, the Docker daemon will read all the available ranges and use the
  1011  following algorithm to create the mapping ranges:
  1012  
  1013  1. The range segments found for the particular user will be sorted by *start ID* ascending.
  1014  2. Map segments will be created from each range in increasing value with a length matching the length of each segment. Therefore the range segment with the lowest numeric starting value will be equal to the remapped root, and continue up through host uid/gid equal to the range segment length. As an example, if the lowest segment starts at ID 1000 and has a length of 100, then a map of 1000 -> 0 (the remapped root) up through 1100 -> 100 will be created from this segment. If the next segment starts at ID 10000, then the next map will start with mapping 10000 -> 101 up to the length of this second segment. This will continue until no more segments are found in the subordinate files for this user.
  1015  3. If more than five range segments exist for a single user, only the first five will be utilized, matching the kernel's limitation of only five entries in `/proc/self/uid_map` and `proc/self/gid_map`.
  1016  
  1017  ### Disable user namespace for a container
  1018  
  1019  If you enable user namespaces on the daemon, all containers are started
  1020  with user namespaces enabled. In some situations you might want to disable
  1021  this feature for a container, for example, to start a privileged container (see
  1022  [user namespace known restrictions](#user-namespace-known-restrictions)).
  1023  To enable those advanced features for a specific container use `--userns=host`
  1024  in the `run/exec/create` command.
  1025  This option will completely disable user namespace mapping for the container's user.
  1026  
  1027  ### User namespace known restrictions
  1028  
  1029  The following standard Docker features are currently incompatible when
  1030  running a Docker daemon with user namespaces enabled:
  1031  
  1032   - sharing PID or NET namespaces with the host (`--pid=host` or `--net=host`)
  1033   - Using `--privileged` mode flag on `docker run` (unless also specifying `--userns=host`)
  1034  
  1035  In general, user namespaces are an advanced feature and will require
  1036  coordination with other capabilities. For example, if volumes are mounted from
  1037  the host, file ownership will have to be pre-arranged if the user or
  1038  administrator wishes the containers to have expected access to the volume
  1039  contents. Note that when using external volume or graph driver plugins, those
  1040  external software programs must be made aware of user and group mapping ranges
  1041  if they are to work seamlessly with user namespace support.
  1042  
  1043  Finally, while the `root` user inside a user namespaced container process has
  1044  many of the expected admin privileges that go along with being the superuser, the
  1045  Linux kernel has restrictions based on internal knowledge that this is a user namespaced
  1046  process. The most notable restriction that we are aware of at this time is the
  1047  inability to use `mknod`. Permission will be denied for device creation even as
  1048  container `root` inside a user namespace.
  1049  
  1050  ## Miscellaneous options
  1051  
  1052  IP masquerading uses address translation to allow containers without a public
  1053  IP to talk to other machines on the Internet. This may interfere with some
  1054  network topologies and can be disabled with `--ip-masq=false`.
  1055  
  1056  Docker supports softlinks for the Docker data directory (`/var/lib/docker`) and
  1057  for `/var/lib/docker/tmp`. The `DOCKER_TMPDIR` and the data directory can be
  1058  set like this:
  1059  
  1060      DOCKER_TMPDIR=/mnt/disk2/tmp /usr/local/bin/dockerd -D -g /var/lib/docker -H unix:// > /var/lib/docker-machine/docker.log 2>&1
  1061      # or
  1062      export DOCKER_TMPDIR=/mnt/disk2/tmp
  1063      /usr/local/bin/dockerd -D -g /var/lib/docker -H unix:// > /var/lib/docker-machine/docker.log 2>&1
  1064  
  1065  ## Default cgroup parent
  1066  
  1067  The `--cgroup-parent` option allows you to set the default cgroup parent
  1068  to use for containers. If this option is not set, it defaults to `/docker` for
  1069  fs cgroup driver and `system.slice` for systemd cgroup driver.
  1070  
  1071  If the cgroup has a leading forward slash (`/`), the cgroup is created
  1072  under the root cgroup, otherwise the cgroup is created under the daemon
  1073  cgroup.
  1074  
  1075  Assuming the daemon is running in cgroup `daemoncgroup`,
  1076  `--cgroup-parent=/foobar` creates a cgroup in
  1077  `/sys/fs/cgroup/memory/foobar`, whereas using `--cgroup-parent=foobar`
  1078  creates the cgroup in `/sys/fs/cgroup/memory/daemoncgroup/foobar`
  1079  
  1080  The systemd cgroup driver has different rules for `--cgroup-parent`. Systemd
  1081  represents hierarchy by slice and the name of the slice encodes the location in
  1082  the tree. So `--cgroup-parent` for systemd cgroups should be a slice name. A
  1083  name can consist of a dash-separated series of names, which describes the path
  1084  to the slice from the root slice. For example, `--cgroup-parent=user-a-b.slice`
  1085  means the memory cgroup for the container is created in
  1086  `/sys/fs/cgroup/memory/user.slice/user-a.slice/user-a-b.slice/docker-<id>.scope`.
  1087  
  1088  This setting can also be set per container, using the `--cgroup-parent`
  1089  option on `docker create` and `docker run`, and takes precedence over
  1090  the `--cgroup-parent` option on the daemon.
  1091  
  1092  ## Daemon configuration file
  1093  
  1094  The `--config-file` option allows you to set any configuration option
  1095  for the daemon in a JSON format. This file uses the same flag names as keys,
  1096  except for flags that allow several entries, where it uses the plural
  1097  of the flag name, e.g., `labels` for the `label` flag.
  1098  
  1099  The options set in the configuration file must not conflict with options set
  1100  via flags. The docker daemon fails to start if an option is duplicated between
  1101  the file and the flags, regardless their value. We do this to avoid
  1102  silently ignore changes introduced in configuration reloads.
  1103  For example, the daemon fails to start if you set daemon labels
  1104  in the configuration file and also set daemon labels via the `--label` flag.
  1105  Options that are not present in the file are ignored when the daemon starts.
  1106  
  1107  ### Linux configuration file
  1108  
  1109  The default location of the configuration file on Linux is
  1110  `/etc/docker/daemon.json`. The `--config-file` flag can be used to specify a
  1111   non-default location.
  1112  
  1113  This is a full example of the allowed configuration options on Linux:
  1114  
  1115  ```json
  1116  {
  1117  	"authorization-plugins": [],
  1118  	"dns": [],
  1119  	"dns-opts": [],
  1120  	"dns-search": [],
  1121  	"exec-opts": [],
  1122  	"exec-root": "",
  1123  	"experimental": false,
  1124  	"storage-driver": "",
  1125  	"storage-opts": [],
  1126  	"labels": [],
  1127  	"live-restore": true,
  1128  	"log-driver": "",
  1129  	"log-opts": {},
  1130  	"mtu": 0,
  1131  	"pidfile": "",
  1132  	"graph": "",
  1133  	"cluster-store": "",
  1134  	"cluster-store-opts": {},
  1135  	"cluster-advertise": "",
  1136  	"max-concurrent-downloads": 3,
  1137  	"max-concurrent-uploads": 5,
  1138  	"shutdown-timeout": 15,
  1139  	"debug": true,
  1140  	"hosts": [],
  1141  	"log-level": "",
  1142  	"tls": true,
  1143  	"tlsverify": true,
  1144  	"tlscacert": "",
  1145  	"tlscert": "",
  1146  	"tlskey": "",
  1147  	"swarm-default-advertise-addr": "",
  1148  	"api-cors-header": "",
  1149  	"selinux-enabled": false,
  1150  	"userns-remap": "",
  1151  	"group": "",
  1152  	"cgroup-parent": "",
  1153  	"default-ulimits": {},
  1154  	"init": false,
  1155  	"init-path": "/usr/libexec/docker-init",
  1156  	"ipv6": false,
  1157  	"iptables": false,
  1158  	"ip-forward": false,
  1159  	"ip-masq": false,
  1160  	"userland-proxy": false,
  1161  	"userland-proxy-path": "/usr/libexec/docker-proxy",
  1162  	"ip": "0.0.0.0",
  1163  	"bridge": "",
  1164  	"bip": "",
  1165  	"fixed-cidr": "",
  1166  	"fixed-cidr-v6": "",
  1167  	"default-gateway": "",
  1168  	"default-gateway-v6": "",
  1169  	"icc": false,
  1170  	"raw-logs": false,
  1171  	"registry-mirrors": [],
  1172  	"insecure-registries": [],
  1173  	"disable-legacy-registry": false,
  1174  	"default-runtime": "runc",
  1175  	"oom-score-adjust": -500,
  1176  	"runtimes": {
  1177  		"runc": {
  1178  			"path": "runc"
  1179  		},
  1180  		"custom": {
  1181  			"path": "/usr/local/bin/my-runc-replacement",
  1182  			"runtimeArgs": [
  1183  				"--debug"
  1184  			]
  1185  		}
  1186  	}
  1187  }
  1188  ```
  1189  
  1190  ### Windows configuration file
  1191  
  1192  The default location of the configuration file on Windows is
  1193   `%programdata%\docker\config\daemon.json`. The `--config-file` flag can be
  1194   used to specify a non-default location.
  1195  
  1196  This is a full example of the allowed configuration options on Windows:
  1197  
  1198  ```json
  1199  {
  1200      "authorization-plugins": [],
  1201      "dns": [],
  1202      "dns-opts": [],
  1203      "dns-search": [],
  1204      "exec-opts": [],
  1205      "experimental": false,
  1206      "storage-driver": "",
  1207      "storage-opts": [],
  1208      "labels": [],
  1209      "live-restore": true,
  1210      "log-driver": "",
  1211      "mtu": 0,
  1212      "pidfile": "",
  1213      "graph": "",
  1214      "cluster-store": "",
  1215      "cluster-advertise": "",
  1216      "max-concurrent-downloads": 3,
  1217      "max-concurrent-uploads": 5,
  1218      "shutdown-timeout": 15,
  1219      "debug": true,
  1220      "hosts": [],
  1221      "log-level": "",
  1222      "tlsverify": true,
  1223      "tlscacert": "",
  1224      "tlscert": "",
  1225      "tlskey": "",
  1226      "swarm-default-advertise-addr": "",
  1227      "group": "",
  1228      "default-ulimits": {},
  1229      "bridge": "",
  1230      "fixed-cidr": "",
  1231      "raw-logs": false,
  1232      "registry-mirrors": [],
  1233      "insecure-registries": [],
  1234      "disable-legacy-registry": false
  1235  }
  1236  ```
  1237  
  1238  ### Configuration reloading
  1239  
  1240  Some options can be reconfigured when the daemon is running without requiring
  1241  to restart the process. We use the `SIGHUP` signal in Linux to reload, and a global event
  1242  in Windows with the key `Global\docker-daemon-config-$PID`. The options can
  1243  be modified in the configuration file but still will check for conflicts with
  1244  the provided flags. The daemon fails to reconfigure itself
  1245  if there are conflicts, but it won't stop execution.
  1246  
  1247  The list of currently supported options that can be reconfigured is this:
  1248  
  1249  - `debug`: it changes the daemon to debug mode when set to true.
  1250  - `cluster-store`: it reloads the discovery store with the new address.
  1251  - `cluster-store-opts`: it uses the new options to reload the discovery store.
  1252  - `cluster-advertise`: it modifies the address advertised after reloading.
  1253  - `labels`: it replaces the daemon labels with a new set of labels.
  1254  - `live-restore`: Enables [keeping containers alive during daemon downtime](https://docs.docker.com/engine/admin/live-restore/).
  1255  - `max-concurrent-downloads`: it updates the max concurrent downloads for each pull.
  1256  - `max-concurrent-uploads`: it updates the max concurrent uploads for each push.
  1257  - `default-runtime`: it updates the runtime to be used if not is
  1258    specified at container creation. It defaults to "default" which is
  1259    the runtime shipped with the official docker packages.
  1260  - `runtimes`: it updates the list of available OCI runtimes that can
  1261    be used to run containers
  1262  - `authorization-plugin`: specifies the authorization plugins to use.
  1263  - `insecure-registries`: it replaces the daemon insecure registries with a new set of insecure registries. If some existing insecure registries in daemon's configuration are not in newly reloaded insecure resgitries, these existing ones will be removed from daemon's config.
  1264  
  1265  Updating and reloading the cluster configurations such as `--cluster-store`,
  1266  `--cluster-advertise` and `--cluster-store-opts` will take effect only if
  1267  these configurations were not previously configured. If `--cluster-store`
  1268  has been provided in flags and `cluster-advertise` not, `cluster-advertise`
  1269  can be added in the configuration file without accompanied by `--cluster-store`.
  1270  Configuration reload will log a warning message if it detects a change in
  1271  previously configured cluster configurations.
  1272  
  1273  
  1274  ## Running multiple daemons
  1275  
  1276  > **Note:** Running multiple daemons on a single host is considered as "experimental". The user should be aware of
  1277  > unsolved problems. This solution may not work properly in some cases. Solutions are currently under development
  1278  > and will be delivered in the near future.
  1279  
  1280  This section describes how to run multiple Docker daemons on a single host. To
  1281  run multiple daemons, you must configure each daemon so that it does not
  1282  conflict with other daemons on the same host. You can set these options either
  1283  by providing them as flags, or by using a [daemon configuration file](#daemon-configuration-file).
  1284  
  1285  The following daemon options must be configured for each daemon:
  1286  
  1287  ```bash
  1288  -b, --bridge=                          Attach containers to a network bridge
  1289  --exec-root=/var/run/docker            Root of the Docker execdriver
  1290  -g, --graph=/var/lib/docker            Root of the Docker runtime
  1291  -p, --pidfile=/var/run/docker.pid      Path to use for daemon PID file
  1292  -H, --host=[]                          Daemon socket(s) to connect to
  1293  --iptables=true                        Enable addition of iptables rules
  1294  --config-file=/etc/docker/daemon.json  Daemon configuration file
  1295  --tlscacert="~/.docker/ca.pem"         Trust certs signed only by this CA
  1296  --tlscert="~/.docker/cert.pem"         Path to TLS certificate file
  1297  --tlskey="~/.docker/key.pem"           Path to TLS key file
  1298  ```
  1299  
  1300  When your daemons use different values for these flags, you can run them on the same host without any problems.
  1301  It is very important to properly understand the meaning of those options and to use them correctly.
  1302  
  1303  - The `-b, --bridge=` flag is set to `docker0` as default bridge network. It is created automatically when you install Docker.
  1304  If you are not using the default, you must create and configure the bridge manually or just set it to 'none': `--bridge=none`
  1305  - `--exec-root` is the path where the container state is stored. The default value is `/var/run/docker`. Specify the path for
  1306  your running daemon here.
  1307  - `--graph` is the path where images are stored. The default value is `/var/lib/docker`. To avoid any conflict with other daemons
  1308  set this parameter separately for each daemon.
  1309  - `-p, --pidfile=/var/run/docker.pid` is the path where the process ID of the daemon is stored. Specify the path for your
  1310  pid file here.
  1311  - `--host=[]` specifies where the Docker daemon will listen for client connections. If unspecified, it defaults to `/var/run/docker.sock`.
  1312  - `--iptables=false` prevents the Docker daemon from adding iptables rules. If
  1313    multiple daemons manage iptables rules, they may overwrite rules set by
  1314    another daemon. Be aware that disabling this option requires you to manually
  1315    add iptables rules to expose container ports.
  1316  - `--config-file=/etc/docker/daemon.json` is the path where configuration file is stored. You can use it instead of
  1317  daemon flags. Specify the path for each daemon.
  1318  - `--tls*` Docker daemon supports `--tlsverify` mode that enforces encrypted and authenticated remote connections.
  1319  The `--tls*` options enable use of specific certificates for individual daemons.
  1320  
  1321  Example script for a separate “bootstrap” instance of the Docker daemon without network:
  1322  
  1323  ```bash
  1324  $ sudo dockerd \
  1325          -H unix:///var/run/docker-bootstrap.sock \
  1326          -p /var/run/docker-bootstrap.pid \
  1327          --iptables=false \
  1328          --ip-masq=false \
  1329          --bridge=none \
  1330          --graph=/var/lib/docker-bootstrap \
  1331          --exec-root=/var/run/docker-bootstrap
  1332  ```