github.com/docker/docker-ce@v17.12.1-ce-rc2+incompatible/components/cli/docs/reference/commandline/dockerd.md (about) 1 --- 2 title: "dockerd" 3 aliases: ["/engine/reference/commandline/daemon/"] 4 description: "The daemon command description and usage" 5 keywords: "container, daemon, runtime" 6 --- 7 8 <!-- This file is maintained within the docker/cli GitHub 9 repository at https://github.com/docker/cli/. Make all 10 pull requests against that repo. If you see this file in 11 another repository, consider it read-only there, as it will 12 periodically be overwritten by the definitive file. Pull 13 requests which include edits to this file in other repositories 14 will be rejected. 15 --> 16 17 # daemon 18 19 ```markdown 20 Usage: dockerd COMMAND 21 22 A self-sufficient runtime for containers. 23 24 Options: 25 --add-runtime runtime Register an additional OCI compatible runtime (default []) 26 --allow-nondistributable-artifacts list Push nondistributable artifacts to specified registries (default []) 27 --api-cors-header string Set CORS headers in the Engine API 28 --authorization-plugin list Authorization plugins to load (default []) 29 --bip string Specify network bridge IP 30 -b, --bridge string Attach containers to a network bridge 31 --cgroup-parent string Set parent cgroup for all containers 32 --cluster-advertise string Address or interface name to advertise 33 --cluster-store string URL of the distributed storage backend 34 --cluster-store-opt map Set cluster store options (default map[]) 35 --config-file string Daemon configuration file (default "/etc/docker/daemon.json") 36 --containerd string Path to containerd socket 37 --cpu-rt-period int Limit the CPU real-time period in microseconds 38 --cpu-rt-runtime int Limit the CPU real-time runtime in microseconds 39 --data-root string Root directory of persistent Docker state (default "/var/lib/docker") 40 -D, --debug Enable debug mode 41 --default-gateway ip Container default gateway IPv4 address 42 --default-gateway-v6 ip Container default gateway IPv6 address 43 --default-runtime string Default OCI runtime for containers (default "runc") 44 --default-ulimit ulimit Default ulimits for containers (default []) 45 --dns list DNS server to use (default []) 46 --dns-opt list DNS options to use (default []) 47 --dns-search list DNS search domains to use (default []) 48 --exec-opt list Runtime execution options (default []) 49 --exec-root string Root directory for execution state files (default "/var/run/docker") 50 --experimental Enable experimental features 51 --fixed-cidr string IPv4 subnet for fixed IPs 52 --fixed-cidr-v6 string IPv6 subnet for fixed IPs 53 -G, --group string Group for the unix socket (default "docker") 54 --help Print usage 55 -H, --host list Daemon socket(s) to connect to (default []) 56 --icc Enable inter-container communication (default true) 57 --init Run an init in the container to forward signals and reap processes 58 --init-path string Path to the docker-init binary 59 --insecure-registry list Enable insecure registry communication (default []) 60 --ip ip Default IP when binding container ports (default 0.0.0.0) 61 --ip-forward Enable net.ipv4.ip_forward (default true) 62 --ip-masq Enable IP masquerading (default true) 63 --iptables Enable addition of iptables rules (default true) 64 --ipv6 Enable IPv6 networking 65 --label list Set key=value labels to the daemon (default []) 66 --live-restore Enable live restore of docker when containers are still running 67 --log-driver string Default driver for container logs (default "json-file") 68 -l, --log-level string Set the logging level ("debug", "info", "warn", "error", "fatal") (default "info") 69 --log-opt map Default log driver options for containers (default map[]) 70 --max-concurrent-downloads int Set the max concurrent downloads for each pull (default 3) 71 --max-concurrent-uploads int Set the max concurrent uploads for each push (default 5) 72 --metrics-addr string Set default address and port to serve the metrics api on 73 --mtu int Set the containers network MTU 74 --node-generic-resources list Advertise user-defined resource 75 --no-new-privileges Set no-new-privileges by default for new containers 76 --oom-score-adjust int Set the oom_score_adj for the daemon (default -500) 77 -p, --pidfile string Path to use for daemon PID file (default "/var/run/docker.pid") 78 --raw-logs Full timestamps without ANSI coloring 79 --registry-mirror list Preferred Docker registry mirror (default []) 80 --seccomp-profile string Path to seccomp profile 81 --selinux-enabled Enable selinux support 82 --shutdown-timeout int Set the default shutdown timeout (default 15) 83 -s, --storage-driver string Storage driver to use 84 --storage-opt list Storage driver options (default []) 85 --swarm-default-advertise-addr string Set default address or interface for swarm advertised address 86 --tls Use TLS; implied by --tlsverify 87 --tlscacert string Trust certs signed only by this CA (default "~/.docker/ca.pem") 88 --tlscert string Path to TLS certificate file (default "~/.docker/cert.pem") 89 --tlskey string Path to TLS key file (default ~/.docker/key.pem") 90 --tlsverify Use TLS and verify the remote 91 --userland-proxy Use userland proxy for loopback traffic (default true) 92 --userland-proxy-path string Path to the userland proxy binary 93 --userns-remap string User/Group setting for user namespaces 94 -v, --version Print version information and quit 95 ``` 96 97 Options with [] may be specified multiple times. 98 99 ## Description 100 101 `dockerd` is the persistent process that manages containers. Docker 102 uses different binaries for the daemon and client. To run the daemon you 103 type `dockerd`. 104 105 To run the daemon with debug output, use `dockerd -D` or add `"debug": true` to 106 the `daemon.json` file. 107 108 > **Note**: In Docker 1.13 and higher, enable experimental features by starting 109 > `dockerd` with the `--experimental` flag or adding `"experimental": true` to the 110 > `daemon.json` file. In earlier Docker versions, a different build was required 111 > to enable experimental features. 112 113 ## Examples 114 115 ### Daemon socket option 116 117 The Docker daemon can listen for [Docker Engine API](../api/) 118 requests via three different types of Socket: `unix`, `tcp`, and `fd`. 119 120 By default, a `unix` domain socket (or IPC socket) is created at 121 `/var/run/docker.sock`, requiring either `root` permission, or `docker` group 122 membership. 123 124 If you need to access the Docker daemon remotely, you need to enable the `tcp` 125 Socket. Beware that the default setup provides un-encrypted and 126 un-authenticated direct access to the Docker daemon - and should be secured 127 either using the [built in HTTPS encrypted socket](https://docs.docker.com/engine/security/https/), or by 128 putting a secure web proxy in front of it. You can listen on port `2375` on all 129 network interfaces with `-H tcp://0.0.0.0:2375`, or on a particular network 130 interface using its IP address: `-H tcp://192.168.59.103:2375`. It is 131 conventional to use port `2375` for un-encrypted, and port `2376` for encrypted 132 communication with the daemon. 133 134 > **Note**: If you're using an HTTPS encrypted socket, keep in mind that only 135 > TLS1.0 and greater are supported. Protocols SSLv3 and under are not 136 > supported anymore for security reasons. 137 138 On Systemd based systems, you can communicate with the daemon via 139 [Systemd socket activation](http://0pointer.de/blog/projects/socket-activation.html), 140 use `dockerd -H fd://`. Using `fd://` will work perfectly for most setups but 141 you can also specify individual sockets: `dockerd -H fd://3`. If the 142 specified socket activated files aren't found, then Docker will exit. You can 143 find examples of using Systemd socket activation with Docker and Systemd in the 144 [Docker source tree](https://github.com/docker/docker/tree/master/contrib/init/systemd/). 145 146 You can configure the Docker daemon to listen to multiple sockets at the same 147 time using multiple `-H` options: 148 149 ```bash 150 # listen using the default unix socket, and on 2 specific IP addresses on this host. 151 152 $ sudo dockerd -H unix:///var/run/docker.sock -H tcp://192.168.59.106 -H tcp://10.10.10.2 153 ``` 154 155 The Docker client will honor the `DOCKER_HOST` environment variable to set the 156 `-H` flag for the client. Use **one** of the following commands: 157 158 ```bash 159 $ docker -H tcp://0.0.0.0:2375 ps 160 ``` 161 162 ```bash 163 $ export DOCKER_HOST="tcp://0.0.0.0:2375" 164 165 $ docker ps 166 ``` 167 168 Setting the `DOCKER_TLS_VERIFY` environment variable to any value other than 169 the empty string is equivalent to setting the `--tlsverify` flag. The following 170 are equivalent: 171 172 ```bash 173 $ docker --tlsverify ps 174 # or 175 $ export DOCKER_TLS_VERIFY=1 176 $ docker ps 177 ``` 178 179 The Docker client will honor the `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` 180 environment variables (or the lowercase versions thereof). `HTTPS_PROXY` takes 181 precedence over `HTTP_PROXY`. 182 183 #### Bind Docker to another host/port or a Unix socket 184 185 > **Warning**: 186 > Changing the default `docker` daemon binding to a 187 > TCP port or Unix *docker* user group will increase your security risks 188 > by allowing non-root users to gain *root* access on the host. Make sure 189 > you control access to `docker`. If you are binding 190 > to a TCP port, anyone with access to that port has full Docker access; 191 > so it is not advisable on an open network. 192 193 With `-H` it is possible to make the Docker daemon to listen on a 194 specific IP and port. By default, it will listen on 195 `unix:///var/run/docker.sock` to allow only local connections by the 196 *root* user. You *could* set it to `0.0.0.0:2375` or a specific host IP 197 to give access to everybody, but that is **not recommended** because 198 then it is trivial for someone to gain root access to the host where the 199 daemon is running. 200 201 Similarly, the Docker client can use `-H` to connect to a custom port. 202 The Docker client will default to connecting to `unix:///var/run/docker.sock` 203 on Linux, and `tcp://127.0.0.1:2376` on Windows. 204 205 `-H` accepts host and port assignment in the following format: 206 207 tcp://[host]:[port][path] or unix://path 208 209 For example: 210 211 - `tcp://` -> TCP connection to `127.0.0.1` on either port `2376` when TLS encryption 212 is on, or port `2375` when communication is in plain text. 213 - `tcp://host:2375` -> TCP connection on 214 host:2375 215 - `tcp://host:2375/path` -> TCP connection on 216 host:2375 and prepend path to all requests 217 - `unix://path/to/socket` -> Unix socket located 218 at `path/to/socket` 219 220 `-H`, when empty, will default to the same value as 221 when no `-H` was passed in. 222 223 `-H` also accepts short form for TCP bindings: `host:` or `host:port` or `:port` 224 225 Run Docker in daemon mode: 226 227 ```bash 228 $ sudo <path to>/dockerd -H 0.0.0.0:5555 & 229 ``` 230 231 Download an `ubuntu` image: 232 233 ```bash 234 $ docker -H :5555 pull ubuntu 235 ``` 236 237 You can use multiple `-H`, for example, if you want to listen on both 238 TCP and a Unix socket 239 240 ```bash 241 # Run docker in daemon mode 242 $ sudo <path to>/dockerd -H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock & 243 # Download an ubuntu image, use default Unix socket 244 $ docker pull ubuntu 245 # OR use the TCP port 246 $ docker -H tcp://127.0.0.1:2375 pull ubuntu 247 ``` 248 249 ### Daemon storage-driver 250 251 On Linux, the Docker daemon has support for several different image layer storage 252 drivers: `aufs`, `devicemapper`, `btrfs`, `zfs`, `overlay` and `overlay2`. 253 254 The `aufs` driver is the oldest, but is based on a Linux kernel patch-set that 255 is unlikely to be merged into the main kernel. These are also known to cause 256 some serious kernel crashes. However `aufs` allows containers to share 257 executable and shared library memory, so is a useful choice when running 258 thousands of containers with the same program or libraries. 259 260 The `devicemapper` driver uses thin provisioning and Copy on Write (CoW) 261 snapshots. For each devicemapper graph location – typically 262 `/var/lib/docker/devicemapper` – a thin pool is created based on two block 263 devices, one for data and one for metadata. By default, these block devices 264 are created automatically by using loopback mounts of automatically created 265 sparse files. Refer to [Devicemapper options](#devicemapper-options) below 266 for a way how to customize this setup. 267 [~jpetazzo/Resizing Docker containers with the Device Mapper plugin](http://jpetazzo.github.io/2014/01/29/docker-device-mapper-resize/) 268 article explains how to tune your existing setup without the use of options. 269 270 The `btrfs` driver is very fast for `docker build` - but like `devicemapper` 271 does not share executable memory between devices. Use 272 `dockerd -s btrfs -g /mnt/btrfs_partition`. 273 274 The `zfs` driver is probably not as fast as `btrfs` but has a longer track record 275 on stability. Thanks to `Single Copy ARC` shared blocks between clones will be 276 cached only once. Use `dockerd -s zfs`. To select a different zfs filesystem 277 set `zfs.fsname` option as described in [ZFS options](#zfs-options). 278 279 The `overlay` is a very fast union filesystem. It is now merged in the main 280 Linux kernel as of [3.18.0](https://lkml.org/lkml/2014/10/26/137). `overlay` 281 also supports page cache sharing, this means multiple containers accessing 282 the same file can share a single page cache entry (or entries), it makes 283 `overlay` as efficient with memory as `aufs` driver. Call 284 `dockerd -s overlay` to use it. 285 286 > **Note**: As promising as `overlay` is, the feature is still quite young and 287 > should not be used in production. Most notably, using `overlay` can cause 288 > excessive inode consumption (especially as the number of images grows), as 289 > well as > being incompatible with the use of RPMs. 290 291 The `overlay2` uses the same fast union filesystem but takes advantage of 292 [additional features](https://lkml.org/lkml/2015/2/11/106) added in Linux 293 kernel 4.0 to avoid excessive inode consumption. Call `dockerd -s overlay2` 294 to use it. 295 296 > **Note**: Both `overlay` and `overlay2` are currently unsupported on `btrfs` 297 > or any Copy on Write filesystem and should only be used over `ext4` partitions. 298 299 On Windows, the Docker daemon supports a single image layer storage driver 300 depending on the image platform: `windowsfilter` for Windows images, and 301 `lcow` for Linux containers on Windows. 302 303 ### Options per storage driver 304 305 Particular storage-driver can be configured with options specified with 306 `--storage-opt` flags. Options for `devicemapper` are prefixed with `dm`, 307 options for `zfs` start with `zfs`, options for `btrfs` start with `btrfs` 308 and options for `lcow` start with `lcow`. 309 310 #### Devicemapper options 311 312 This is an example of the configuration file for devicemapper on Linux: 313 314 ```json 315 { 316 "storage-driver": "devicemapper", 317 "storage-opts": [ 318 "dm.thinpooldev=/dev/mapper/thin-pool", 319 "dm.use_deferred_deletion=true", 320 "dm.use_deferred_removal=true" 321 ] 322 } 323 ``` 324 325 ##### `dm.thinpooldev` 326 327 Specifies a custom block storage device to use for the thin pool. 328 329 If using a block device for device mapper storage, it is best to use `lvm` 330 to create and manage the thin-pool volume. This volume is then handed to Docker 331 to exclusively create snapshot volumes needed for images and containers. 332 333 Managing the thin-pool outside of Engine makes for the most feature-rich 334 method of having Docker utilize device mapper thin provisioning as the 335 backing storage for Docker containers. The highlights of the lvm-based 336 thin-pool management feature include: automatic or interactive thin-pool 337 resize support, dynamically changing thin-pool features, automatic thinp 338 metadata checking when lvm activates the thin-pool, etc. 339 340 As a fallback if no thin pool is provided, loopback files are 341 created. Loopback is very slow, but can be used without any 342 pre-configuration of storage. It is strongly recommended that you do 343 not use loopback in production. Ensure your Engine daemon has a 344 `--storage-opt dm.thinpooldev` argument provided. 345 346 ###### Example: 347 348 ```bash 349 $ sudo dockerd --storage-opt dm.thinpooldev=/dev/mapper/thin-pool 350 ``` 351 352 ##### `dm.directlvm_device` 353 354 As an alternative to providing a thin pool as above, Docker can setup a block 355 device for you. 356 357 ###### Example: 358 359 ```bash 360 $ sudo dockerd --storage-opt dm.directlvm_device=/dev/xvdf 361 ``` 362 363 ##### `dm.thinp_percent` 364 365 Sets the percentage of passed in block device to use for storage. 366 367 ###### Example: 368 369 ```bash 370 $ sudo dockerd --storage-opt dm.thinp_percent=95 371 ``` 372 373 ##### `dm.thinp_metapercent` 374 375 Sets the percentage of the passed in block device to use for metadata storage. 376 377 ###### Example: 378 379 ```bash 380 $ sudo dockerd --storage-opt dm.thinp_metapercent=1 381 ``` 382 383 ##### `dm.thinp_autoextend_threshold` 384 385 Sets the value of the percentage of space used before `lvm` attempts to 386 autoextend the available space [100 = disabled] 387 388 ###### Example: 389 390 ```bash 391 $ sudo dockerd --storage-opt dm.thinp_autoextend_threshold=80 392 ``` 393 394 ##### `dm.thinp_autoextend_percent` 395 396 Sets the value percentage value to increase the thin pool by when `lvm` 397 attempts to autoextend the available space [100 = disabled] 398 399 ###### Example: 400 401 ```bash 402 $ sudo dockerd --storage-opt dm.thinp_autoextend_percent=20 403 ``` 404 405 406 ##### `dm.basesize` 407 408 Specifies the size to use when creating the base device, which limits the 409 size of images and containers. The default value is 10G. Note, thin devices 410 are inherently "sparse", so a 10G device which is mostly empty doesn't use 411 10 GB of space on the pool. However, the filesystem will use more space for 412 the empty case the larger the device is. 413 414 The base device size can be increased at daemon restart which will allow 415 all future images and containers (based on those new images) to be of the 416 new base device size. 417 418 ###### Examples 419 420 ```bash 421 $ sudo dockerd --storage-opt dm.basesize=50G 422 ``` 423 424 This will increase the base device size to 50G. The Docker daemon will throw an 425 error if existing base device size is larger than 50G. A user can use 426 this option to expand the base device size however shrinking is not permitted. 427 428 This value affects the system-wide "base" empty filesystem 429 that may already be initialized and inherited by pulled images. Typically, 430 a change to this value requires additional steps to take effect: 431 432 ```bash 433 $ sudo service docker stop 434 435 $ sudo rm -rf /var/lib/docker 436 437 $ sudo service docker start 438 ``` 439 440 441 ##### `dm.loopdatasize` 442 443 > **Note**: This option configures devicemapper loopback, which should not 444 > be used in production. 445 446 Specifies the size to use when creating the loopback file for the 447 "data" device which is used for the thin pool. The default size is 448 100G. The file is sparse, so it will not initially take up this 449 much space. 450 451 ###### Example 452 453 ```bash 454 $ sudo dockerd --storage-opt dm.loopdatasize=200G 455 ``` 456 457 ##### `dm.loopmetadatasize` 458 459 > **Note**: This option configures devicemapper loopback, which should not 460 > be used in production. 461 462 Specifies the size to use when creating the loopback file for the 463 "metadata" device which is used for the thin pool. The default size 464 is 2G. The file is sparse, so it will not initially take up 465 this much space. 466 467 ###### Example 468 469 ```bash 470 $ sudo dockerd --storage-opt dm.loopmetadatasize=4G 471 ``` 472 473 ##### `dm.fs` 474 475 Specifies the filesystem type to use for the base device. The supported 476 options are "ext4" and "xfs". The default is "xfs" 477 478 ###### Example 479 480 ```bash 481 $ sudo dockerd --storage-opt dm.fs=ext4 482 ``` 483 484 ##### `dm.mkfsarg` 485 486 Specifies extra mkfs arguments to be used when creating the base device. 487 488 ###### Example 489 490 ```bash 491 $ sudo dockerd --storage-opt "dm.mkfsarg=-O ^has_journal" 492 ``` 493 494 ##### `dm.mountopt` 495 496 Specifies extra mount options used when mounting the thin devices. 497 498 ###### Example 499 500 ```bash 501 $ sudo dockerd --storage-opt dm.mountopt=nodiscard 502 ``` 503 504 ##### `dm.datadev` 505 506 (Deprecated, use `dm.thinpooldev`) 507 508 Specifies a custom blockdevice to use for data for the thin pool. 509 510 If using a block device for device mapper storage, ideally both `datadev` and 511 `metadatadev` should be specified to completely avoid using the loopback 512 device. 513 514 ###### Example 515 516 ```bash 517 $ sudo dockerd \ 518 --storage-opt dm.datadev=/dev/sdb1 \ 519 --storage-opt dm.metadatadev=/dev/sdc1 520 ``` 521 522 ##### `dm.metadatadev` 523 524 (Deprecated, use `dm.thinpooldev`) 525 526 Specifies a custom blockdevice to use for metadata for the thin pool. 527 528 For best performance the metadata should be on a different spindle than the 529 data, or even better on an SSD. 530 531 If setting up a new metadata pool it is required to be valid. This can be 532 achieved by zeroing the first 4k to indicate empty metadata, like this: 533 534 ```bash 535 $ dd if=/dev/zero of=$metadata_dev bs=4096 count=1 536 ``` 537 538 ###### Example 539 540 ```bash 541 $ sudo dockerd \ 542 --storage-opt dm.datadev=/dev/sdb1 \ 543 --storage-opt dm.metadatadev=/dev/sdc1 544 ``` 545 546 ##### `dm.blocksize` 547 548 Specifies a custom blocksize to use for the thin pool. The default 549 blocksize is 64K. 550 551 ###### Example 552 553 ```bash 554 $ sudo dockerd --storage-opt dm.blocksize=512K 555 ``` 556 557 ##### `dm.blkdiscard` 558 559 Enables or disables the use of `blkdiscard` when removing devicemapper 560 devices. This is enabled by default (only) if using loopback devices and is 561 required to resparsify the loopback file on image/container removal. 562 563 Disabling this on loopback can lead to *much* faster container removal 564 times, but will make the space used in `/var/lib/docker` directory not be 565 returned to the system for other use when containers are removed. 566 567 ###### Examples 568 569 ```bash 570 $ sudo dockerd --storage-opt dm.blkdiscard=false 571 ``` 572 573 ##### `dm.override_udev_sync_check` 574 575 Overrides the `udev` synchronization checks between `devicemapper` and `udev`. 576 `udev` is the device manager for the Linux kernel. 577 578 To view the `udev` sync support of a Docker daemon that is using the 579 `devicemapper` driver, run: 580 581 ```bash 582 $ docker info 583 [...] 584 Udev Sync Supported: true 585 [...] 586 ``` 587 588 When `udev` sync support is `true`, then `devicemapper` and udev can 589 coordinate the activation and deactivation of devices for containers. 590 591 When `udev` sync support is `false`, a race condition occurs between 592 the`devicemapper` and `udev` during create and cleanup. The race condition 593 results in errors and failures. (For information on these failures, see 594 [docker#4036](https://github.com/docker/docker/issues/4036)) 595 596 To allow the `docker` daemon to start, regardless of `udev` sync not being 597 supported, set `dm.override_udev_sync_check` to true: 598 599 ```bash 600 $ sudo dockerd --storage-opt dm.override_udev_sync_check=true 601 ``` 602 603 When this value is `true`, the `devicemapper` continues and simply warns 604 you the errors are happening. 605 606 > **Note**: The ideal is to pursue a `docker` daemon and environment that does 607 > support synchronizing with `udev`. For further discussion on this 608 > topic, see [docker#4036](https://github.com/docker/docker/issues/4036). 609 > Otherwise, set this flag for migrating existing Docker daemons to 610 > a daemon with a supported environment. 611 612 ##### `dm.use_deferred_removal` 613 614 Enables use of deferred device removal if `libdm` and the kernel driver 615 support the mechanism. 616 617 Deferred device removal means that if device is busy when devices are 618 being removed/deactivated, then a deferred removal is scheduled on 619 device. And devices automatically go away when last user of the device 620 exits. 621 622 For example, when a container exits, its associated thin device is removed. 623 If that device has leaked into some other mount namespace and can't be 624 removed, the container exit still succeeds and this option causes the 625 system to schedule the device for deferred removal. It does not wait in a 626 loop trying to remove a busy device. 627 628 ###### Example 629 630 ```bash 631 $ sudo dockerd --storage-opt dm.use_deferred_removal=true 632 ``` 633 634 ##### `dm.use_deferred_deletion` 635 636 Enables use of deferred device deletion for thin pool devices. By default, 637 thin pool device deletion is synchronous. Before a container is deleted, 638 the Docker daemon removes any associated devices. If the storage driver 639 can not remove a device, the container deletion fails and daemon returns. 640 641 ```none 642 Error deleting container: Error response from daemon: Cannot destroy container 643 ``` 644 645 To avoid this failure, enable both deferred device deletion and deferred 646 device removal on the daemon. 647 648 ```bash 649 $ sudo dockerd \ 650 --storage-opt dm.use_deferred_deletion=true \ 651 --storage-opt dm.use_deferred_removal=true 652 ``` 653 654 With these two options enabled, if a device is busy when the driver is 655 deleting a container, the driver marks the device as deleted. Later, when 656 the device isn't in use, the driver deletes it. 657 658 In general it should be safe to enable this option by default. It will help 659 when unintentional leaking of mount point happens across multiple mount 660 namespaces. 661 662 ##### `dm.min_free_space` 663 664 Specifies the min free space percent in a thin pool require for new device 665 creation to succeed. This check applies to both free data space as well 666 as free metadata space. Valid values are from 0% - 99%. Value 0% disables 667 free space checking logic. If user does not specify a value for this option, 668 the Engine uses a default value of 10%. 669 670 Whenever a new a thin pool device is created (during `docker pull` or during 671 container creation), the Engine checks if the minimum free space is 672 available. If sufficient space is unavailable, then device creation fails 673 and any relevant `docker` operation fails. 674 675 To recover from this error, you must create more free space in the thin pool 676 to recover from the error. You can create free space by deleting some images 677 and containers from the thin pool. You can also add more storage to the thin 678 pool. 679 680 To add more space to a LVM (logical volume management) thin pool, just add 681 more storage to the volume group container thin pool; this should automatically 682 resolve any errors. If your configuration uses loop devices, then stop the 683 Engine daemon, grow the size of loop files and restart the daemon to resolve 684 the issue. 685 686 ###### Example 687 688 ```bash 689 $ sudo dockerd --storage-opt dm.min_free_space=10% 690 ``` 691 692 ##### `dm.xfs_nospace_max_retries` 693 694 Specifies the maximum number of retries XFS should attempt to complete 695 IO when ENOSPC (no space) error is returned by underlying storage device. 696 697 By default XFS retries infinitely for IO to finish and this can result 698 in unkillable process. To change this behavior one can set 699 xfs_nospace_max_retries to say 0 and XFS will not retry IO after getting 700 ENOSPC and will shutdown filesystem. 701 702 ###### Example 703 704 ```bash 705 $ sudo dockerd --storage-opt dm.xfs_nospace_max_retries=0 706 ``` 707 708 ##### `dm.libdm_log_level` 709 710 Specifies the maxmimum `libdm` log level that will be forwarded to the 711 `dockerd` log (as specified by `--log-level`). This option is primarily 712 intended for debugging problems involving `libdm`. Using values other than the 713 defaults may cause false-positive warnings to be logged. 714 715 Values specified must fall within the range of valid `libdm` log levels. At the 716 time of writing, the following is the list of `libdm` log levels as well as 717 their corresponding levels when output by `dockerd`. 718 719 | `libdm` Level | Value | `--log-level` | 720 | ------------- | -----:| ------------- | 721 | `_LOG_FATAL` | 2 | error | 722 | `_LOG_ERR` | 3 | error | 723 | `_LOG_WARN` | 4 | warn | 724 | `_LOG_NOTICE` | 5 | info | 725 | `_LOG_INFO` | 6 | info | 726 | `_LOG_DEBUG` | 7 | debug | 727 728 ###### Example 729 730 ```bash 731 $ sudo dockerd \ 732 --log-level debug \ 733 --storage-opt dm.libdm_log_level=7 734 ``` 735 736 #### ZFS options 737 738 ##### `zfs.fsname` 739 740 Set zfs filesystem under which docker will create its own datasets. 741 By default docker will pick up the zfs filesystem where docker graph 742 (`/var/lib/docker`) is located. 743 744 ###### Example 745 746 ```bash 747 $ sudo dockerd -s zfs --storage-opt zfs.fsname=zroot/docker 748 ``` 749 750 #### Btrfs options 751 752 ##### `btrfs.min_space` 753 754 Specifies the minimum size to use when creating the subvolume which is used 755 for containers. If user uses disk quota for btrfs when creating or running 756 a container with **--storage-opt size** option, docker should ensure the 757 **size** cannot be smaller than **btrfs.min_space**. 758 759 ###### Example 760 761 ```bash 762 $ sudo dockerd -s btrfs --storage-opt btrfs.min_space=10G 763 ``` 764 765 #### Overlay2 options 766 767 ##### `overlay2.override_kernel_check` 768 769 Overrides the Linux kernel version check allowing overlay2. Support for 770 specifying multiple lower directories needed by overlay2 was added to the 771 Linux kernel in 4.0.0. However, some older kernel versions may be patched 772 to add multiple lower directory support for OverlayFS. This option should 773 only be used after verifying this support exists in the kernel. Applying 774 this option on a kernel without this support will cause failures on mount. 775 776 ##### `overlay2.size` 777 778 Sets the default max size of the container. It is supported only when the 779 backing fs is `xfs` and mounted with `pquota` mount option. Under these 780 conditions the user can pass any size less then the backing fs size. 781 782 ###### Example 783 784 ```bash 785 $ sudo dockerd -s overlay2 --storage-opt overlay2.size=1G 786 ``` 787 788 789 #### Windowsfilter options 790 791 ##### `size` 792 793 Specifies the size to use when creating the sandbox which is used for containers. 794 Defaults to 20G. 795 796 ###### Example 797 798 ```PowerShell 799 C:\> dockerd --storage-opt size=40G 800 ``` 801 802 #### LCOW (Linux Containers on Windows) options 803 804 ##### `lcow.globalmode` 805 806 Specifies whether the daemon instantiates utility VM instances as required 807 (recommended and default if omitted), or uses single global utility VM (better 808 performance, but has security implications and not recommended for production 809 deployments). 810 811 ###### Example 812 813 ```PowerShell 814 C:\> dockerd --storage-opt lcow.globalmode=false 815 ``` 816 817 ##### `lcow.kirdpath` 818 819 Specifies the folder path to the location of a pair of kernel and initrd files 820 used for booting a utility VM. Defaults to `%ProgramFiles%\Linux Containers`. 821 822 ###### Example 823 824 ```PowerShell 825 C:\> dockerd --storage-opt lcow.kirdpath=c:\path\to\files 826 ``` 827 828 ##### `lcow.kernel` 829 830 Specifies the filename of a kernel file located in the `lcow.kirdpath` path. 831 Defaults to `bootx64.efi`. 832 833 ###### Example 834 835 ```PowerShell 836 C:\> dockerd --storage-opt lcow.kernel=kernel.efi 837 ``` 838 839 ##### `lcow.initrd` 840 841 Specifies the filename of an initrd file located in the `lcow.kirdpath` path. 842 Defaults to `initrd.img`. 843 844 ###### Example 845 846 ```PowerShell 847 C:\> dockerd --storage-opt lcow.initrd=myinitrd.img 848 ``` 849 850 ##### `lcow.bootparameters` 851 852 Specifies additional boot parameters for booting utility VMs when in kernel/ 853 initrd mode. Ignored if the utility VM is booting from VHD. These settings 854 are kernel specific. 855 856 ###### Example 857 858 ```PowerShell 859 C:\> dockerd --storage-opt "lcow.bootparameters='option=value'" 860 ``` 861 862 ##### `lcow.vhdx` 863 864 Specifies a custom VHDX to boot a utility VM, as an alternate to kernel 865 and initrd booting. Defaults to `uvm.vhdx` under `lcow.kirdpath`. 866 867 ###### Example 868 869 ```PowerShell 870 C:\> dockerd --storage-opt lcow.vhdx=custom.vhdx 871 ``` 872 873 ##### `lcow.timeout` 874 875 Specifies the timeout for utility VM operations in seconds. Defaults 876 to 300. 877 878 ###### Example 879 880 ```PowerShell 881 C:\> dockerd --storage-opt lcow.timeout=240 882 ``` 883 884 ##### `lcow.sandboxsize` 885 886 Specifies the size in GB to use when creating the sandbox which is used for 887 containers. Defaults to 20. Cannot be less than 20. 888 889 ###### Example 890 891 ```PowerShell 892 C:\> dockerd --storage-opt lcow.sandboxsize=40 893 ``` 894 895 ### Docker runtime execution options 896 897 The Docker daemon relies on a 898 [OCI](https://github.com/opencontainers/runtime-spec) compliant runtime 899 (invoked via the `containerd` daemon) as its interface to the Linux 900 kernel `namespaces`, `cgroups`, and `SELinux`. 901 902 By default, the Docker daemon automatically starts `containerd`. If you want to 903 control `containerd` startup, manually start `containerd` and pass the path to 904 the `containerd` socket using the `--containerd` flag. For example: 905 906 ```bash 907 $ sudo dockerd --containerd /var/run/dev/docker-containerd.sock 908 ``` 909 910 Runtimes can be registered with the daemon either via the 911 configuration file or using the `--add-runtime` command line argument. 912 913 The following is an example adding 2 runtimes via the configuration: 914 915 ```json 916 { 917 "default-runtime": "runc", 918 "runtimes": { 919 "runc": { 920 "path": "runc" 921 }, 922 "custom": { 923 "path": "/usr/local/bin/my-runc-replacement", 924 "runtimeArgs": [ 925 "--debug" 926 ] 927 } 928 } 929 } 930 ``` 931 932 This is the same example via the command line: 933 934 ```bash 935 $ sudo dockerd --add-runtime runc=runc --add-runtime custom=/usr/local/bin/my-runc-replacement 936 ``` 937 938 > **Note**: Defining runtime arguments via the command line is not supported. 939 940 #### Options for the runtime 941 942 You can configure the runtime using options specified 943 with the `--exec-opt` flag. All the flag's options have the `native` prefix. A 944 single `native.cgroupdriver` option is available. 945 946 The `native.cgroupdriver` option specifies the management of the container's 947 cgroups. You can only specify `cgroupfs` or `systemd`. If you specify 948 `systemd` and it is not available, the system errors out. If you omit the 949 `native.cgroupdriver` option,` cgroupfs` is used. 950 951 This example sets the `cgroupdriver` to `systemd`: 952 953 ```bash 954 $ sudo dockerd --exec-opt native.cgroupdriver=systemd 955 ``` 956 957 Setting this option applies to all containers the daemon launches. 958 959 Also Windows Container makes use of `--exec-opt` for special purpose. Docker user 960 can specify default container isolation technology with this, for example: 961 962 ```console 963 > dockerd --exec-opt isolation=hyperv 964 ``` 965 966 Will make `hyperv` the default isolation technology on Windows. If no isolation 967 value is specified on daemon start, on Windows client, the default is 968 `hyperv`, and on Windows server, the default is `process`. 969 970 ### Daemon DNS options 971 972 To set the DNS server for all Docker containers, use: 973 974 ```bash 975 $ sudo dockerd --dns 8.8.8.8 976 ``` 977 978 To set the DNS search domain for all Docker containers, use: 979 980 ```bash 981 $ sudo dockerd --dns-search example.com 982 ``` 983 984 ### Allow push of nondistributable artifacts 985 986 Some images (e.g., Windows base images) contain artifacts whose distribution is 987 restricted by license. When these images are pushed to a registry, restricted 988 artifacts are not included. 989 990 To override this behavior for specific registries, use the 991 `--allow-nondistributable-artifacts` option in one of the following forms: 992 993 * `--allow-nondistributable-artifacts myregistry:5000` tells the Docker daemon 994 to push nondistributable artifacts to myregistry:5000. 995 * `--allow-nondistributable-artifacts 10.1.0.0/16` tells the Docker daemon to 996 push nondistributable artifacts to all registries whose resolved IP address 997 is within the subnet described by the CIDR syntax. 998 999 This option can be used multiple times. 1000 1001 This option is useful when pushing images containing nondistributable artifacts 1002 to a registry on an air-gapped network so hosts on that network can pull the 1003 images without connecting to another server. 1004 1005 > **Warning**: Nondistributable artifacts typically have restrictions on how 1006 > and where they can be distributed and shared. Only use this feature to push 1007 > artifacts to private registries and ensure that you are in compliance with 1008 > any terms that cover redistributing nondistributable artifacts. 1009 1010 ### Insecure registries 1011 1012 Docker considers a private registry either secure or insecure. In the rest of 1013 this section, *registry* is used for *private registry*, and `myregistry:5000` 1014 is a placeholder example for a private registry. 1015 1016 A secure registry uses TLS and a copy of its CA certificate is placed on the 1017 Docker host at `/etc/docker/certs.d/myregistry:5000/ca.crt`. An insecure 1018 registry is either not using TLS (i.e., listening on plain text HTTP), or is 1019 using TLS with a CA certificate not known by the Docker daemon. The latter can 1020 happen when the certificate was not found under 1021 `/etc/docker/certs.d/myregistry:5000/`, or if the certificate verification 1022 failed (i.e., wrong CA). 1023 1024 By default, Docker assumes all, but local (see local registries below), 1025 registries are secure. Communicating with an insecure registry is not possible 1026 if Docker assumes that registry is secure. In order to communicate with an 1027 insecure registry, the Docker daemon requires `--insecure-registry` in one of 1028 the following two forms: 1029 1030 * `--insecure-registry myregistry:5000` tells the Docker daemon that 1031 myregistry:5000 should be considered insecure. 1032 * `--insecure-registry 10.1.0.0/16` tells the Docker daemon that all registries 1033 whose domain resolve to an IP address is part of the subnet described by the 1034 CIDR syntax, should be considered insecure. 1035 1036 The flag can be used multiple times to allow multiple registries to be marked 1037 as insecure. 1038 1039 If an insecure registry is not marked as insecure, `docker pull`, 1040 `docker push`, and `docker search` will result in an error message prompting 1041 the user to either secure or pass the `--insecure-registry` flag to the Docker 1042 daemon as described above. 1043 1044 Local registries, whose IP address falls in the 127.0.0.0/8 range, are 1045 automatically marked as insecure as of Docker 1.3.2. It is not recommended to 1046 rely on this, as it may change in the future. 1047 1048 Enabling `--insecure-registry`, i.e., allowing un-encrypted and/or untrusted 1049 communication, can be useful when running a local registry. However, 1050 because its use creates security vulnerabilities it should ONLY be enabled for 1051 testing purposes. For increased security, users should add their CA to their 1052 system's list of trusted CAs instead of enabling `--insecure-registry`. 1053 1054 #### Legacy Registries 1055 1056 Starting with Docker 17.12, operations against registries supporting only the 1057 legacy v1 protocol are no longer supported. Specifically, the daemon will not 1058 attempt `push`, `pull` and `login` to v1 registries. The exception to this is 1059 `search` which can still be performed on v1 registries. 1060 1061 The `disable-legacy-registry` configuration option has been removed and, when 1062 used, will produce an error on daemon startup. 1063 1064 1065 ### Running a Docker daemon behind an HTTPS_PROXY 1066 1067 When running inside a LAN that uses an `HTTPS` proxy, the Docker Hub 1068 certificates will be replaced by the proxy's certificates. These certificates 1069 need to be added to your Docker host's configuration: 1070 1071 1. Install the `ca-certificates` package for your distribution 1072 2. Ask your network admin for the proxy's CA certificate and append them to 1073 `/etc/pki/tls/certs/ca-bundle.crt` 1074 3. Then start your Docker daemon with `HTTPS_PROXY=http://username:password@proxy:port/ dockerd`. 1075 The `username:` and `password@` are optional - and are only needed if your 1076 proxy is set up to require authentication. 1077 1078 This will only add the proxy and authentication to the Docker daemon's requests - 1079 your `docker build`s and running containers will need extra configuration to 1080 use the proxy 1081 1082 ### Default `ulimit` settings 1083 1084 `--default-ulimit` allows you to set the default `ulimit` options to use for 1085 all containers. It takes the same options as `--ulimit` for `docker run`. If 1086 these defaults are not set, `ulimit` settings will be inherited, if not set on 1087 `docker run`, from the Docker daemon. Any `--ulimit` options passed to 1088 `docker run` will overwrite these defaults. 1089 1090 Be careful setting `nproc` with the `ulimit` flag as `nproc` is designed by Linux to 1091 set the maximum number of processes available to a user, not to a container. For details 1092 please check the [run](run.md) reference. 1093 1094 ### Node discovery 1095 1096 The `--cluster-advertise` option specifies the `host:port` or `interface:port` 1097 combination that this particular daemon instance should use when advertising 1098 itself to the cluster. The daemon is reached by remote hosts through this value. 1099 If you specify an interface, make sure it includes the IP address of the actual 1100 Docker host. For Engine installation created through `docker-machine`, the 1101 interface is typically `eth1`. 1102 1103 The daemon uses [libkv](https://github.com/docker/libkv/) to advertise 1104 the node within the cluster. Some key-value backends support mutual 1105 TLS. To configure the client TLS settings used by the daemon can be configured 1106 using the `--cluster-store-opt` flag, specifying the paths to PEM encoded 1107 files. For example: 1108 1109 ```bash 1110 $ sudo dockerd \ 1111 --cluster-advertise 192.168.1.2:2376 \ 1112 --cluster-store etcd://192.168.1.2:2379 \ 1113 --cluster-store-opt kv.cacertfile=/path/to/ca.pem \ 1114 --cluster-store-opt kv.certfile=/path/to/cert.pem \ 1115 --cluster-store-opt kv.keyfile=/path/to/key.pem 1116 ``` 1117 1118 The currently supported cluster store options are: 1119 1120 | Option | Description | 1121 |:----------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| 1122 | `discovery.heartbeat` | Specifies the heartbeat timer in seconds which is used by the daemon as a `keepalive` mechanism to make sure discovery module treats the node as alive in the cluster. If not configured, the default value is 20 seconds. | 1123 | `discovery.ttl` | Specifies the TTL (time-to-live) in seconds which is used by the discovery module to timeout a node if a valid heartbeat is not received within the configured ttl value. If not configured, the default value is 60 seconds. | 1124 | `kv.cacertfile` | Specifies the path to a local file with PEM encoded CA certificates to trust. | 1125 | `kv.certfile` | Specifies the path to a local file with a PEM encoded certificate. This certificate is used as the client cert for communication with the Key/Value store. | 1126 | `kv.keyfile` | Specifies the path to a local file with a PEM encoded private key. This private key is used as the client key for communication with the Key/Value store. | 1127 | `kv.path` | Specifies the path in the Key/Value store. If not configured, the default value is 'docker/nodes'. | 1128 1129 ### Access authorization 1130 1131 Docker's access authorization can be extended by authorization plugins that your 1132 organization can purchase or build themselves. You can install one or more 1133 authorization plugins when you start the Docker `daemon` using the 1134 `--authorization-plugin=PLUGIN_ID` option. 1135 1136 ```bash 1137 $ sudo dockerd --authorization-plugin=plugin1 --authorization-plugin=plugin2,... 1138 ``` 1139 1140 The `PLUGIN_ID` value is either the plugin's name or a path to its specification 1141 file. The plugin's implementation determines whether you can specify a name or 1142 path. Consult with your Docker administrator to get information about the 1143 plugins available to you. 1144 1145 Once a plugin is installed, requests made to the `daemon` through the 1146 command line or Docker's Engine API are allowed or denied by the plugin. 1147 If you have multiple plugins installed, each plugin, in order, must 1148 allow the request for it to complete. 1149 1150 For information about how to create an authorization plugin, see [authorization 1151 plugin](../../extend/plugins_authorization.md) section in the Docker extend section of this documentation. 1152 1153 1154 ### Daemon user namespace options 1155 1156 The Linux kernel 1157 [user namespace support](http://man7.org/linux/man-pages/man7/user_namespaces.7.html) 1158 provides additional security by enabling a process, and therefore a container, 1159 to have a unique range of user and group IDs which are outside the traditional 1160 user and group range utilized by the host system. Potentially the most important 1161 security improvement is that, by default, container processes running as the 1162 `root` user will have expected administrative privilege (with some restrictions) 1163 inside the container but will effectively be mapped to an unprivileged `uid` on 1164 the host. 1165 1166 For details about how to use this feature, as well as limitations, see 1167 [Isolate containers with a user namespace](https://docs.docker.com/engine/security/userns-remap/). 1168 1169 ### Miscellaneous options 1170 1171 IP masquerading uses address translation to allow containers without a public 1172 IP to talk to other machines on the Internet. This may interfere with some 1173 network topologies and can be disabled with `--ip-masq=false`. 1174 1175 Docker supports softlinks for the Docker data directory (`/var/lib/docker`) and 1176 for `/var/lib/docker/tmp`. The `DOCKER_TMPDIR` and the data directory can be 1177 set like this: 1178 1179 DOCKER_TMPDIR=/mnt/disk2/tmp /usr/local/bin/dockerd -D -g /var/lib/docker -H unix:// > /var/lib/docker-machine/docker.log 2>&1 1180 # or 1181 export DOCKER_TMPDIR=/mnt/disk2/tmp 1182 /usr/local/bin/dockerd -D -g /var/lib/docker -H unix:// > /var/lib/docker-machine/docker.log 2>&1 1183 1184 #### Default cgroup parent 1185 1186 The `--cgroup-parent` option allows you to set the default cgroup parent 1187 to use for containers. If this option is not set, it defaults to `/docker` for 1188 fs cgroup driver and `system.slice` for systemd cgroup driver. 1189 1190 If the cgroup has a leading forward slash (`/`), the cgroup is created 1191 under the root cgroup, otherwise the cgroup is created under the daemon 1192 cgroup. 1193 1194 Assuming the daemon is running in cgroup `daemoncgroup`, 1195 `--cgroup-parent=/foobar` creates a cgroup in 1196 `/sys/fs/cgroup/memory/foobar`, whereas using `--cgroup-parent=foobar` 1197 creates the cgroup in `/sys/fs/cgroup/memory/daemoncgroup/foobar` 1198 1199 The systemd cgroup driver has different rules for `--cgroup-parent`. Systemd 1200 represents hierarchy by slice and the name of the slice encodes the location in 1201 the tree. So `--cgroup-parent` for systemd cgroups should be a slice name. A 1202 name can consist of a dash-separated series of names, which describes the path 1203 to the slice from the root slice. For example, `--cgroup-parent=user-a-b.slice` 1204 means the memory cgroup for the container is created in 1205 `/sys/fs/cgroup/memory/user.slice/user-a.slice/user-a-b.slice/docker-<id>.scope`. 1206 1207 This setting can also be set per container, using the `--cgroup-parent` 1208 option on `docker create` and `docker run`, and takes precedence over 1209 the `--cgroup-parent` option on the daemon. 1210 1211 #### Daemon metrics 1212 1213 The `--metrics-addr` option takes a tcp address to serve the metrics API. 1214 This feature is still experimental, therefore, the daemon must be running in experimental 1215 mode for this feature to work. 1216 1217 To serve the metrics API on localhost:1337 you would specify `--metrics-addr 127.0.0.1:1337` 1218 allowing you to make requests on the API at `127.0.0.1:1337/metrics` to receive metrics in the 1219 [prometheus](https://prometheus.io/docs/instrumenting/exposition_formats/) format. 1220 1221 If you are running a prometheus server you can add this address to your scrape configs 1222 to have prometheus collect metrics on Docker. For more information 1223 on prometheus you can view the website [here](https://prometheus.io/). 1224 1225 ```none 1226 scrape_configs: 1227 - job_name: 'docker' 1228 static_configs: 1229 - targets: ['127.0.0.1:1337'] 1230 ``` 1231 1232 Please note that this feature is still marked as experimental as metrics and metric 1233 names could change while this feature is still in experimental. Please provide 1234 feedback on what you would like to see collected in the API. 1235 1236 #### Node Generic Resources 1237 1238 The `--node-generic-resources` option takes a list of key-value 1239 pair (`key=value`) that allows you to advertise user defined resources 1240 in a swarm cluster. 1241 1242 The current expected use case is to advertise NVIDIA GPUs so that services 1243 requesting `NVIDIA-GPU=[0-16]` can land on a node that has enough GPUs for 1244 the task to run. 1245 1246 Example of usage: 1247 ```json 1248 { 1249 "node-generic-resources": ["NVIDIA-GPU=UUID1", "NVIDIA-GPU=UUID2"] 1250 } 1251 ``` 1252 1253 ### Daemon configuration file 1254 1255 The `--config-file` option allows you to set any configuration option 1256 for the daemon in a JSON format. This file uses the same flag names as keys, 1257 except for flags that allow several entries, where it uses the plural 1258 of the flag name, e.g., `labels` for the `label` flag. 1259 1260 The options set in the configuration file must not conflict with options set 1261 via flags. The docker daemon fails to start if an option is duplicated between 1262 the file and the flags, regardless their value. We do this to avoid 1263 silently ignore changes introduced in configuration reloads. 1264 For example, the daemon fails to start if you set daemon labels 1265 in the configuration file and also set daemon labels via the `--label` flag. 1266 Options that are not present in the file are ignored when the daemon starts. 1267 1268 ##### On Linux 1269 1270 The default location of the configuration file on Linux is 1271 `/etc/docker/daemon.json`. The `--config-file` flag can be used to specify a 1272 non-default location. 1273 1274 This is a full example of the allowed configuration options on Linux: 1275 1276 ```json 1277 { 1278 "authorization-plugins": [], 1279 "data-root": "", 1280 "dns": [], 1281 "dns-opts": [], 1282 "dns-search": [], 1283 "exec-opts": [], 1284 "exec-root": "", 1285 "experimental": false, 1286 "storage-driver": "", 1287 "storage-opts": [], 1288 "labels": [], 1289 "live-restore": true, 1290 "log-driver": "", 1291 "log-opts": {}, 1292 "mtu": 0, 1293 "pidfile": "", 1294 "cluster-store": "", 1295 "cluster-store-opts": {}, 1296 "cluster-advertise": "", 1297 "max-concurrent-downloads": 3, 1298 "max-concurrent-uploads": 5, 1299 "default-shm-size": "64M", 1300 "shutdown-timeout": 15, 1301 "debug": true, 1302 "hosts": [], 1303 "log-level": "", 1304 "tls": true, 1305 "tlsverify": true, 1306 "tlscacert": "", 1307 "tlscert": "", 1308 "tlskey": "", 1309 "swarm-default-advertise-addr": "", 1310 "api-cors-header": "", 1311 "selinux-enabled": false, 1312 "userns-remap": "", 1313 "group": "", 1314 "cgroup-parent": "", 1315 "default-ulimits": {}, 1316 "init": false, 1317 "init-path": "/usr/libexec/docker-init", 1318 "ipv6": false, 1319 "iptables": false, 1320 "ip-forward": false, 1321 "ip-masq": false, 1322 "userland-proxy": false, 1323 "userland-proxy-path": "/usr/libexec/docker-proxy", 1324 "ip": "0.0.0.0", 1325 "bridge": "", 1326 "bip": "", 1327 "fixed-cidr": "", 1328 "fixed-cidr-v6": "", 1329 "default-gateway": "", 1330 "default-gateway-v6": "", 1331 "icc": false, 1332 "raw-logs": false, 1333 "allow-nondistributable-artifacts": [], 1334 "registry-mirrors": [], 1335 "seccomp-profile": "", 1336 "insecure-registries": [], 1337 "no-new-privileges": false, 1338 "default-runtime": "runc", 1339 "oom-score-adjust": -500, 1340 "node-generic-resources": ["NVIDIA-GPU=UUID1", "NVIDIA-GPU=UUID2"], 1341 "runtimes": { 1342 "cc-runtime": { 1343 "path": "/usr/bin/cc-runtime" 1344 }, 1345 "custom": { 1346 "path": "/usr/local/bin/my-runc-replacement", 1347 "runtimeArgs": [ 1348 "--debug" 1349 ] 1350 } 1351 } 1352 } 1353 ``` 1354 1355 > **Note:** You cannot set options in `daemon.json` that have already been set on 1356 > daemon startup as a flag. 1357 > On systems that use `systemd` to start the Docker daemon, `-H` is already set, so 1358 > you cannot use the `hosts` key in `daemon.json` to add listening addresses. 1359 > See https://docs.docker.com/engine/admin/systemd/#custom-docker-daemon-options for how 1360 > to accomplish this task with a systemd drop-in file. 1361 1362 ##### On Windows 1363 1364 The default location of the configuration file on Windows is 1365 `%programdata%\docker\config\daemon.json`. The `--config-file` flag can be 1366 used to specify a non-default location. 1367 1368 This is a full example of the allowed configuration options on Windows: 1369 1370 ```json 1371 { 1372 "authorization-plugins": [], 1373 "data-root": "", 1374 "dns": [], 1375 "dns-opts": [], 1376 "dns-search": [], 1377 "exec-opts": [], 1378 "experimental": false, 1379 "storage-driver": "", 1380 "storage-opts": [], 1381 "labels": [], 1382 "log-driver": "", 1383 "mtu": 0, 1384 "pidfile": "", 1385 "cluster-store": "", 1386 "cluster-advertise": "", 1387 "max-concurrent-downloads": 3, 1388 "max-concurrent-uploads": 5, 1389 "shutdown-timeout": 15, 1390 "debug": true, 1391 "hosts": [], 1392 "log-level": "", 1393 "tlsverify": true, 1394 "tlscacert": "", 1395 "tlscert": "", 1396 "tlskey": "", 1397 "swarm-default-advertise-addr": "", 1398 "group": "", 1399 "default-ulimits": {}, 1400 "bridge": "", 1401 "fixed-cidr": "", 1402 "raw-logs": false, 1403 "allow-nondistributable-artifacts": [], 1404 "registry-mirrors": [], 1405 "insecure-registries": [] 1406 } 1407 ``` 1408 1409 #### Configuration reload behavior 1410 1411 Some options can be reconfigured when the daemon is running without requiring 1412 to restart the process. We use the `SIGHUP` signal in Linux to reload, and a global event 1413 in Windows with the key `Global\docker-daemon-config-$PID`. The options can 1414 be modified in the configuration file but still will check for conflicts with 1415 the provided flags. The daemon fails to reconfigure itself 1416 if there are conflicts, but it won't stop execution. 1417 1418 The list of currently supported options that can be reconfigured is this: 1419 1420 - `debug`: it changes the daemon to debug mode when set to true. 1421 - `cluster-store`: it reloads the discovery store with the new address. 1422 - `cluster-store-opts`: it uses the new options to reload the discovery store. 1423 - `cluster-advertise`: it modifies the address advertised after reloading. 1424 - `labels`: it replaces the daemon labels with a new set of labels. 1425 - `live-restore`: Enables [keeping containers alive during daemon downtime](https://docs.docker.com/engine/admin/live-restore/). 1426 - `max-concurrent-downloads`: it updates the max concurrent downloads for each pull. 1427 - `max-concurrent-uploads`: it updates the max concurrent uploads for each push. 1428 - `default-runtime`: it updates the runtime to be used if not is 1429 specified at container creation. It defaults to "default" which is 1430 the runtime shipped with the official docker packages. 1431 - `runtimes`: it updates the list of available OCI runtimes that can 1432 be used to run containers 1433 - `authorization-plugin`: specifies the authorization plugins to use. 1434 - `allow-nondistributable-artifacts`: Replaces the set of registries to which the daemon will push nondistributable artifacts with a new set of registries. 1435 - `insecure-registries`: it replaces the daemon insecure registries with a new set of insecure registries. If some existing insecure registries in daemon's configuration are not in newly reloaded insecure resgitries, these existing ones will be removed from daemon's config. 1436 - `registry-mirrors`: it replaces the daemon registry mirrors with a new set of registry mirrors. If some existing registry mirrors in daemon's configuration are not in newly reloaded registry mirrors, these existing ones will be removed from daemon's config. 1437 1438 Updating and reloading the cluster configurations such as `--cluster-store`, 1439 `--cluster-advertise` and `--cluster-store-opts` will take effect only if 1440 these configurations were not previously configured. If `--cluster-store` 1441 has been provided in flags and `cluster-advertise` not, `cluster-advertise` 1442 can be added in the configuration file without accompanied by `--cluster-store`. 1443 Configuration reload will log a warning message if it detects a change in 1444 previously configured cluster configurations. 1445 1446 1447 ### Run multiple daemons 1448 1449 > **Note:** Running multiple daemons on a single host is considered as "experimental". The user should be aware of 1450 > unsolved problems. This solution may not work properly in some cases. Solutions are currently under development 1451 > and will be delivered in the near future. 1452 1453 This section describes how to run multiple Docker daemons on a single host. To 1454 run multiple daemons, you must configure each daemon so that it does not 1455 conflict with other daemons on the same host. You can set these options either 1456 by providing them as flags, or by using a [daemon configuration file](#daemon-configuration-file). 1457 1458 The following daemon options must be configured for each daemon: 1459 1460 ```none 1461 -b, --bridge= Attach containers to a network bridge 1462 --exec-root=/var/run/docker Root of the Docker execdriver 1463 --data-root=/var/lib/docker Root of persisted Docker data 1464 -p, --pidfile=/var/run/docker.pid Path to use for daemon PID file 1465 -H, --host=[] Daemon socket(s) to connect to 1466 --iptables=true Enable addition of iptables rules 1467 --config-file=/etc/docker/daemon.json Daemon configuration file 1468 --tlscacert="~/.docker/ca.pem" Trust certs signed only by this CA 1469 --tlscert="~/.docker/cert.pem" Path to TLS certificate file 1470 --tlskey="~/.docker/key.pem" Path to TLS key file 1471 ``` 1472 1473 When your daemons use different values for these flags, you can run them on the same host without any problems. 1474 It is very important to properly understand the meaning of those options and to use them correctly. 1475 1476 - The `-b, --bridge=` flag is set to `docker0` as default bridge network. It is created automatically when you install Docker. 1477 If you are not using the default, you must create and configure the bridge manually or just set it to 'none': `--bridge=none` 1478 - `--exec-root` is the path where the container state is stored. The default value is `/var/run/docker`. Specify the path for 1479 your running daemon here. 1480 - `--data-root` is the path where persisted data such as images, volumes, and 1481 cluster state are stored. The default value is `/var/lib/docker`. To avoid any 1482 conflict with other daemons, set this parameter separately for each daemon. 1483 - `-p, --pidfile=/var/run/docker.pid` is the path where the process ID of the daemon is stored. Specify the path for your 1484 pid file here. 1485 - `--host=[]` specifies where the Docker daemon will listen for client connections. If unspecified, it defaults to `/var/run/docker.sock`. 1486 - `--iptables=false` prevents the Docker daemon from adding iptables rules. If 1487 multiple daemons manage iptables rules, they may overwrite rules set by another 1488 daemon. Be aware that disabling this option requires you to manually add 1489 iptables rules to expose container ports. If you prevent Docker from adding 1490 iptables rules, Docker will also not add IP masquerading rules, even if you set 1491 `--ip-masq` to `true`. Without IP masquerading rules, Docker containers will not be 1492 able to connect to external hosts or the internet when using network other than 1493 default bridge. 1494 - `--config-file=/etc/docker/daemon.json` is the path where configuration file is stored. You can use it instead of 1495 daemon flags. Specify the path for each daemon. 1496 - `--tls*` Docker daemon supports `--tlsverify` mode that enforces encrypted and authenticated remote connections. 1497 The `--tls*` options enable use of specific certificates for individual daemons. 1498 1499 Example script for a separate “bootstrap” instance of the Docker daemon without network: 1500 1501 ```bash 1502 $ sudo dockerd \ 1503 -H unix:///var/run/docker-bootstrap.sock \ 1504 -p /var/run/docker-bootstrap.pid \ 1505 --iptables=false \ 1506 --ip-masq=false \ 1507 --bridge=none \ 1508 --data-root=/var/lib/docker-bootstrap \ 1509 --exec-root=/var/run/docker-bootstrap 1510 ```