github.com/webwurst/docker@v1.7.0/daemon/graphdriver/devmapper/README.md

github.com/webwurst/docker@v1.7.0/daemon/graphdriver/devmapper/README.md (about)

1 ## devicemapper - a storage backend based on Device Mapper
2
3 ### Theory of operation
4
5 The device mapper graphdriver uses the device mapper thin provisioning
6 module (dm-thinp) to implement CoW snapshots. For each devicemapper
7 graph location (typically `/var/lib/docker/devicemapper`, $graph below)
8 a thin pool is created based on two block devices, one for data and
9 one for metadata. By default these block devices are created
10 automatically by using loopback mounts of automatically created sparse
11 files.
12
13 The default loopback files used are `$graph/devicemapper/data` and
14 `$graph/devicemapper/metadata`. Additional metadata required to map
15 from docker entities to the corresponding devicemapper volumes is
16 stored in the `$graph/devicemapper/json` file (encoded as Json).
17
18 In order to support multiple devicemapper graphs on a system, the thin
19 pool will be named something like: `docker-0:33-19478248-pool`, where
20 the `0:33` part is the minor/major device nr and `19478248` is the
21 inode number of the $graph directory.
22
23 On the thin pool, docker automatically creates a base thin device,
24 called something like `docker-0:33-19478248-base` of a fixed
25 size. This is automatically formatted with an empty filesystem on
26 creation. This device is the base of all docker images and
27 containers. All base images are snapshots of this device and those
28 images are then in turn used as snapshots for other images and
29 eventually containers.
30
31 ### Information on `docker info`
32
33 As of docker-1.4.1, `docker info` when using the `devicemapper` storage driver
34 will display something like:
35
36 $ sudo docker info
37 [...]
38 Storage Driver: devicemapper
39 Pool Name: docker-253:1-17538953-pool
40 Pool Blocksize: 65.54 kB
41 Data file: /dev/loop4
42 Metadata file: /dev/loop4
43 Data Space Used: 2.536 GB
44 Data Space Total: 107.4 GB
45 Data Space Available: 104.8 GB
46 Metadata Space Used: 7.93 MB
47 Metadata Space Total: 2.147 GB
48 Metadata Space Available: 2.14 GB
49 Udev Sync Supported: true
50 Data loop file: /home/docker/devicemapper/devicemapper/data
51 Metadata loop file: /home/docker/devicemapper/devicemapper/metadata
52 Library Version: 1.02.82-git (2013-10-04)
53 [...]
54
55 #### status items
56
57 Each item in the indented section under `Storage Driver: devicemapper` are
58 status information about the driver.
59 * `Pool Name` name of the devicemapper pool for this driver.
60 * `Pool Blocksize` tells the blocksize the thin pool was initialized with. This only changes on creation.
61 * `Data file` blockdevice file used for the devicemapper data
62 * `Metadata file` blockdevice file used for the devicemapper metadata
63 * `Data Space Used` tells how much of `Data file` is currently used
64 * `Data Space Total` tells max size the `Data file`
65 * `Data Space Available` tells how much free space there is in the `Data file`. If you are using a loop device this will report the actual space available to the loop device on the underlying filesystem.
66 * `Metadata Space Used` tells how much of `Metadata file` is currently used
67 * `Metadata Space Total` tells max size the `Metadata file`
68 * `Metadata Space Available` tells how much free space there is in the `Metadata file`. If you are using a loop device this will report the actual space available to the loop device on the underlying filesystem.
69 * `Udev Sync Supported` tells whether devicemapper is able to sync with Udev. Should be `true`.
70 * `Data loop file` file attached to `Data file`, if loopback device is used
71 * `Metadata loop file` file attached to `Metadata file`, if loopback device is used
72 * `Library Version` from the libdevmapper used
73
74 ### options
75
76 The devicemapper backend supports some options that you can specify
77 when starting the docker daemon using the `--storage-opt` flags.
78 This uses the `dm` prefix and would be used something like `docker -d --storage-opt dm.foo=bar`.
79
80 Here is the list of supported options:
81
82 * `dm.basesize`
83
84 Specifies the size to use when creating the base device, which
85 limits the size of images and containers. The default value is
86 10G. Note, thin devices are inherently "sparse", so a 10G device
87 which is mostly empty doesn't use 10 GB of space on the
88 pool. However, the filesystem will use more space for the empty
89 case the larger the device is. **Warning**: This value affects the
90 system-wide "base" empty filesystem that may already be
91 initialized and inherited by pulled images. Typically, a change
92 to this value will require additional steps to take effect: 1)
93 stop `docker -d`, 2) `rm -rf /var/lib/docker`, 3) start `docker -d`.
94
95 Example use:
96
97 ``docker -d --storage-opt dm.basesize=20G``
98
99 * `dm.loopdatasize`
100
101 Specifies the size to use when creating the loopback file for the
102 "data" device which is used for the thin pool. The default size is
103 100G. Note that the file is sparse, so it will not initially take
104 up this much space.
105
106 Example use:
107
108 ``docker -d --storage-opt dm.loopdatasize=200G``
109
110 * `dm.loopmetadatasize`
111
112 Specifies the size to use when creating the loopback file for the
113 "metadadata" device which is used for the thin pool. The default size is
114 2G. Note that the file is sparse, so it will not initially take
115 up this much space.
116
117 Example use:
118
119 ``docker -d --storage-opt dm.loopmetadatasize=4G``
120
121 * `dm.fs`
122
123 Specifies the filesystem type to use for the base device. The supported
124 options are "ext4" and "xfs". The default is "ext4"
125
126 Example use:
127
128 ``docker -d --storage-opt dm.fs=xfs``
129
130 * `dm.mkfsarg`
131
132 Specifies extra mkfs arguments to be used when creating the base device.
133
134 Example use:
135
136 ``docker -d --storage-opt "dm.mkfsarg=-O ^has_journal"``
137
138 * `dm.mountopt`
139
140 Specifies extra mount options used when mounting the thin devices.
141
142 Example use:
143
144 ``docker -d --storage-opt dm.mountopt=nodiscard``
145
146 * `dm.thinpooldev`
147
148 Specifies a custom blockdevice to use for the thin pool.
149
150 If using a block device for device mapper storage, ideally lvm2
151 would be used to create/manage the thin-pool volume that is then
152 handed to docker to exclusively create/manage the thin and thin
153 snapshot volumes needed for its containers. Managing the thin-pool
154 outside of docker makes for the most feature-rich method of having
155 docker utilize device mapper thin provisioning as the backing
156 storage for docker's containers. lvm2-based thin-pool management
157 feature highlights include: automatic or interactive thin-pool
158 resize support, dynamically change thin-pool features, automatic
159 thinp metadata checking when lvm2 activates the thin-pool, etc.
160
161 Example use:
162
163 ``docker -d --storage-opt dm.thinpooldev=/dev/mapper/thin-pool``
164
165 * `dm.datadev`
166
167 Specifies a custom blockdevice to use for data for the thin pool.
168
169 If using a block device for device mapper storage, ideally both
170 datadev and metadatadev should be specified to completely avoid
171 using the loopback device.
172
173 Example use:
174
175 ``docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1``
176
177 * `dm.metadatadev`
178
179 Specifies a custom blockdevice to use for metadata for the thin
180 pool.
181
182 For best performance the metadata should be on a different spindle
183 than the data, or even better on an SSD.
184
185 If setting up a new metadata pool it is required to be valid. This
186 can be achieved by zeroing the first 4k to indicate empty
187 metadata, like this:
188
189 ``dd if=/dev/zero of=$metadata_dev bs=4096 count=1``
190
191 Example use:
192
193 ``docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1``
194
195 * `dm.blocksize`
196
197 Specifies a custom blocksize to use for the thin pool. The default
198 blocksize is 64K.
199
200 Example use:
201
202 ``docker -d --storage-opt dm.blocksize=512K``
203
204 * `dm.blkdiscard`
205
206 Enables or disables the use of blkdiscard when removing
207 devicemapper devices. This is enabled by default (only) if using
208 loopback devices and is required to resparsify the loopback file
209 on image/container removal.
210
211 Disabling this on loopback can lead to *much* faster container
212 removal times, but will make the space used in /var/lib/docker
213 directory not be returned to the system for other use when
214 containers are removed.
215
216 Example use:
217
218 ``docker -d --storage-opt dm.blkdiscard=false``
219
220 * `dm.override_udev_sync_check`
221
222 Overrides the `udev` synchronization checks between `devicemapper` and `udev`.
223 `udev` is the device manager for the Linux kernel.
224
225 To view the `udev` sync support of a Docker daemon that is using the
226 `devicemapper` driver, run:
227
228 $ docker info
229 [...]
230 Udev Sync Supported: true
231 [...]
232
233 When `udev` sync support is `true`, then `devicemapper` and udev can
234 coordinate the activation and deactivation of devices for containers.
235
236 When `udev` sync support is `false`, a race condition occurs between
237 the`devicemapper` and `udev` during create and cleanup. The race condition
238 results in errors and failures. (For information on these failures, see
239 [docker#4036](https://github.com/docker/docker/issues/4036))
240
241 To allow the `docker` daemon to start, regardless of `udev` sync not being
242 supported, set `dm.override_udev_sync_check` to true:
243
244 $ docker -d --storage-opt dm.override_udev_sync_check=true
245
246 When this value is `true`, the `devicemapper` continues and simply warns
247 you the errors are happening.
248
249 > **Note**: The ideal is to pursue a `docker` daemon and environment that
250 > does support synchronizing with `udev`. For further discussion on this
251 > topic, see [docker#4036](https://github.com/docker/docker/issues/4036).
252 > Otherwise, set this flag for migrating existing Docker daemons to a
253 > daemon with a supported environment.
254
255 * `dm.use_deferred_removal`
256
257 Enables use of deferred device removal if libdm and kernel driver
258 support the mechanism.
259
260 Deferred device removal means that if device is busy when devices is
261 being removed/deactivated, then a deferred removal is scheduled on
262 device. And devices automatically goes away when last user of device
263 exits.
264
265 For example, when contianer exits, its associated thin device is
266 removed. If that devices has leaked into some other mount namespace
267 can can't be removed now, container exit will still be successful
268 and this option will just schedule device for deferred removal and
269 will not wait in a loop trying to remove a busy device.
270
271 Example use:
272
273 ``docker -d --storage-opt dm.use_deferred_removal=true``
274