github.com/vieux/docker@v0.6.3-0.20161004191708-e097c2a938c7/docs/userguide/storagedriver/aufs-driver.md (about)

     1  <!--[metadata]>
     2  +++
     3  title = "AUFS storage driver in practice"
     4  description = "Learn how to optimize your use of AUFS driver."
     5  keywords = ["container, storage, driver, AUFS "]
     6  [menu.main]
     7  parent = "engine_driver"
     8  +++
     9  <![end-metadata]-->
    10  
    11  # Docker and AUFS in practice
    12  
    13  AUFS was the first storage driver in use with Docker. As a result, it has a
    14  long and close history with Docker, is very stable, has a lot of real-world
    15  deployments, and has strong community support. AUFS has several features that
    16  make it a good choice for Docker. These features enable:
    17  
    18  - Fast container startup times.
    19  - Efficient use of storage.
    20  - Efficient use of memory.
    21  
    22  Despite its capabilities and long history with Docker, some Linux distributions
    23   do not support AUFS. This is usually because AUFS is not included in the
    24  mainline (upstream) Linux kernel.
    25  
    26  The following sections examine some AUFS features and how they relate to
    27  Docker.
    28  
    29  ## Image layering and sharing with AUFS
    30  
    31  AUFS is a *unification filesystem*. This means that it takes multiple
    32  directories on a single Linux host, stacks them on top of each other, and
    33  provides a single unified view. To achieve this, AUFS uses a *union mount*.
    34  
    35  AUFS stacks multiple directories and exposes them as a unified view through a
    36  single mount point. All of the directories in the stack, as well as the union
    37  mount point, must all exist on the same Linux host. AUFS refers to each
    38  directory that it stacks as a *branch*.
    39  
    40  Within Docker, AUFS union mounts enable image layering. The AUFS storage driver
    41   implements Docker image layers using this union mount system. AUFS branches
    42  correspond to Docker image layers. The diagram below shows a Docker container
    43  based on the `ubuntu:latest` image.
    44  
    45  ![](images/aufs_layers.jpg)
    46  
    47  This diagram shows that each image layer, and the container layer, is
    48  represented in the Docker hosts filesystem as a directory under
    49  `/var/lib/docker/`. The union mount point provides the unified view of all
    50  layers. As of Docker 1.10, image layer IDs do not correspond to the names of
    51  the directories that contain their data.
    52  
    53  AUFS also supports the copy-on-write technology (CoW). Not all storage drivers
    54  do.
    55  
    56  ## Container reads and writes with AUFS
    57  
    58  Docker leverages AUFS CoW technology to enable image sharing and minimize the
    59  use of disk space. AUFS works at the file level. This means that all AUFS CoW
    60  operations copy entire files - even if only a small part of the file is being
    61  modified. This behavior can have a noticeable impact on container performance,
    62   especially if the files being copied are large, below a lot of image layers,
    63  or the CoW operation must search a deep directory tree.
    64  
    65  Consider, for example, an application running in a container needs to add a
    66  single new value to a large key-value store (file). If this is the first time
    67  the file is modified, it does not yet exist in the container's top writable
    68  layer. So, the CoW must *copy up* the file from the underlying image. The AUFS
    69  storage driver searches each image layer for the file. The search order is from
    70   top to bottom. When it is found, the entire file is *copied up* to the
    71  container's top writable layer. From there, it can be opened and modified.
    72  
    73  Larger files obviously take longer to *copy up* than smaller files, and files
    74  that exist in lower image layers take longer to locate than those in higher
    75  layers. However, a *copy up* operation only occurs once per file on any given
    76  container. Subsequent reads and writes happen against the file's copy already
    77  *copied-up* to the container's top layer.
    78  
    79  ## Deleting files with the AUFS storage driver
    80  
    81  The AUFS storage driver deletes a file from a container by placing a *whiteout
    82  file* in the container's top layer. The whiteout file effectively obscures the
    83  existence of the file in the read-only image layers below. The simplified
    84  diagram below shows a container based on an image with three image layers.
    85  
    86  ![](images/aufs_delete.jpg)
    87  
    88  The `file3` was deleted from the container. So, the AUFS storage driver  placed
    89  a whiteout file in the container's top layer. This whiteout file effectively
    90  "deletes" `file3` from the container by obscuring any of the original file's
    91  existence in the image's read-only layers. This works the same no matter which
    92  of the image's read-only layers the file exists in.
    93  
    94  ## Renaming directories with the AUFS storage driver
    95  
    96  Calling `rename(2)` for a directory is not fully supported on AUFS. It returns 
    97  `EXDEV` ("cross-device link not permitted"), even when both of the source and 
    98  the destination path are on a same AUFS layer, unless the directory has no 
    99  children.
   100  
   101  So your application has to be designed so that it can handle `EXDEV` and fall 
   102  back to a "copy and unlink" strategy.
   103  
   104  ## Configure Docker with AUFS
   105  
   106  You can only use the AUFS storage driver on Linux systems with AUFS installed.
   107  Use the following command to determine if your system supports AUFS.
   108  
   109      $ grep aufs /proc/filesystems
   110  
   111      nodev   aufs
   112  
   113  This output indicates the system supports AUFS. Once you've verified your
   114  system supports AUFS, you can must instruct the Docker daemon to use it. You do
   115  this from the command line with the `dockerd` command:
   116  
   117      $ sudo dockerd --storage-driver=aufs &
   118  
   119  
   120  Alternatively, you can edit the Docker config file and add the
   121  `--storage-driver=aufs` option to the `DOCKER_OPTS` line.
   122  
   123      # Use DOCKER_OPTS to modify the daemon startup options.
   124      DOCKER_OPTS="--storage-driver=aufs"
   125  
   126  Once your daemon is running, verify the storage driver with the `docker info`
   127  command.
   128  
   129      $ sudo docker info
   130  
   131      Containers: 1
   132      Images: 4
   133      Storage Driver: aufs
   134       Root Dir: /var/lib/docker/aufs
   135       Backing Filesystem: extfs
   136       Dirs: 6
   137       Dirperm1 Supported: false
   138      Execution Driver: native-0.2
   139      ...output truncated...
   140  
   141  The output above shows that the Docker daemon is running the AUFS storage
   142  driver on top of an existing `ext4` backing filesystem.
   143  
   144  ## Local storage and AUFS
   145  
   146  As the `dockerd` runs with the AUFS driver, the driver stores images and
   147  containers within the Docker host's local storage area under
   148  `/var/lib/docker/aufs/`.
   149  
   150  ### Images
   151  
   152  Image layers and their contents are stored under
   153  `/var/lib/docker/aufs/diff/`. With Docker 1.10 and higher, image layer IDs do
   154  not correspond to directory names.
   155  
   156  The `/var/lib/docker/aufs/layers/` directory contains metadata about how image
   157  layers are stacked. This directory contains one file for every image or
   158  container layer on the Docker host (though file names no longer match image
   159  layer IDs). Inside each file are the names of the directories that exist below
   160  it in the stack
   161  
   162  The command below shows the contents of a metadata file in
   163  `/var/lib/docker/aufs/layers/` that lists the three directories that are
   164  stacked below it in the union mount. Remember, these directory names do no map
   165  to image layer IDs with Docker 1.10 and higher.
   166  
   167      $ cat /var/lib/docker/aufs/layers/91e54dfb11794fad694460162bf0cb0a4fa710cfa3f60979c177d920813e267c
   168  
   169      d74508fb6632491cea586a1fd7d748dfc5274cd6fdfedee309ecdcbc2bf5cb82
   170      c22013c8472965aa5b62559f2b540cd440716ef149756e7b958a1b2aba421e87
   171      d3a1f33e8a5a513092f01bb7eb1c2abf4d711e5105390a3fe1ae2248cfde1391
   172  
   173  The base layer in an image has no image layers below it, so its file is empty.
   174  
   175  ### Containers
   176  
   177  Running containers are mounted below `/var/lib/docker/aufs/mnt/<container-id>`.
   178   This is where the AUFS union mount point that exposes the container and all
   179  underlying image layers as a single unified view exists. If a container is not
   180  running, it still has a directory here but it is empty. This is because AUFS
   181  only mounts a container when it is running. With Docker 1.10 and higher,
   182   container IDs no longer correspond to directory names under
   183  `/var/lib/docker/aufs/mnt/<container-id>`.
   184  
   185  Container metadata and various config files that are placed into the running
   186  container are stored in `/var/lib/docker/containers/<container-id>`. Files in
   187  this directory exist for all containers on the system, including ones that are
   188  stopped. However, when a container is running the container's log files are
   189  also in this directory.
   190  
   191  A container's thin writable layer is stored in a directory under
   192  `/var/lib/docker/aufs/diff/`. With Docker 1.10 and higher, container IDs no
   193  longer correspond to directory names. However, the containers thin writable
   194  layer still exists under here and is stacked by AUFS as the top writable layer
   195  and is where all changes to the container are stored. The directory exists even
   196   if the container is stopped. This means that restarting a container will not
   197  lose changes made to it. Once a container is deleted, it's thin writable layer
   198  in this directory is deleted.
   199  
   200  ## AUFS and Docker performance
   201  
   202  To summarize some of the performance related aspects already mentioned:
   203  
   204  - The AUFS storage driver is a good choice for PaaS and other similar use-cases
   205   where container density is important. This is because AUFS efficiently shares
   206  images between multiple running containers, enabling fast container start times
   207   and minimal use of disk space.
   208  
   209  - The underlying mechanics of how AUFS shares files between image layers and
   210  containers uses the systems page cache very efficiently.
   211  
   212  - The AUFS storage driver can introduce significant latencies into container
   213  write performance. This is because the first time a container writes to any
   214  file, the file has be located and copied into the containers top writable
   215  layer. These latencies increase and are compounded when these files exist below
   216   many image layers and the files themselves are large.
   217  
   218  One final point. Data volumes provide the best and most predictable
   219  performance. This is because they bypass the storage driver and do not incur 
   220  any of the potential overheads introduced by thin provisioning and
   221  copy-on-write. For this reason, you may want to place heavy write workloads on
   222  data volumes.
   223  
   224  ## AUFS compatibility
   225  
   226  To summarize the AUFS's aspect which is incompatible with other filesystems:
   227  
   228  - The AUFS does not fully support the `rename(2)` system call. Your application 
   229  needs to detect its failure and fall back to a "copy and unlink" strategy.
   230  
   231  ## Related information
   232  
   233  * [Understand images, containers, and storage drivers](imagesandcontainers.md)
   234  * [Select a storage driver](selectadriver.md)
   235  * [Btrfs storage driver in practice](btrfs-driver.md)
   236  * [Device Mapper storage driver in practice](device-mapper-driver.md)