github.com/containerd/containerd@v22.0.0-20200918172823-438c87b8e050+incompatible/docs/remote-snapshotter.md (about)

     1  # Remote Snapshotter
     2  
     3  Containerd allows snapshotters to reuse snapshots existing somewhere managed by them.
     4  
     5  _Remote Snapshotter_ is a snapshotter that leverages this functionality and reuses snapshots that are stored in a remotely shared place.
     6  These remotely shared snapshots are called _remote snapshots_.
     7  Remote snapshotter allows containerd to prepare these remote snapshots without pulling layers from registries, which hopefully shorten the time to take for image pull.
     8  
     9  One of the remote snapshotter implementations is [Stargz Snapshotter](https://github.com/containerd/stargz-snapshotter).
    10  This enables containerd to lazily pull images from standard-compliant registries leveraging remote snapshotter functionality and stargz images by google/crfs.
    11  
    12  ## The containerd client API
    13  
    14  The containerd client's `Pull` API with unpacking-mode allows the underlying snapshotter to query for remote snapshots before fetching content.
    15  Remote snapshotter needs to be plugged into containerd in [the same ways as normal snapshotters](/PLUGINS.md).
    16  
    17  ```go
    18  image, err := client.Pull(ctx, ref,
    19  	containerd.WithPullUnpack,
    20  	containerd.WithPullSnapshotter("my-remote-snapshotter"),
    21  )
    22  ```
    23  
    24  ## Passing snapshotter-specific information
    25  
    26  Some remote snapshotters requires snapshotter-specific information through `Pull` API.
    27  The information will be used in various ways including searching snapshot contents from a remote store.
    28  One of the example snapshotters that requires snapshotter-specific information is stargz snapshotter.
    29  It requires the image reference name and layer digests, etc. for searching layer contents from registries.
    30  
    31  Snapshotters receive the information through user-defined labels prefixed by `containerd.io/snapshot/`.
    32  The containerd client supports two ways to pass these labels to the underlying snapshotter.
    33  
    34  ### Using snapshotter's `WithLabels` option
    35  
    36  User-defined labels can be passed down to the underlying snapshotter using snapshotter option `WithLabels`.
    37  Specified labels will be passed down every time the containerd client queries a remote snapshot.
    38  This is useful if the values of these labels are determined statically regardless of the snapshots.
    39  These user-defined labels must be prefixed by `containerd.io/snapshot/`.
    40  
    41  ```go
    42  import "github.com/containerd/containerd/snapshots"
    43  
    44  image, err := client.Pull(ctx, ref,
    45  	containerd.WithPullUnpack,
    46  	containerd.WithPullSnapshotter(
    47  		"my-remote-snapshotter",
    48  		snapshots.WithLabels(map[string]string{
    49  			"containerd.io/snapshot/reference": ref,
    50  		}),
    51  	),
    52  )
    53  ```
    54  
    55  ### Using the containerd client's `WithImageHandlerWrapper` option
    56  
    57  User-defined labels can also be passed using an image handler wrapper.
    58  This is useful when labels vary depending on the snapshot.
    59  
    60  Every time the containerd client queries remote snapshot, it passes `Annotations` appended to the targeting layer descriptor (means the layer descriptor that will be pulled and unpacked for preparing that snapshot) to the underlying snapshotter.
    61  These annotations are passed to the snapshotter as user-defined labels.
    62  The values of annotations can be dynamically added and modified in the handler wrapper.
    63  Note that annotations must be prefixed by `containerd.io/snapshot/`.
    64  [CRI plugin](https://github.com/containerd/cri/blob/09d6426f33cac217528158ddc6d254ca7d597a7b/pkg/server/image_pull.go#L127) and [stargz snapshotter](https://github.com/containerd/stargz-snapshotter/blob/875ec333403a885f5b6e5b64c94ec4dc713e0596/cmd/ctr-remote/commands/rpull.go#L97) leverage this method.
    65  
    66  ```go
    67  import "github.com/ktock/snapshotter/handler"
    68  
    69  if _, err := client.Pull(ctx, ref,
    70  	containerd.WithPullUnpack,
    71  	containerd.WithPullSnapshotter("my-remote-snapshotter"),
    72  	containerd.WithImageHandlerWrapper(handler.Wrapper(ref)),
    73  )
    74  ```
    75  
    76  ## Snapshotter APIs for querying remote snapshots
    77  
    78  The containerd client queries remote snapshots to the underlying remote snapshotter using snapshotter APIs.
    79  This section describes the high-level overview of how snapshotter APIs are used for remote snapshots functionality, with some piece of pseudo-codes that describe the simplified logic implemented in the containerd client.
    80  For more details, see [`unpacker.go`](/unpacker.go) that implements this logic.
    81  
    82  During image pull, the containerd client calls `Prepare` API with the label `containerd.io/snapshot.ref`.
    83  This is a containerd-defined label which contains ChainID that targets a committed snapshot that the client is trying to prepare.
    84  At this moment, user-defined labels (prefixed by `containerd.io/snapshot/`) will also be merged into the labels option.
    85  
    86  ```go
    87  // Gets annotations appended to the targetting layer which would contain
    88  // snapshotter-specific information passed by the user.
    89  labels := snapshots.FilterInheritedLabels(desc.Annotations)
    90  if labels == nil {
    91  	labels = make(map[string]string)
    92  }
    93  
    94  // Specifies ChainID of the targeting committed snapshot.
    95  labels["containerd.io/snapshot.ref"] = chainID
    96  
    97  // Merges snapshotter options specified by the user which would contain
    98  // snapshotter-specific information passed by the user.
    99  opts := append(rCtx.SnapshotterOpts, snapshots.WithLabels(labels))
   100  
   101  // Calls `Prepare` API with target indentifier and snapshotter-specific
   102  // information.
   103  mounts, err = sn.Prepare(ctx, key, parent.String(), opts...)
   104  ```
   105  
   106  If this snapshotter is a remote snapshotter, that committed snapshot hopefully exists, for example, in a shared remote store.
   107  Remote snapshotter must define and enforce policies about whether it will use an existing snapshot.
   108  When remote snapshotter allows the user to use that snapshot, it must return `ErrAlreadyExists`.
   109  
   110  If the containerd client gets `ErrAlreadyExists` by `Prepare`, it ensures the existence of that committed snapshot by calling `Stat` with the ChainID.
   111  If this snapshot is available, the containerd client skips pulling and unpacking layer that would otherwise be needed for preparing and committing that snapshot.
   112  
   113  ```go
   114  mounts, err = sn.Prepare(ctx, key, parent.String(), opts...)
   115  if err != nil {
   116  	if errdefs.IsAlreadyExists(err) {
   117  		// Ensures the layer existence
   118  		if _, err := sn.Stat(ctx, chainID); err != nil {
   119  			// Handling error
   120  		} else {
   121  			// snapshot found with ChainID
   122  			// pulling/unpacking will be skipped
   123  			continue
   124  		}
   125  	} else {
   126  		return err
   127  	}
   128  }
   129  ```