github.com/containerd/Containerd@v1.4.13/reports/2017-06-23.md (about)

     1  # Development Report for June 23, 2017
     2  
     3  ## Events Service
     4  
     5  We added an events service for getting events across various services including
     6  containers, content, execution, images, namespaces, and snapshots. Additionally
     7  we added `ctr events` to view events emitted by the service.
     8  
     9  ```
    10  $ ctr events
    11  2017-06-23 23:21:30.271802153 +0000 UTC   /snapshot/prepare   key=registry parent=sha256:dc22a13eb565d14bfe2b16f6fa731a05da0eeff02a52059c7b59cdc2c232a2b2
    12  2017-06-23 23:21:30.28045193 +0000 UTC   /containers/create   id=registry image=docker.io/library/registry:latest runtime=&ContainerCreate_Runtime{Name:io.containerd.runtime.v1.linux,Options:map[string]string{},}
    13  2017-06-23 23:21:30.347842352 +0000 UTC   /runtime/task-create   id=registry type=CREATE pid=30411 status=0 exited=0001-01-01 00:00:00 +0000 UTC
    14  2017-06-23 23:21:30.355290368 +0000 UTC   /runtime/create   id=registry bundle=/var/lib/containerd/io.containerd.runtime.v1.linux/default/registry rootfs=type=overlay:src=overlay checkpoint=
    15  2017-06-23 23:21:30.362012776 +0000 UTC   /tasks/create   id=registry
    16  2017-06-23 23:21:30.369742117 +0000 UTC   /runtime/task-start   id=registry type=START pid=30411 status=0 exited=0001-01-01 00:00:00 +0000 UTC
    17  2017-06-23 23:21:30.369793151 +0000 UTC   /tasks/start   id=registry
    18  2017-06-23 23:21:40.169884207 +0000 UTC   /runtime/task-exit   id=registry type=EXIT pid=30411 status=130 exited=2017-06-23 23:21:40.16962998 +0000 UTC
    19  2017-06-23 23:21:40.185850194 +0000 UTC   /runtime/delete   id=registry runtime=io.containerd.runtime.v1.linux status=130 exited=2017-06-23 23:21:40.16962998 +0000 UTC
    20  2017-06-23 23:21:40.225633455 +0000 UTC   /tasks/delete   id=registry pid=30411 status=130
    21  2017-06-23 23:21:42.053154149 +0000 UTC   /snapshot/remove   key=registry
    22  2017-06-23 23:21:42.061579495 +0000 UTC   /containers/delete   id=registry
    23  ```
    24  
    25  [#956: Events service](https://github.com/containerd/containerd/pull/956)
    26  
    27  ## Filters
    28  
    29  We added the syntax to use for filtration of items over the containerd API. The
    30  `filter` package defines a syntax and parser that can be used across types and
    31  use cases in a uniform manner. This will be used commonly across the API for
    32  images, containers, events, snapshots, etc.
    33  
    34  The syntax is fairly familiar, if you've used container ecosystem
    35  projects.  At the core, we base it on the concept of protobuf field
    36  paths, augmenting with the ability to quote portions of the field path
    37  to match arbitrary labels. These "selectors" come in the following
    38  syntax:
    39  
    40  ```
    41  <fieldpath>[<operator><value>]
    42  ```
    43  
    44  A basic example is as follows:
    45  
    46  ```
    47  name==foo
    48  ```
    49  
    50  This would match all objects that have a field `name` with the value
    51  `foo`. If we only want to test if the field is present, we can omit the
    52  operator. This is most useful for matching labels in containerd. The
    53  following will match objects that has the field labels and have the
    54  label "foo" defined:
    55  
    56  ```
    57  labels.foo
    58  ```
    59  
    60  We also allow for quoting of parts of the field path to allow matching
    61  of arbitrary items:
    62  
    63  ```
    64  labels."very complex label"==something
    65  ```
    66  
    67  We also define `!=` and `~=` as operators. The `!=` operator will match
    68  all objects that don't match the value for a field and `~=` will compile
    69  the target value as a regular expression and match the field value
    70  against that.
    71  
    72  Selectors can be combined using a comma, such that the resulting
    73  selector will require all selectors are matched for the object to match.
    74  The following example will match objects that are named `foo` and have
    75  the label `bar`:
    76  
    77  ```
    78  name==foo,labels.bar
    79  ```
    80  
    81  [#995: Add package for filters](https://github.com/containerd/containerd/pull/995)
    82  
    83  [#1050: Container list filters](https://github.com/containerd/containerd/pull/1050)
    84  
    85  [#1069: Event filters](https://github.com/containerd/containerd/pull/1069)
    86  
    87  ## Support for pushing and pulling OCI indexes
    88  
    89  We added support for pushing and pulling OCI indexes. Currently all content
    90  referenced by the list are pulled and further work on the client will be done
    91  to allow selection of the pulled manifest to extract and run.
    92  
    93  [#1022: Support pull for OCI indexes](https://github.com/containerd/containerd/pull/1022)
    94  
    95  ## Snapshot list
    96  
    97  We added `ctr snapshot list` to snapshots from containerd. This will output all
    98  snapshots, not just the active snapshots used by containers.
    99  
   100  ```
   101  $ ctr snapshot list
   102  ID                                                                      Parent                                                                  State     Readonly
   103  registry2                                                               sha256:dc22a13eb565d14bfe2b16f6fa731a05da0eeff02a52059c7b59cdc2c232a2b2 active    false
   104  registry3                                                               sha256:dc22a13eb565d14bfe2b16f6fa731a05da0eeff02a52059c7b59cdc2c232a2b2 active    false
   105  sha256:4ac69ce655ab8aa97362915793348d31361fb3c047e223c2b58be706e89c48fc sha256:ba2cc2690e31f63847e4bc0d266b354f8f11dc04474d45d44312ff70edae9c98 committed true
   106  sha256:ba2cc2690e31f63847e4bc0d266b354f8f11dc04474d45d44312ff70edae9c98                                                                         committed true
   107  sha256:bfe0b04fc169b94099b29dbf5a527f6a11db627cd0a6126803edf8f42bd7b4b3 sha256:4ac69ce655ab8aa97362915793348d31361fb3c047e223c2b58be706e89c48fc committed true
   108  sha256:d959def87dadbb9ba85070c09e99b46d994967b12f5748f617c377073b8d1e39 sha256:bfe0b04fc169b94099b29dbf5a527f6a11db627cd0a6126803edf8f42bd7b4b3 committed true
   109  sha256:dc22a13eb565d14bfe2b16f6fa731a05da0eeff02a52059c7b59cdc2c232a2b2 sha256:d959def87dadbb9ba85070c09e99b46d994967b12f5748f617c377073b8d1e39 committed true
   110  ```
   111  
   112  [#1058: Add list snapshot subcommand](https://github.com/containerd/containerd/pull/1058)
   113  
   114  ## API cleanup
   115  
   116  As part of our API review process we have started implementing some changes to
   117  make the API clearer and more consistent.
   118  
   119  [#1040: Update GRPC APIs](https://github.com/containerd/containerd/pull/1040)
   120  
   121  [#1047: Rename execution service to tasks](https://github.com/containerd/containerd/pull/1047)
   122  
   123  [#1059: enforce a character set for namespaces](https://github.com/containerd/containerd/pull/1059)
   124  
   125  [#1052: version protobuf service packages](https://github.com/containerd/containerd/pull/1062)
   126  
   127  [#1072: move events types into service package](https://github.com/containerd/containerd/pull/1072)
   128  
   129  [#1073: consolidate API types packages](https://github.com/containerd/containerd/pull/1073)
   130  
   131  [#1074: replace whale with D](https://github.com/containerd/containerd/pull/1074)
   132  
   133  ## Containerd notes from Moby Summit
   134  
   135  At the Moby summit on 06/19/2017 there was a containerd round table meeting.
   136  This was a good opportunity to discuss the upcoming API freeze and release of
   137  containerd with others working with it in the community. Always remember that
   138  these summit round tables are not the only opportunity to have these topics
   139  discussed and everyone is encouraged to open issues and engage the community on
   140  Slack.
   141  
   142  * "What are the plans for a resolver-resolver and image reference namespacing?"
   143      - Maintainers are trying to figure out what everyone’s plans/needs are for a resolver-resolver.
   144      - A resolver-resolver allows configuring where push/pull happens. Could take in a configuration format which specifies how to push, pull, and authenticate. Needed in order to do discover of names to registry configurations.
   145      - Stephen confirms we are thinking of more configuration driven rather programmatic. The resolver-resolver and any configuration is always client side, allowing the integrator to design any interface or configuration suits their needs. But we are also looking to define something that could be consistently applied by containerd clients.
   146      - The resolver-resolver will be compatible with Docker, but could allow Docker to expand its resolution support.
   147  
   148  * "What is the plan for garbage collection?"
   149      - Current design is oriented around being garbage collected. The data model is designed around references which are used to walk a root set. Everything unreferenced would be garbage collected.
   150      - Another more complicated aspect of garbage collection is around policy, allowing to clean up resources based on age or client specified policies.
   151      - Client side implementations would allow establishing policies.
   152      - Containerd will require a stop the world to do the garbage collection.
   153      - A heavy delete which does not garbage collect is an option and similar to to the interface today. The API does not guarantee that disk is cleaned up after a deletion, only that the resource is freed. Inline deletion would require reference counting to know when to delete. This would also require global locking to protect the references.
   154      - How to handle content which is active but unreferenced, leasing vs pre-allocate. This has not been decided on.
   155  
   156  * "What will need to change in Docker in regards to graphdrivers for accommodating the containerd snapshotters?"
   157      - The goal is to have graphdrivers be able to be used as snapshot drivers.
   158      - To accomplish this graphdrivers need to be able to return mounts without needing to require action on unmount. Unmount will no longer contact the graphdriver since snapshotters to do not control the mount/unmount lifecycle.
   159      - For implementation, in the repository tree only overlayfs and btrfs are supported. Everything else will be out of tree and require special builds or proxying.
   160  
   161  * "Version numbers on events and API objects"
   162      - Are objects mutable? Some objects are mutable but not necessarily relevant to clients.
   163      - Are containers mutable? It can have multiple tasks, can be started and stopped.
   164      - We may need to have task id separate from the container to differentiate between tasks within a container.
   165      - Tasks have a pid which could be used to identify the task. Someone had a customer that ran out of memory from running too many tasks, could be caused by repeated execs in the same container.
   166      - Getting state of a containers task involves requesting containers and tasks, or just tasks could be listed.
   167      - What are the API costs, is there measurements of the cost of API calls and comparisons with Docker? Calling docker ps can be expensive due to locks, containerd should be much cheaper and faster to call. This need to be verified and measurements added.
   168  
   169  * "How can clients understand the capabilities of the containerd instance it is talking to?"
   170      - As an orchestrator, interested in understanding what can be done with the plugins. Example, docker log drivers change feature set but the log driver names do not change. Stuck on checking docker version.
   171      - There have been problems in the past with requiring multiple clients of docker to handle changes.
   172      - GRPC is adding introspection, plan is to wait for this work rather than making something ourselves.
   173      - The maintainers would like to understand what were the really bad changes in Docker that caused problems with version support for users. Problems around needing to bump the whole API to get a new feature. Containerd API versions each service and v1 interface will be stable and supported.