github.com/docker/cnab-to-oci@v0.3.0-beta4/README.md (about)

     1  [![Documentation](https://godoc.org/github.com/docker/cnab-to-oci/remotes?status.svg)](http://godoc.org/github.com/docker/cnab-to-oci/remotes)
     2  
     3  # CNAB to OCI
     4  
     5  The intent of CNAB to OCI is to propose a reference implementation for sharing a
     6  CNAB using an OCI or Docker registry.
     7  
     8  [Jump to the example](#example).
     9  
    10  ## Rationale for this approach
    11  
    12  Goals:
    13  - Package the information from a CNAB [`bundle.json`](https://github.com/deislabs/cnab-spec/blob/master/101-bundle-json.md) into a format that can be stored in container registries.
    14  - Require no or only minor changes to the [OCI specification](https://github.com/opencontainers/image-spec).
    15      - Major changes would take long to get approved.
    16      - Anything that diverges from the current specification will require coordination with registries to ensure compatibility.
    17  - Store all container images required for the CNAB in the same repository and reference them from the same manifest.
    18      - If a user can access the CNAB, they can access all the parts needed to install it.
    19      - Moving a CNAB from one repository to another is atomic.
    20  - Ensure that registries can reason over these CNABs.
    21      - Provide enough information for registries to understand how to present these artifacts.
    22  
    23  Non-goals:
    24  - A perfectly clean solution.
    25      - The authors acknowledge that there is a tension between getting something working today and the ideal solution.
    26  
    27  ### Selection of OCI index
    28  
    29  The CNAB specification references a
    30  [list of invocation images](https://github.com/deislabs/cnab-spec/blob/master/101-bundle-json.md#invocation-images)
    31  and a
    32  [map of other images](https://github.com/deislabs/cnab-spec/blob/master/101-bundle-json.md#the-image-map).
    33  An [OCI index](#what-is-an-oci-index) is already used for handling multiple
    34  images so this was seen as a natural fit.
    35  
    36  The only disadvantage of the OCI index is its lack of a top-level
    37  mechanism for communicating type which may make the artifacts more difficult for
    38  registries to understand. The authors propose overcoming this using
    39  [annotations](#annotations).
    40  
    41  ### Annotations
    42  
    43  [Annotations](https://github.com/opencontainers/image-spec/blob/master/annotations.md)
    44  are an optional part of the OCI specification. They can be included in the
    45  top-level of an OCI index, at the top-level of a manifest, or as part of a
    46  descriptor.
    47  
    48  While they are generally unrestricted key-value pairs, [some guidance is given
    49  for keys by the OCI](https://github.com/opencontainers/image-spec/blob/master/annotations.md#pre-defined-annotation-keys).
    50  
    51  Formalising some common annotations across various artifact types will make them
    52  useful for registries to use to better understand the artifacts.
    53  
    54  ### The future
    55  
    56  There is a clear trend towards more types of artifacts being stored in container
    57  registries. While it's too early to predict exactly what will be stored in
    58  registries, a couple of observations can be made.
    59  
    60  Just as how OCI indices evolved from a need, it's likely that the OCI will
    61  specify new schemas and media types once there are several new artifacts being
    62  stored in registries. This needs to be done, in part, with hindsight so that the
    63  specification captures the requirements for storing all artifacts.
    64  
    65  It's likely that other artifacts will also want to reference multiple images
    66  and/or artifacts. This means that the approach of using an OCI index for a
    67  single object is likely to continue to be valid and useful.
    68  
    69  Agreeing upon and using several common annotations will provide a solid
    70  foundation for future specification work. If Helm, CNAB, and whatever comes next
    71  find annotations that work well, these can be promoted to fields of the next OCI
    72  specification.
    73  
    74  ## FAQ
    75  
    76  ### What is CNAB?
    77  
    78  CNAB stands for Cloud Native Application Bundle. It aims to be the equivalent of
    79  a deb (or MSI) package but for all things Cloud Native. See
    80  [this site](https://cnab.io) for more.
    81  
    82  ### What is an OCI index?
    83  
    84  A container image is presented by a registry as a
    85  [manifest](https://github.com/opencontainers/image-spec/blob/master/manifest.md).
    86  Each manifest is platform specific which means that in order to use an image on
    87  multiple platforms, one needs to fetch the correct manifest for that platform.
    88  
    89  Initially this was solved by indicating the platform as part of the tag, e.g.:
    90  `myimage:tag-<platform>`. This is undesirable for base images used on multiple
    91  platforms as it requires platform specific code. As such a manifest list was
    92  added where multiple manifests could be presented behind the same code.
    93  
    94  The client can fetch the manifest list (or
    95  [OCI index](https://github.com/opencontainers/image-spec/blob/master/image-index.md))
    96  and match its platform to those presented so that it gets the correct image
    97  manifest. Registries are content addressable so the manifest can be found using
    98  the digest.
    99  
   100  An example of this is the `golang:alpine` image, note that a Docker manifest
   101  list is the older version of an OCI index and they serve the same purpose:
   102  
   103  ```console
   104  $ DOCKER_CLI_EXPERIMENTAL=enabled docker manifest inspect golang:alpine
   105  {
   106     "schemaVersion": 2,
   107     "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   108     "manifests": [
   109        {
   110           "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   111           "size": 1365,
   112           "digest": "sha256:9ba4afd1011b9151c3967651538b600f19e48eff2ddde987feb2b72ab2c0bb69",
   113           "platform": {
   114              "architecture": "amd64",
   115              "os": "linux"
   116           }
   117        },
   118        {
   119           "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   120           "size": 1572,
   121           "digest": "sha256:89cc8193f7abc4237b0df2417e0b9fa61687017cd507456b21241d9ea4d94dd3",
   122           "platform": {
   123              "architecture": "arm",
   124              "os": "linux",
   125              "variant": "v6"
   126           }
   127        },
   128        {
   129           "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   130           "size": 1572,
   131           "digest": "sha256:6803d818bb3dd6edfccbe35b70477483fc75ed11d925e80e4af443f737146328",
   132           "platform": {
   133              "architecture": "arm64",
   134              "os": "linux",
   135              "variant": "v8"
   136           }
   137        },
   138        {
   139           "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   140           "size": 1572,
   141           "digest": "sha256:923201b72b9dcf9e96290f9f171c34a9c743047d707afe69d9c167d430607db7",
   142           "platform": {
   143              "architecture": "386",
   144              "os": "linux"
   145           }
   146        },
   147        {
   148           "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   149           "size": 1572,
   150           "digest": "sha256:1782df1f6fa4ede250547f7aa491d3d11fa974bc62a1b9b0e493f07c3ba4430f",
   151           "platform": {
   152              "architecture": "ppc64le",
   153              "os": "linux"
   154           }
   155        },
   156        {
   157           "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   158           "size": 1572,
   159           "digest": "sha256:d50b32798b5e99eb046ee557c567c83d25e39cbaf42f1d4f24af708644a69123",
   160           "platform": {
   161              "architecture": "s390x",
   162              "os": "linux"
   163           }
   164        }
   165     ]
   166  }
   167  ```
   168  
   169  ## Getting started
   170  
   171  Clone the project into your GOPATH. You can then build it using:
   172  
   173  ```console
   174  $ make
   175  ```
   176  
   177  You should now have a `cnab-to-oci` binary the `bin/` folder. To run it,
   178  execute:
   179  
   180  ```console
   181  $ bin/cnab-to-oci --help
   182  ```
   183  
   184  ### Prerequisites
   185  
   186  - Make
   187  - Golang 1.9+
   188  - Git
   189  
   190  ### Installing
   191  
   192  For installing, make sure your `$GOPATH/bin` is part of your `$PATH`.
   193  
   194  ```console
   195  $ make install
   196  ```
   197  
   198  This will build and install `cnab-to-oci` into `$GOPATH/bin`.
   199  
   200  ### Usage
   201  
   202  The `cnab-to-oci` binary is a demonstration tool to `push` and `pull` a CNAB
   203  to a registry. It has three commands: `push`, `pull` and `fixup` which are
   204  described in the following sections.
   205  
   206  #### Push
   207  
   208  The `push` command packages a `bundle.json` file into an OCI image index
   209  (falling back to a Docker manifest if the registry does not support this) and
   210  pushes this to the registry. As part of this process, [`fixup`](#fixup) process
   211  is implicitly run.
   212  
   213  ```console
   214  $ bin/cnab-to-oci push examples/helloworld-cnab/bundle.json --target myhubusername/repo
   215  Ensuring image cnab/helloworld:0.1.1 is present in repository docker.io/myhubusername/repo
   216  Image is not present in repository
   217  Mounting descriptor sha256:bbffe37bb3899b1384bf1483cdcff44bd148d52078b4655e69cd23d534ea043d with media type application/vnd.docker.image.rootfs.diff.tar.gzip (size: 203)
   218  Mounting descriptor sha256:e280b57a032b8bb2ab45f26ea67f42b5d47fd5aca2dd63c5bcdbbd1f753f20b7 with media type application/vnd.docker.image.rootfs.diff.tar.gzip (size: 370)
   219  Mounting descriptor sha256:8e3ba11ec2a2b39ab372c60c16b421536e50e5ce64a0bc81765c2e38381bcff6 with media type application/vnd.docker.image.rootfs.diff.tar.gzip (size: 2206542)
   220  Mounting descriptor sha256:58e6f39290459b6563b348052b2a1a8cf2a44fac19a80ae0da36c82a32f151f8 with media type application/vnd.docker.container.image.v1+json (size: 2135)
   221  Copying descriptor sha256:a59a4e74d9cc89e4e75dfb2cc7ea5c108e4236ba6231b53081a9e2506d1197b6 with media type application/vnd.docker.distribution.manifest.v2+json (size: 942)
   222  {"errors":[{"code":"MANIFEST_INVALID","message":"manifest invalid","detail":{}}]}
   223  
   224  Pushed successfully, with digest "sha256:6cabd752cb01d2efb9485225baf7fc26f4322c1f45f537f76c5eeb67ba8d83e0"
   225  ```
   226  
   227  **Note:** if your images -invocation images as well as service images- are not already
   228  pushed on a registry, `cnab-to-oci` will try to resolve them locally and push them
   229  from your docker daemon image store.
   230  
   231  **Note:** The `MANIFEST_INVALID` error in the above case is because the Docker Hub
   232  does not currently support the OCI image index type.
   233  
   234  **Note**: When using the Docker Hub, no tag will show up in the Hub interface.
   235  The artifact must be referenced by its SHA - see [`pull`](#pull).
   236  
   237  #### Pull
   238  
   239  The `pull` command is used to fetch a CNAB packaged as an OCI image index or
   240  Docker manifest from a registry. This must be done using the digest returned by
   241  the [`push`](#push) command. By default the output is saved to `pulled.json`.
   242  
   243  ```console
   244  $ bin/cnab-to-oci pull myhubusername/repo@sha256:6cabd752cb01d2efb9485225baf7fc26f4322c1f45f537f76c5eeb67ba8d83e0
   245  
   246  $ cat pulled.json
   247  {
   248   "name": "helloworld",
   249   "version": "0.1.1",
   250   "description": "A short description of your bundle",
   251   "keywords": [
   252    "helloworld",
   253    "cnab",
   254    "tutorial"
   255   ],
   256   "maintainers": [
   257    {
   258     "name": "Jane Doe",
   259     "email": "jane.doe@example.com",
   260     "url": "https://example.com"
   261    }
   262   ],
   263   "invocationImages": [
   264    {
   265     "imageType": "docker",
   266     "image": "myhubusername/repo@sha256:a59a4e74d9cc89e4e75dfb2cc7ea5c108e4236ba6231b53081a9e2506d1197b6",
   267     "size": 942,
   268     "mediaType": "application/vnd.docker.distribution.manifest.v2+json"
   269    }
   270   ],
   271   "images": null,
   272   "parameters": null,
   273   "credentials": null
   274  }
   275  ```
   276  
   277  #### Fixup
   278  
   279  The `fixup` command resolves all the image digest references (for the
   280  _invocationImages_ as well as the _images_ in the `bundle.json`) from the
   281  relevant registries and pushes them to the _target_ repository to ensure they're
   282  available to anyone who has access to the CNAB in the target repository. A
   283  patched `bundle.json` is saved by default to `fixed-bundle.json`
   284  
   285  ```console
   286  $ bin/cnab-to-oci fixup examples/helloworld-cnab/bundle.json --target myhubusername/repo
   287  Ensuring image cnab/helloworld:0.1.1 is present in repository docker.io/myhubusername/repo
   288  Image is not present in repository
   289  Mounting descriptor sha256:bbffe37bb3899b1384bf1483cdcff44bd148d52078b4655e69cd23d534ea043d with media type application/vnd.docker.image.rootfs.diff.tar.gzip (size: 203)
   290  Mounting descriptor sha256:e280b57a032b8bb2ab45f26ea67f42b5d47fd5aca2dd63c5bcdbbd1f753f20b7 with media type application/vnd.docker.image.rootfs.diff.tar.gzip (size: 370)
   291  Mounting descriptor sha256:8e3ba11ec2a2b39ab372c60c16b421536e50e5ce64a0bc81765c2e38381bcff6 with media type application/vnd.docker.image.rootfs.diff.tar.gzip (size: 2206542)
   292  Mounting descriptor sha256:58e6f39290459b6563b348052b2a1a8cf2a44fac19a80ae0da36c82a32f151f8 with media type application/vnd.docker.container.image.v1+json (size: 2135)
   293  Copying descriptor sha256:a59a4e74d9cc89e4e75dfb2cc7ea5c108e4236ba6231b53081a9e2506d1197b6 with media type application/vnd.docker.distribution.manifest.v2+json (size: 942)
   294  
   295  $ cat fixed-bundle.json
   296  {
   297    "name": "helloworld",
   298    "version": "0.1.1",
   299    "description": "A short description of your bundle",
   300    "keywords": [
   301      "helloworld",
   302      "cnab",
   303      "tutorial"
   304    ],
   305    "maintainers": [
   306      {
   307        "name": "Jane Doe",
   308        "email": "jane.doe@example.com",
   309        "url": "https://example.com"
   310      }
   311    ],
   312    "invocationImages": [
   313      {
   314        "imageType": "docker",
   315        "image": "myhubusername/repo@sha256:a59a4e74d9cc89e4e75dfb2cc7ea5c108e4236ba6231b53081a9e2506d1197b6",
   316        "size": 942,
   317        "mediaType": "application/vnd.docker.distribution.manifest.v2+json"
   318      }
   319    ],
   320    "images": null,
   321    "parameters": null,
   322    "credentials": null
   323  }
   324  ```
   325  
   326  **Note:** In the above example, the invocation image reference now matches the
   327  target repository.
   328  
   329  ### Example
   330  
   331  The following is an example of an OCI image index sent to the registry.
   332  
   333  ```json
   334  {
   335    "schemaVersion": 2,
   336    "manifests": [
   337      {
   338        "mediaType": "application/vnd.oci.image.manifest.v1+json",
   339        "digest": "sha256:d59a1aa7866258751a261bae525a1842c7ff0662d4f34a355d5f36826abc0341",
   340        "size": 285,
   341        "annotations": {
   342          "io.cnab.manifest.type": "config"
   343        }
   344      },
   345      {
   346        "mediaType": "application/vnd.oci.image.manifest.v1+json",
   347        "digest": "sha256:196d12cf6ab19273823e700516e98eb1910b03b17840f9d5509f03858484d321",
   348        "size": 506,
   349        "annotations": {
   350          "io.cnab.manifest.type": "invocation"
   351        }
   352      },
   353      {
   354        "mediaType": "application/vnd.oci.image.manifest.v1+json",
   355        "digest": "sha256:6bb891430fb6e2d3b4db41fd1f7ece08c5fc769d8f4823ec33c7c7ba99679213",
   356        "size": 507,
   357        "annotations": {
   358          "io.cnab.component.name": "image-1",
   359          "io.cnab.manifest.type": "component"
   360        }
   361      }
   362    ],
   363    "annotations": {
   364      "io.cnab.keywords": "[\"keyword1\",\"keyword2\"]",
   365      "io.cnab.runtime_version": "v1.0.0",
   366      "org.opencontainers.artifactType": "application/vnd.cnab.manifest.v1",
   367      "org.opencontainers.image.authors": "[{\"name\":\"docker\",\"email\":\"docker@docker.com\",\"url\":\"docker.com\"}]",
   368      "org.opencontainers.image.description": "description",
   369      "org.opencontainers.image.title": "my-app",
   370      "org.opencontainers.image.version": "0.1.0"
   371    }
   372  }
   373  ```
   374  
   375  The first manifest in the manifest list references the CNAB configuration. An
   376  example of this follows:
   377  
   378  ```json
   379  {
   380    "schemaVersion": 2,
   381    "config": {
   382      "mediaType": "application/vnd.cnab.config.v1+json",
   383      "digest": "sha256:4bc453b53cb3d914b45f4b250294236adba2c0e09ff6f03793949e7e39fd4cc1",
   384      "size": 578
   385    },
   386    "layers": []
   387  }
   388  ```
   389  
   390  Subsequent manifests in the manifest list are standard OCI images.
   391  
   392  This example proposes two OCI specification and registry changes:
   393  1. It proposes the addition of an `org.opencontainers.artifactType` annotation to be included in the OCI specification.
   394  1. It requires that registries support the `application/vnd.cnab.config.v1+json` media type for a config type.
   395  
   396  ## Development
   397  
   398  ### Running the tests
   399  
   400  ```console
   401  $ make test
   402  ```
   403  
   404  ### Running the e2e tests
   405  
   406  ```console
   407  $ make e2e
   408  ```
   409  
   410  ## Contributing
   411  
   412  Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of
   413  conduct, and the process for submitting pull requests to us.
   414  
   415  ## Maintainers
   416  
   417  See also the list of [maintainers](MAINTAINERS) who participated in this
   418  project.
   419  
   420  ## Contributors
   421  
   422  See also the list of
   423  [contributors](https://github.com/docker/cnab-to-oci/graphs/contributors) who
   424  participated in this project.
   425  
   426  ## License
   427  
   428  This project is licensed under the Apache 2 License - see the [LICENSE](LICENSE)
   429  file for details.