github.com/coreos/rocket@v1.30.1-0.20200224141603-171c416fac02/Documentation/devel/distribution-point.md (about) 1 # Distribution points 2 3 A *distribution point* represents a method for fetching a container image from an input string. This string does not specify an image's type. 4 5 A distribution point can provide one or more image formats. Some *Docker* registries also provide *OCI Images*. *Rkt* can fetch a Docker/OCI image from a registry and convert it on the fly to its native image format, ACI. The [docker2aci][docker2aci_GH] tool can perform this conversion in advance. 6 7 Before distribution points, rkt used *ImageTypes*. These mapped a specifically formatted input string to things like the distribution, transport and image type. This information is hidden now since all images are appc ACIs. 8 9 Distribution points are used as the primary expression of container image information in the different layers of rkt. This includes fetching and referencing in a CAS/ref store. 10 11 ## Distribution points types 12 13 Distribution points are either direct or indirect. Direct distribution points provide the final information needed to fetch the image. Indirect distribution points take some indirect steps, like discovery, before getting the final image location. An indirect distribution point may resolve to a direct distribution point. 14 15 ## Distribution points format 16 17 A distribution point is represented as a URI with the URI scheme as "cimd" and the remaining parts (URI opaque data and query/fragments parts) as the distribution point data. See [rfc3986][3986] for more information on this. Distribution points clearly map to a resource name, otherwise they will not fit inside a resource locator (URL). We will then use the term URIs instead of URNs because it's the suggested name from the rfc (and URNs are defined, by rfc2141, to have the `urn` scheme). 18 19 Every distribution starts the same: `cimd:DISTTYPE:v=uint32(VERSION):` where 20 21 * `cimd` is the *container image distribution scheme* 22 * `DISTTYPE` is the *distribution type* 23 * `v=uint32(VERSION)` is the *distribution type format version* 24 25 ## Current rkt distribution points 26 27 Rkt has three types of distribution points: 28 29 * `Appc` 30 * `ACIArchive` 31 * `Docker` 32 33 ### Appc 34 35 This is an indirect distribution point. 36 37 Appc defines a distribution point using appc image discovery 38 39 * The format is: `cimd:appc:v=0:name?label01=....&label02=....` 40 * The distribution type is "appc" 41 * The labels values must be Query escaped 42 43 **Example:** `cimd:appc:v=0:coreos.com/etcd?version=v3.0.3&os=linux&arch=amd64` 44 45 ### ACIArchive 46 47 This is a direct distribution point since it directly define the final image location. 48 49 ACIArchive defines a distribution point using an archive file 50 51 * The format is: `cimd:aci-archive:v=0:ArchiveURL?query...` 52 * The distribution type is "aci-archive" 53 * ArchiveURL must be query escaped 54 55 **Examples**: 56 57 * `cimd:aci-archive:v=0:file%3A%2F%2Fabsolute%2Fpath%2Fto%2Ffile` 58 * `cimd:aci-archive:v=0:https%3A%2F%2Fexample.com%2Fapp.aci` 59 60 ### Docker 61 62 Docker is an indirect distribution point. 63 64 This defines a distribution point using a docker registry 65 66 The format is: 67 68 * `cimd:docker:v=0:[REGISTRY_HOST[:REGISTRY_PORT]/]NAME[:TAG|@DIGEST]` 69 * Removing the common distribution point section, the format is the same as the docker image string format (man docker-pull). 70 71 **Examples**: 72 73 * `cimd:docker:v=0:busybox` 74 * `cimd:docker:v=0:busybox:latest` 75 * `cimd:docker:v=0:registry-1.docker.io/library/busybox@sha256:a59906e33509d14c036c8678d687bd4eec81ed7c4b8ce907b888c607f6a1e0e6` 76 77 ### Future distribution points 78 79 #### OCI Image distribution(s) 80 81 This is an Indirect distribution point. 82 83 OCI images can be retrieved using a Docker registry but in future the OCI image spec will define one or more own kinds of distribution starting from an image name (with additional tags/labels). 84 85 #### OCI Image layout 86 87 This is a Direct distribution point. 88 89 This can fetch an image starting from a [OCI image layout][oci_layout] format. The 'location' can point to: 90 91 * A single file archive 92 * A local directory based layout 93 * A remote directory based layout 94 * Other types of locations 95 96 This will probably end up being the final distribution used by the above OCI image distributions (like ACIArchive is the final distribution point for the Appc distribution point): 97 98 * `cimd:oci-image-layout:v=0:file%3A%2F%2Fabsolute%2Fpath%2Fto%2Ffile?ref=refname` 99 * `cimd:oci-image-layout:v=0:https%3A%2F%2Fdir%2F?ref=refname` 100 101 Since the OCI image layout can provide multiple images selectable by a ref, one needs to specify which ref to use in the archive distribution URI (see the above ref query parameter). Since distribution only covers one image, it is not possible to import all refs with a single distribution URI. 102 103 **TODO(sgotti)**: Define if oci-image-layout. It should internally handle both archive and directory based layouts or use two different distributions or a query parameter the explicitly define the layout (to avoid guessing if the URL points to a single file or to a directory).* 104 105 **Note** Considering [this OCI image spec README section][oci_image_spec_readme], the final distribution format will probably be similar to the Appc distribution. There is a need to distinguish their User Friendly string (prepending an appc: or oci: ?). 106 107 To sum it up: 108 109 | Distribution Point | Type | URI Format | Final Distribution Point | 110 |--------------------|----------|---------------------------------------------------------------------------|--------------------------| 111 | Appc | Direct | `cimd:appc:v=0:name?label01=....&label02=...` | ACIArchive | 112 | Docker | Direct | `cimd:docker:v=0:[REGISTRY_HOST[:REGISTRY_PORT]/]NAME[:TAG|@DIGEST]` | <none> | 113 | ACIArchive | Indirect | `cimd:aci-archive:v=0:ArchiveURL?query...` | | 114 | OCI | Direct | `cimd:oci:v=0:TODO` | OCIImageLayout | 115 | OCIImageLayout | Indirect | `cimd:oci-image-layout:v=0:URL?ref=...` | | 116 117 ## User-friendly distribution strings 118 119 The distribution URI can be long and complex. It is helpful to have a friendly string for users to request an image with. Rkt supports a couple of image string input styles. These are mapped to an `AppImageType`: 120 121 * Appc discovery string: `example.com/app01:v1.0.0,label01=value01,...` or `example.com/app01,version=v1.0.0,label01=value01,...` etc. 122 * File paths are absolute (`/full/path/to/file`) or relative. 123 124 The above two may overlap so some heuristic is needed to distinguish them (removing this heuristic will break backward compatibility in the CLI). 125 126 * File URL: `file:///full/path/to/file` 127 * Http(s) URL: `http(s)://host:port/path` 128 * Docker URL: This is a strange URL since it the schemeful (`docker://`) version of the docker image string 129 130 To maintain backward compatibility these image string will be converted to a distribution URI: 131 132 | Current ImageType | Distribution Point URI | 133 |----------------------------------------|---------------------------------------------------------------------------| 134 | appc string | `cimd:appc:v=0:name?label01=....&label02=...` | 135 | file path | `cimd:aci-archive:v=0:ArchiveURL` | 136 | file URL | `cimd:aci-archive:v=0:ArchiveURL` | 137 | https URL | `cimd:aci-archive:v=0:ArchiveURL` | 138 | docker URI/URL (docker: and docker://) | `cimd:docker:v=0:[REGISTRY_HOST[:REGISTRY_PORT]/]NAME[:TAG|@DIGEST]` | 139 140 The above table also adds Docker URI (`docker:`) as a user friendly string and its clearer than the URL version (`docker://`) 141 142 The parsing and generation of user friendly string is done outside the distribution package (to let distribution pkg users implement their own user friendly strings). 143 144 Rkt has two jobs: 145 146 * Parse a user friendly string to a distribution URI. 147 * Generate a user friendly string from a distribution URI. This is useful when showing the refs from a refs store. They can easily be understood and copy/pasted. 148 149 A user can provide as an input image as a "user friendly" string or a complete distribution URI. 150 151 ## Comparing Distribution Points URIs 152 153 A Distribution Point implementation will also provide a function to compare if Distribution Point URIs are the same (e.g. ordering the query parameters). 154 155 ## Fetching logic with Distribution Points 156 157 A Distribution Point will be the base for a future refactor of the fetchers logic (see [#2964][rkt-2964]) 158 159 This also creates a better separation between the distribution points and the transport layers. 160 161 For example there may exist multiple transport plugins (file, http, s3, bittorrent etc...) to be called by an ACIArchive distribution point. 162 163 164 [3986]: https://tools.ietf.org/html/rfc3986 165 [docker2aci_GH]: https://github.com/appc/docker2aci 166 [oci_image_spec_readme]: https://github.com/opencontainers/image-spec#running-an-oci-image 167 [oci_layout]: https://github.com/opencontainers/image-spec/blob/master/image-layout.md 168 [rkt-2964]: https://github.com/rkt/rkt/pull/2964