github.com/opencontainers/umoci@v0.4.8-0.20240508124516-656e4836fb0d/doc/site/reference/roadmap.md (about)

     1  +++
     2  title = "Roadmap"
     3  weight = 15
     4  +++
     5  
     6  This document describes an informal roadmap for the future of the umoci
     7  project, in both technical and operational aspects. If you feel there is
     8  something missing from this document, feel free to
     9  [contribute][contributing.md].
    10  
    11  [contributing.md]: /contributing
    12  
    13  ### Improved API Design ###
    14  
    15  One of the main goals of umoci is to act as a "defacto" implementation of the
    16  relevant image-spec interfaces. In particular, the `oci/cas` and `oci/casext`
    17  packages are perfect examples of the style of interface that would be
    18  incredibly reusable. In addition, the `oci/cas/driver` model allows for
    19  different backend implementations to be used as long as they provide the
    20  `oci/cas.Engine` interface.
    21  
    22  However, at the moment several of the other packages in `oci/` need some love
    23  in creating proper re-usable interfaces. There are many useful components in
    24  `oci/layer` (for example), but they are all hidden behind the top-level
    25  entrypoints that are used by `umoci unpack` and `umoci repack`. Other `oci/`
    26  libraries have similar issues. This is arguably the highest priority item in
    27  this roadmap, as it is blocking the adoption of our `oci/` libraries as well as
    28  making other refactors much harder to do.
    29  
    30  Also note that the `mutate/` package should also be reworked to provide a much
    31  cleaner interface (as well as providing support for proper nested handling). To
    32  be quite honest, I'm not actually sure if it handles the arbitrary nesting
    33  feature of `1.0.0-rc5` and later (and that's an issue we need to resolve).
    34  
    35  ### Arbitrary Tree Structures Interface ###
    36  
    37  With the upgrade to `1.0.0-rc5` of the image-spec, it is now possible to have
    38  arbitrarily nested `Index` objects. This means that users may wish to create
    39  arbitrary tree structures, as well as interact with such structures. Noting
    40  that ultimately a user will just want to interact with a `Manifest` (at the end
    41  of the day), there are several things that need to be improved in the interface
    42  design.
    43  
    44  * When querying an image to get a `Manifest`, we need to have a "resolution"
    45    interface (which must also be operated in a non-interactive way) that allows
    46    for increasing the specificity when describing what manifest we are
    47    interested in. After applying `org.opencontainers.image.ref.name` and
    48    platform filters, we have to provide additional filtering and referencing
    49    support. Unfortunately it currently is not very clear how this is meant to
    50    work with an arbitrary image, or if it is expected the referencing will
    51    always be contextual for a particular image. In addition, there is no part of
    52    the spec that actually describes the algorithm for such referencing.
    53  
    54  * When replacing a `Manifest`, we just need to replace that manifest and all of
    55    the objects along the resolution path. Unfortunately, the last part of that
    56    requirement is quite complicated to deal with now that dereferencing has been
    57    punted to us (especially when it comes to the concept of creating a new tag
    58    for the modified object -- should we therefore re-create the entire tree of
    59    the original "reference" and work from there?). Ultimately I feel it may be
    60    required to store the `Manifest` descriptor as well as the highest-level
    61    (from the `index.json`) descriptor so that umoci knows where to re-create
    62    the structure from.
    63  
    64  * When creating a new `Manifest` in an image, we need to be able to handle any
    65    arbitrary tree structure. Ultimately I think the only proper way of handling
    66    this will be to require the user to provide a specification-style
    67    description. I have toyed with the idea of automatically restructuring images
    68    as necessary (something that would be required if you wanted to avoid the
    69    creation of duplicate tags in the top-level image -- something that I really
    70    would like to make sure happens) but it's not clear if users will always want
    71    us to be maximally clever. Maybe we should provide that and in addition
    72    provide the specification-style interface for power users (then we have to
    73    figure out what happens if the two schemes clash or if the specification
    74    invalidates our own preferences).
    75  
    76  I'm really hoping the above issues may be solvable through some sort of fancy
    77  algorithm (especially when it comes to resolving conflicts). Further research
    78  is required in this area.
    79  
    80  An additional point to the above is that the current `--image` interface is
    81  quite ugly (mainly because I copied Docker's referencing interface). We really
    82  should dispense with this and go straight to [proper URI semantics][rfc3986]
    83  with `#partial` tags being used to indicate `org.opencontainers.image.ref.name`.
    84  In addition, I really would like `--layout` to no longer be necessary. Maybe if
    85  we made `#partial` tags required this would allow us to combine the two flags,
    86  though it would break the ability to have a "default" tag.
    87  
    88  [rfc3986]: https://tools.ietf.org/html/rfc3986
    89  
    90  ### Direct `mtree`-from-layer Generation ###
    91  
    92  Currently umoci generates its `mtree(8)` manifest from the real filesystem
    93  after the entire bundle has been extracted. This works well enough in practice
    94  (and actually guarantees us that a `repack` will contain an empty archive if we
    95  just did an `unpack` and nothing else). However, it has several issues.
    96  
    97  * With `--rootless` we do several tricks to make `umoci unpack` work as an
    98    unprivileged user. Many of the tricks don't change how things are extracted
    99    (such as the whole `pkg/unpriv` trick), but there are several cases where we
   100    will intentionally incorrectly extract objects to avoid privilege errors.
   101    Examples of this include `security.capabilities` xattrs (which are currently
   102    not user namespace safe). In order for a from-layer system to work, we would
   103    need to modify the generated `mtree(8)` manifest so that it matches the
   104    on-disk state that we expect given what tricks we know we will employ.
   105  
   106  * In addition, binding the `mtree(8)` tricks to the layer representation will
   107    mean that any future improvements to the OCI image-spec (such as removing the
   108    need for linear archives) will result in far more complicated changes to
   109    umoci. However, if the generator is designed in an extensible fashion this
   110    should not be a huge deal.
   111  
   112  Vincent Batts (the author of `gomtree`) has expressed interest in this sort of
   113  functionality, but I'm not entirely convinced that it will work as well as
   114  expected (especially given the enormous amount of special casing in `go-mtree`
   115  that already exists in order to implement `tar` archive support).