github.com/Racer159/jackal@v0.32.7-0.20240401174413-0bd2339e4f2e/adr/0014-oci-publish.md (about)

     1  # 14. Jackal Packages as OCI Artifacts
     2  
     3  Date: 2023-03-10
     4  
     5  ## Status
     6  
     7  Accepted
     8  
     9  ## Context
    10  
    11  Jackal packages are currently only available if built locally or through manual file transfers. This is not a scalable way to distribute packages. We wanted to find a way to distribute and publish packages in a way that is easily consumable for the majority of users. When considering the goal of being able to share packages, security and trust are very important considerations. We wanted our publishing solution and architecture changes to keep in mind signing of packages and the ability to verify the integrity of packages.
    12  
    13  We know we are successful when:
    14  
    15  1. (Priority) Users can use Jackal to natively publish a Jackal package to an OCI compliant registry
    16  2. (Secondary goal) Package creators can sign Jackal packages to enable package deployers can trust a packages supply chain security
    17  
    18  ## Decision
    19  
    20  We decided that changing the structure of Jackal packages to be an OCI artifact would be the best way to distribute and publish packages as registries are already an integral part of the container ecosystem.
    21  
    22  ## Implementation
    23  
    24  A handful of changes were introduced to the structure of Jackal packages.
    25  
    26  ```text
    27  jackal-package-adr-arm64.tar.zst
    28  ├── checksums.txt
    29  ├── components
    30  │   └── [...].tar
    31  ├── images
    32  │   ├── index.json
    33  │   ├── oci-layout
    34  │   └── blobs
    35  │       └── sha256
    36  │           └── ... # OCI image layers
    37  ├── sboms.tar
    38  └── jackal.yaml
    39  ```
    40  
    41  - Each component folder is now a tarball instead of a directory
    42    - This enables us to treat each component as a layer within the package artifact
    43  - Images are now stored in a flattened state instead of an images.tar file
    44    - This enables us to keep each image layer as a layer within the package artifact (allowing for server side de-duping)
    45  - SBOM files are now stored in a tarball instead of a directory
    46    - This enables us to treat the SBOM artifacts as a single layer within the package artifact
    47  
    48  With this new structure in place, we can now publish Jackal packages as OCI artifacts. Under the hood this implements the `oras` Go library using Docker's authentication system. For interacting with these packages, the `oci://` package path prefix has been added (ex. `jackal package publish oci://...`).
    49  
    50  For an example of this in action, please see the corresponding [tutorial](../docs/5-jackal-tutorials/7-publish-and-deploy.md).
    51  
    52  ## Consequences
    53  
    54  Backwards compatibility was an important considering when making these changes. We had to implement logic to make sure a new version of the Jackal binary could still operate with older versions of Jackal packages.
    55  
    56  At the moment we are testing the backwards compatibility by virtue of maintaining the `./src/test/e2e/27_cosign_deploy_test.go` where we are deploying an old Jackal package via `sget` (which itself is now deprecated).
    57  
    58  One thing we may want to look at more in the future is how we can get more intricate tests around the backwards compatibility.
    59  
    60  The reason why testing backwards compatibility is difficult is because this isn't a `jackal.yaml` schema change (like we had recently with the 'Scripts to Actions' PR) but an compiled package architecture change. This means that we will either need to maintain an 'old' Jackal package that will follow future `jackal.yaml` schema changes OR we maintain a modified Jackal binary that creates the old package structure.