github.com/containerd/nerdctl/v2@v2.0.0-beta.5.0.20240520001846-b5758f54fa28/examples/nerdctl-ipfs-registry-kubernetes/ipfs/README.md (about)

     1  # Example: Node-to-Node image sharing on Kubernetes using `nerdctl ipfs registry`
     2  
     3  This directory contains an example Kubernetes setup for node-to-node image sharing.
     4  
     5  Usage:
     6  - Generate `bootstrap.yaml` by executing `bootstrap.yaml.sh` (e.g. `./bootstrap.yaml.sh > ${DIR_LOCATION}/bootstrap.yaml`)
     7    - [`ipfs-swarm-key-gen`](https://github.com/Kubuxu/go-ipfs-swarm-key-gen) is required (see https://github.com/ipfs/kubo/blob/v0.15.0/docs/experimental-features.md#private-networks)
     8  - Deploy `bootstrap.yaml` and `nerdctl-ipfs-registry.yaml` (e.g. using `kubectl apply`)
     9  - Make sure nodes contain containerd >= v1.5.8
    10  - You might want to change some configuration written in `nerdctl-ipfs-registry.yaml` (e.g. [chaning profile based on your node's resouce requirements](https://docs.ipfs.tech/how-to/default-profile/#available-profiles))
    11  
    12  ## Example on kind
    13  
    14  Prepare cluster (make sure kind nodes contain containerd >= v1.5.8).
    15  
    16  ```console
    17  $ cat <<EOF > /tmp/kindconfig.yaml
    18  kind: Cluster
    19  apiVersion: kind.x-k8s.io/v1alpha4
    20  nodes:
    21  - role: control-plane
    22  - role: worker
    23  - role: worker
    24  EOF
    25  $ kind create cluster --image=kindest/node:v1.25.2 --config=/tmp/kindconfig.yaml
    26  $ ./bootstrap.yaml.sh > ./bootstrap.yaml
    27  $ kubectl apply -f .
    28  ```
    29  
    30  Prepare `kind-worker` (1st node) for importing an image to IPFS
    31  
    32  (in `kind-worker`)
    33  
    34  ```console
    35  $ docker exec -it kind-worker /bin/bash
    36  (kind-worker)# NERDCTL_VERSION=0.23.0
    37  (kind-worker)# curl -sSL --output /tmp/nerdctl.tgz https://github.com/containerd/nerdctl/releases/download/v${NERDCTL_VERSION}/nerdctl-${NERDCTL_VERSION}-linux-amd64.tar.gz
    38  (kind-worker)# tar zxvf /tmp/nerdctl.tgz -C /usr/local/bin/
    39  ```
    40  
    41  Add an image to `kind-worker`.
    42  
    43  ```console
    44  $ docker exec -it kind-worker /bin/bash
    45  (kind-worker)# mkdir -p /tmp/ipfsapi ; echo -n /ip4/127.0.0.1/tcp/5001 >  /tmp/ipfsapi/api
    46  (kind-worker)# export IPFS_PATH=/tmp/ipfsapi
    47  (kind-worker)# nerdctl pull ghcr.io/stargz-containers/jenkins:2.60.3-org
    48  (kind-worker)# nerdctl push ipfs://ghcr.io/stargz-containers/jenkins:2.60.3-org
    49  (kind-worker)# nerdctl rmi ghcr.io/stargz-containers/jenkins:2.60.3-org
    50  ```
    51  
    52  The image added to `kind-worker` is shared to `kind-worker2` via IPFS.
    53  You can run this image on all worker nodes using the following manifest.
    54  CID of the pushed image is printed when `nerdctl push` succeeded (we assume that the image is added to IPFS as CID `bafkreictyyoysj56v772xbfhyfrcvmgmfpa4vodmqaroz53ytvai7nof6u`).
    55  
    56  ```console
    57  $ cat <<EOF | kubectl apply -f -
    58  apiVersion: apps/v1
    59  kind: Deployment
    60  metadata:
    61    name: jenkins
    62  spec:
    63    replicas: 2
    64    selector:
    65      matchLabels:
    66        app: jenkins
    67    template:
    68      metadata:
    69        labels:
    70          app: jenkins
    71      spec:
    72        containers:
    73        - name: jenkins
    74          image: localhost:5050/ipfs/bafkreictyyoysj56v772xbfhyfrcvmgmfpa4vodmqaroz53ytvai7nof6u
    75          resources:
    76            requests:
    77              cpu: 1
    78  EOF
    79  ```
    80  
    81  > NOTE: Kubernetes doesn't support `ipfs://CID` URL on YAML as of now so we need to use `localhost:5050/ipfs/CID` form instead. In the future, this limitation should be eliminated.
    82  
    83  The image runs on all nodes.
    84  
    85  ```console
    86  $ kubectl get pods -owide | grep jenkins
    87  jenkins-7bd8f96d79-2jbc6          1/1     Running   0          69s    10.244.1.3   kind-worker    <none>           <none>
    88  jenkins-7bd8f96d79-jb5lm          1/1     Running   0          69s    10.244.2.4   kind-worker2   <none>           <none>
    89  ```