github.com/uber/kraken@v0.1.4/docs/INTEGRATEWITHHARBOR.md (about)

     1  ## Concept
     2  Harbor is the only container repository project in CNCF. And it supports image management, replication, CVE scan and so on. It could be used in production.
     3  
     4  Kraken is also a container repository with P2P distribution. It supports S3, HDFS, docker registry as backend storage.
     5  
     6  I will introduce how to integrate Harbor and Kraken here. And using the image management function of Harbor and the P2P distribution function of Kraken both.
     7  
     8  ## Architecture
     9  ![image](https://gitlab.com/pmm123/pics/raw/master/work/p2p/%E9%95%9C%E5%83%8F%E4%BB%93%E5%BA%93P2P_3_.png)
    10  
    11  So you can see that Harbor and Kraken share the same docker registry as backend storage. It will decouple Harbor and Kraken if they just share the storage but not higher level sharing. So we can push images into Harbor and pull them from Harbor or kraken.
    12  
    13  ## Deployment
    14  Here we use [helm chart](https://github.com/goharbor/harbor-helm) from Harbor community to deploy them. and also [helm chart](https://github.com/uber/kraken/tree/master/helm) from Kraken community to deploy them.
    15  
    16  We should modify two config file additionally,
    17  1. modify the backend of kraken-origin and kraken-buildindex and make it refer to harbor-registry.
    18  2. modify config file of harbor-registry. add another notification hook for kraken.
    19  
    20  As following,
    21  
    22  harbor-registry configmap
    23  ```
    24  ## add the second endpoint in notifications
    25      notifications:
    26        endpoints:
    27          - name: harbor
    28            disabled: false
    29            url: http://harbor-core.repo.svc.cluster.local/service/notifications
    30            timeout: 3000ms
    31            threshold: 5
    32            backoff: 1s
    33          - name: kraken
    34            disabled: false
    35            url: http://kraken-proxy.p2p.svc.cluster.local:10050/registry/notifications
    36            timeout: 3000ms
    37            threshold: 5
    38            backoff: 1s
    39  ```
    40  
    41  kraken-origin,kraken-buildindex,
    42  
    43  ```
    44      backends:
    45        - namespace: .*
    46          backend:
    47            registry_blob:
    48              address: harbor-registry.repo.svc.cluster.local:5000
    49              security:
    50                basic:
    51                  username: "admin"
    52                  password: "XXXXX"
    53  ```
    54  
    55  You should add more configuration about TLS if the domain name of your Harbor use a self-signed SSL certificate. It will throws a X509 error if not.
    56  
    57  ```
    58      backends:
    59        - namespace: .*
    60          backend:
    61            registry_blob:
    62              address: harbor-registry.repo.svc.cluster.local:5000
    63              security:
    64                basic:
    65                  username: "admin"
    66                  password: "XXXXX"
    67                tls:
    68                  client:
    69                    cert:
    70                      path: /etc/certs/XXX.crt
    71                    key:
    72                      path: /etc/certs/XXX.key
    73                  cas:
    74                    - path: /etc/certs/ca.crt
    75                    
    76  ```
    77  
    78  You can pull images using a `localhost` domain by P2P distribution after you deploy kraken-agent in your k8s nodes using daemonSet.
    79  
    80  For example, there is an image called  `hub.harbor.com/library/debain:latest`, then you can pull it using `localhost:13000/library/debain:latest`.
    81  
    82  ## Work flow
    83  I describe the work flow of pushing and pulling images briefly here,
    84  1. User push an image named `docker push hub.harbor.com/library/debain:latest` to Harbor.
    85  2. Harbor-registry will trigger a notification of pushManifest event to kraken-proxy.
    86  3. Kraken-proxy will fetch the manifest and notify kraken-origin to cache related blobs after receving the pushManifest event.
    87  4. User will try to pull image by `docker pull localhost:13000/library/debain:latest`
    88  5. Then the P2P distribution work flow. but kraken-origin has cached these blobs before P2P distribution starts. it will save the time of fetching blobs from harbor-registry.
    89  6. Pulling image is completed.
    90  
    91  ## Use Case
    92  It has been a long time that this solution is used in Qingzhou Platform in NetEase Cloud.
    93  
    94  Now we use Harbor for managing images and Kraken for distributing images, and it resolves the problem of distribution of huge number of images. there are thousands of images distributed everyday in our Qingzhou Platform.
    95  
    96  ## Notes
    97  It is important to limit the resouce quota of kraken-agent in k8s for it is deployed with containers of business. It will affect the containers of business if kraken-agent takes too much resource.
    98  
    99  There are two aspects for resource quota. One is the resource limit of k8s pod, the other one is the bandwidth.