github.com/alibaba/sealer@v0.8.6-0.20220430115802-37a2bdaa8173/applications/rookceph/rookceph-object/README.md (about)

     1  # Overview
     2  
     3  This image will create a CephObjectStore that starts the RGW service in the cluster with an S3 API and also will create
     4  default storage class named `rook-ceph-bucket` for use.
     5  
     6  Components included in this image:
     7  
     8  Ceph cluster:
     9  
    10  * 1 Deployment for rookceph operator.
    11  * 3 ceph mon for ceph cluster.
    12  * 3 ceph osd for ceph cluster.
    13  * 2 ceph mgr for ceph cluster.
    14  * enable ceph dashboard with ssl port 8443.
    15  
    16  CephObjectStore:
    17  
    18  * 3 replicated datapool for ceph filesystem.
    19  * 3 replicated metadatapool for ceph filesystem.
    20  * 1 object store gateway with port 80.
    21  
    22  ## How to run it
    23  
    24  Use default Clusterfile to apply the ceph cluster.
    25  
    26  see : [default ceph object store Clusterfile examples](../../../applications/rookceph/rookceph-object/examples/Clusterfile.yaml)
    27  
    28  ## How to use it
    29  
    30  Connect to ceph cluster using below tools.Then run `ceph status` to check the status of ceph cluster.
    31  
    32  ```yaml
    33  apiVersion: apps/v1
    34  kind: Deployment
    35  metadata:
    36    name: rook-ceph-tools
    37    namespace: rook-ceph
    38    labels:
    39      app: rook-ceph-tools
    40  spec:
    41    replicas: 1
    42    selector:
    43      matchLabels:
    44        app: rook-ceph-tools
    45    template:
    46      metadata:
    47        labels:
    48          app: rook-ceph-tools
    49      spec:
    50        dnsPolicy: ClusterFirstWithHostNet
    51        containers:
    52          - name: rook-ceph-tools
    53            image: rook/ceph:v1.7.2
    54            command: [ "/tini" ]
    55            args: [ "-g", "--", "/usr/local/bin/toolbox.sh" ]
    56            imagePullPolicy: IfNotPresent
    57            env:
    58              - name: ROOK_CEPH_USERNAME
    59                valueFrom:
    60                  secretKeyRef:
    61                    name: rook-ceph-mon
    62                    key: ceph-username
    63              - name: ROOK_CEPH_SECRET
    64                valueFrom:
    65                  secretKeyRef:
    66                    name: rook-ceph-mon
    67                    key: ceph-secret
    68            volumeMounts:
    69              - mountPath: /etc/ceph
    70                name: ceph-config
    71              - name: mon-endpoint-volume
    72                mountPath: /etc/rook
    73        volumes:
    74          - name: mon-endpoint-volume
    75            configMap:
    76              name: rook-ceph-mon-endpoints
    77              items:
    78                - key: data
    79                  path: mon-endpoints
    80          - name: ceph-config
    81            emptyDir: { }
    82        tolerations:
    83          - key: "node.kubernetes.io/unreachable"
    84            operator: "Exists"
    85            effect: "NoExecute"
    86            tolerationSeconds: 5
    87  
    88  ```
    89  
    90  Launch the rook-ceph-tools pod:
    91  
    92  `kubectl create -f toolbox.yaml`
    93  
    94  Wait for the toolbox pod to download its container and get to the running state:
    95  
    96  `kubectl -n rook-ceph rollout status deploy/rook-ceph-tools`
    97  
    98  Once the rook-ceph-tools pod is running, you can connect to it with:
    99  
   100  `kubectl -n rook-ceph exec -it deploy/rook-ceph-tools -- bash`
   101  
   102  Use ceph as the object store backend act as AWS S3.
   103  
   104  create a bucket.
   105  
   106  ```yaml
   107  apiVersion: objectbucket.io/v1alpha1
   108  kind: ObjectBucketClaim
   109  metadata:
   110    name: ceph-bucket
   111  spec:
   112    generateBucketName: rookbucket
   113    storageClassName: rook-ceph-bucket
   114  ```
   115  
   116  Client Connections
   117  
   118  ```shell
   119  #config-map, secret, OBC will part of default if no specific name space mentioned
   120  export AWS_BUCKET_NAME=$(kubectl -n default get cm ceph-bucket -o jsonpath='{.data.BUCKET_NAME}')
   121  export AWS_ACCESS_KEY_ID=$(kubectl -n default get secret ceph-bucket -o jsonpath='{.data.AWS_ACCESS_KEY_ID}' | base64 --decode)
   122  export AWS_SECRET_ACCESS_KEY=$(kubectl -n default get secret ceph-bucket -o jsonpath='{.data.AWS_SECRET_ACCESS_KEY}' | base64 --decode)
   123  ```
   124  
   125  Get ceph-rgw service cluster ip and port.
   126  
   127  ```shell
   128  kubectl -n rook-ceph get svc rook-ceph-rgw-my-store
   129  NAME                     TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
   130  rook-ceph-rgw-my-store   ClusterIP   10.96.3.195   <none>        80/TCP    3h38m
   131  ```
   132  
   133  Install the s3cmd tool into the toolbox pod.
   134  
   135  `yum -y install s3cmd`
   136  
   137  Edit s3cmd config file at : "$HOME/.s3cfg"
   138  
   139  example :
   140  
   141  ```shell
   142  access_key = ${AWS_ACCESS_KEY_ID}
   143  secret_key = ${AWS_SECRET_ACCESS_KEY}
   144  host_bucket = 10.96.3.195/${AWS_BUCKET_NAME}
   145  host_base = 10.96.3.195:80
   146  use_https = False
   147  ```
   148  
   149  Test the CephObjectStore to upload a file.
   150  
   151  ```shell
   152  echo "Hello Rook" > /tmp/rookObj
   153  s3cmd put /tmp/rookObj s3://rookbucket-7d02c61a-892f-4dc8-a947-9a2234ae0610
   154  ```
   155  
   156  Download and verify the file from the bucket.
   157  
   158  ```shell
   159  s3cmd get s3://rookbucket-7d02c61a-892f-4dc8-a947-9a2234ae0610/rookObj
   160  cat rookObj
   161  ```
   162  
   163  ## How to rebuild it
   164  
   165  Modify manifest.yaml or cephobject.yaml file according to your needs, then run below command to rebuild it.
   166  
   167  ```shell
   168  sealer build -t {Your Image Name} -f Kubefile -m cloud .
   169  ```
   170  
   171  More parameters see [official docs here](https://rook.io/docs/rook/v1.7/ceph-object.html).