github.com/1aal/kubeblocks@v0.0.0-20231107070852-e1c03e598921/docs/user_docs/backup-and-restore/backup/backup-repo.md (about) 1 --- 2 title: Configure BackupRepo 3 description: How to configure BackupRepo 4 keywords: [introduction, backup, restore] 5 sidebar_position: 2 6 sidebar_label: Configure BackupRepo 7 --- 8 9 import Tabs from '@theme/Tabs'; 10 import TabItem from '@theme/TabItem'; 11 12 # Introduction 13 14 BackupRepo is the storage repository for backup data, which uses CSI driver to upload backup data to various storage systems, such as object storage systems like S3 and GCS, or storage servers like FTP and NFS. Currently, BackupRepo of Kubeblocks only supports S3 and other S3-compatible storage systems, and more types of storage will be supported in the future. 15 16 Users can create multiple BackupRepos to suit different scenarios. For example, based on different businesses, the data of business A is stored in repository A, and the data of business B is stored in repository B. Or you can configure multiple repositories by region to realize geo-disaster recovery. But it is required to specify backup repositories when creating a backup. You can also create a default backup repository and KubeBlocks uses this default repository to store backup data if no specific repository is specified when creating a backup. 17 18 The following instructions take AWS S3 as an example to demonstrate how to configure BackupRepo. There are two options for configuring BackupRepo. 19 20 * Automatic BackupRepo configuration: you can provide the necessary configuration information (for example, the AccessKey of object storage, etc.) when installing KubeBlocks, and the installer automatically creates a default BackupRepo and the CSI driver add-on required for automatic installation. 21 * Manual BackupRepo configuration: you can install S3 CSI driver and then create BackupRepo manually. 22 23 ## Automatic BackupRepo configuration 24 25 You can specify the BackupRepo information in a YAML configuration file when installing KubeBlocks, and KubeBlocks creates a BackupRepo automatically. 26 27 1. Write the following configuration to the `backuprepo.yaml` file. 28 29 ```yaml 30 backupRepo: 31 create: true 32 storageProvider: s3 33 config: 34 region: cn-northwest-1 35 bucket: test-kb-backup 36 secrets: 37 accessKeyId: <ACCESS KEY> 38 secretAccessKey: <SECRET KEY> 39 ``` 40 41 * `region`: specifies the region where S3 is located. 42 * `bucket`: specifies the bucket name of S3. 43 * `accessKeyId`: specifies the Access Key of AWS. 44 * `secretAccessKey`: specifies the Secret Key of AWS. 45 46 :::note 47 48 1. For KubeBlocks v0.6.0, the available `storageProvider` are `s3`, `oss`, and `minio`. 49 2. For different storage providers, the configuration may differ. `config` and `secrets` in the above example are applied to S3. 50 3. You can run the command below to view the supported storage provider. 51 52 ```bash 53 kubectl get storageproviders.storage.kubeblocks.io 54 ``` 55 56 ::: 57 58 2. Install KubeBlocks. 59 60 ```bash 61 kbcli kubeblocks install -f backuprepo.yaml 62 ``` 63 64 ## Manual BackupRepo configuration 65 66 If you do not configure the BackupRepo information when installing KubeBlocks, you can manually configure it by the following instructions. 67 68 ### Before you start 69 70 There are various ways to create a BackupRepo. Make sure you have done all the necessary preparations before creating it. If you want to use MinIO, you need to make the following configurations in advance. 71 72 1. Install MinIO. 73 74 ```bash 75 helm install minio oci://registry-1.docker.io/bitnamicharts/minio --set "extraEnvVars[0].name=MINIO_BROWSER_LOGIN_ANIMATION" --set "extraEnvVars[0].value=off" 76 ``` 77 To get initial username and password, try the following. 78 79 ```bash 80 # Initial username 81 echo $(kubectl get secret --namespace default minio -o jsonpath="{.data.root-user}" | base64 -d) 82 83 # Initial password 84 echo $(kubectl get secret --namespace default minio -o jsonpath="{.data.root-password}" | base64 -d) 85 ``` 86 87 2. Generate credentials. 88 89 Access the login page by running `kubectl port-forward --namespace default svc/minio 9001:9001` and then accessing `127.0.0.1:9001`. 90 91 Once you are logged in to the dashboard, you can generate an `access key` and `secret key`. 92 93  94 95 3. Create a bucket. 96 97 Create a bucket named `test-minio` in the MinIO dashboard. 98 99  100  101 102 ### Install S3 CSI driver 103 104 <Tabs> 105 106 <TabItem value="kbcli" label="kbcli" default> 107 108 ```bash 109 # Enable the CSI-S3 add-on 110 kbcli addon enable csi-s3 111 112 # You can add flags to customize the installation of this add-on 113 # CSI-S3 install a daemonSet Pod on all nodes by default and you can set tolerations to install it on the specified node 114 kbcli addon enable csi-s3 \ 115 --tolerations '[{"key":"taintkey","operator":"Equal","effect":"NoSchedule","value":"true"}]' \ 116 --tolerations 'daemonset:[{"key":"taintkey","operator":"Equal","effect":"NoSchedule","value":"true"}]' 117 118 # View the status of CSI-S3 driver and make sure it is Enabled 119 kbcli addon list csi-s3 120 ``` 121 122 </TabItem> 123 124 <TabItem value="Helm" label="Helm"> 125 126 ```bash 127 helm repo add kubeblocks https://jihulab.com/api/v4/projects/85949/packages/helm/stable 128 helm install csi-s3 kubeblocks/csi-s3 --version=0.6.0 -n kb-system 129 130 # You can add flags to customize the installation of this add-on 131 # CSI-S3 install a daemonSet Pod on all nodes by default and you can set tolerations to install it on the specified node 132 --set-json tolerations='[{"key":"taintkey","operator":"Equal","effect":"NoSchedule","value":"taintValue"}]' 133 --set-json daemonsetTolerations='[{"key":"taintkey","operator":"Equal","effect":"NoSchedule","value":"taintValue"}]' 134 ``` 135 136 </TabItem> 137 138 </Tabs> 139 140 ### Create BackupRepo 141 142 <Tabs> 143 144 <TabItem value="kbcli" label="kbcli" default> 145 146 <Tabs> 147 148 <TabItem value="S3" label="S3" default> 149 150 ```bash 151 kbcli backuprepo create my-repo \ 152 --provider s3 \ 153 --region cn-northwest-1 \ 154 --bucket test-kb-backup \ 155 --access-key-id <ACCESS KEY> \ 156 --secret-access-key <SECRET KEY> \ 157 --default 158 ``` 159 160 The above command creates a default backup repository `my-repo`. 161 162 * `my-repo` is the name of the created backup repository. If you do not specify a name, the system creates a random name, following the format `backuprepo-xxxxx`. 163 * `--default` means that this repository is set as the default repository. Note that there can only be one default global repository. If there exist multiple default repositories, KubeBlocks cannot decide which one to use (similar to the default StorageClass of Kubernetes), which further results in backup failure. Using `kbcli` to create BackupRepo can avoid such problems because `kbcli` checks whether there is another default repository before creating a new one. 164 * `--provider` specifies the storage type, i.e. `storageProvider`, and is required for creating a BakcupRepo. The available values are `s3`, `oss`, and `minio`. Parameters for different storage providers vary and you can run `kbcli backuprepo create --provider STORAGE-PROVIDER-NAME -h` to view the flags for different storage providers. 165 166 </TabItem> 167 168 <TabItem value="OSS" label="OSS"> 169 170 ```bash 171 kbcli backuprepo create my-repo \ 172 --provider oss \ 173 --region cn-zhangjiakou \ 174 --bucket test-kb-backup \ 175 # --endpoint https://oss-cn-zhangjiakou-internal.aliyuncs.com \ To display the specified oss endpoint 176 --access-key-id <ACCESS KEY> \ 177 --secret-access-key <SECRET KEY> \ 178 --default 179 ``` 180 181 The above command creates a default backup repository `my-repo`. 182 183 * `my-repo` is the name of the created backup repository. If you do not specify a name, the system creates a random name, following the format `backuprepo-xxxxx`. 184 * `--default` means that this repository is set as the default repository. Note that there can only be one default global repository. If there exist multiple default repositories, KubeBlocks cannot decide which one to use (similar to the default StorageClass of Kubernetes), which further results in backup failure. Using `kbcli` to create BackupRepo can avoid such problems because `kbcli` checks whether there is another default repository before creating a new one. 185 * `--provider` specifies the storage type, i.e. `storageProvider`, and is required for creating a BakcupRepo. The available values are `s3`, `oss`, and `minio`. Parameters for different storage providers vary and you can run `kbcli backuprepo create --provider STORAGE-PROVIDER-NAME -h` to view the flags for different storage providers. 186 187 </TabItem> 188 189 <TabItem value="MinIO" label="MinIO"> 190 191 ```bash 192 kbcli backuprepo create my-repo \ 193 --provider minio \ 194 --endpoint <ip:port> \ 195 --bucket test-minio \ 196 --access-key-id <ACCESS KEY> \ 197 --secret-access-key <SECRET KEY> \ 198 --default 199 ``` 200 201 The above command creates a default backup repository `my-repo`. 202 203 * `my-repo` is the name of the created backup repository. If you do not specify a name, the system creates a random name, following the format `backuprepo-xxxxx`. 204 * `--default` means that this repository is set as the default repository. Note that there can only be one default global repository. If there exist multiple default repositories, KubeBlocks cannot decide which one to use (similar to the default StorageClass of Kubernetes), which further results in backup failure. Using `kbcli` to create BackupRepo can avoid such problems because `kbcli` checks whether there is another default repository before creating a new one. 205 * `--provider` specifies the storage type, i.e. `storageProvider`, and is required for creating a BakcupRepo. The available values are `s3`, `oss`, and `minio`. Parameters for different storage providers vary and you can run `kbcli backuprepo create --provider STORAGE-PROVIDER-NAME -h` to view the flags for different storage providers. 206 * `--endpoint` specifies the endpoint of MinIO server. If you install MinIO by above instructions, the endpoint is `http://minio.default.svc.cluster.local:9000`, and the `default` is the namespace where MinIO is installed. 207 208 </TabItem> 209 210 </Tabs> 211 212 After `kbcli backuprepo create` is executed successfully, the system creates the K8s resource whose type is `BackupRepo`. You can modify the annotation of this resource to adjust the default repository. 213 214 ```bash 215 # Canel the default repository 216 kubectl annotate backuprepo old-default-repo \ 217 --overwrite=true \ 218 dataprotection.kubeblocks.io/is-default-repo=false 219 ``` 220 221 ```bash 222 # Set a new default repository 223 kubectl annotate backuprepo backuprepo-4qms6 \ 224 --overwrite=true \ 225 dataprotection.kubeblocks.io/is-default-repo=true 226 ``` 227 228 </TabItem> 229 230 <TabItem value="kubectl" label="kubectl"> 231 232 `kubectl` is another option to create a BackupRepo, but the commands do not include parameter and default repository verification compared with kbcli, which is not convenient. 233 234 <Tabs> 235 236 <TabItem value="S3" label="S3" default> 237 238 ```bash 239 # Create a secret to save the access key for S3 240 kubectl create secret generic s3-credential-for-backuprepo \ 241 -n kb-system \ 242 --from-literal=accessKeyId=<ACCESS KEY> \ 243 --from-literal=secretAccessKey=<SECRET KEY> 244 245 # Create the BackupRepo resource 246 kubectl apply -f - <<-'EOF' 247 apiVersion: dataprotection.kubeblocks.io/v1alpha1 248 kind: BackupRepo 249 metadata: 250 name: my-repo 251 annotations: 252 dataprotection.kubeblocks.io/is-default-repo: "true" 253 spec: 254 storageProviderRef: s3 255 pvReclaimPolicy: Retain 256 volumeCapacity: 100Gi 257 config: 258 bucket: test-kb-backup 259 endpoint: "" 260 mountOptions: --memory-limit 1000 --dir-mode 0777 --file-mode 0666 261 region: cn-northwest-1 262 credential: 263 name: s3-credential-for-backuprepo 264 namespace: kb-system 265 EOF 266 ``` 267 268 </TabItem> 269 270 <TabItem value="OSS" label="OSS"> 271 272 ```bash 273 # Create a secret to save the access key for OSS 274 kubectl create secret generic oss-credential-for-backuprepo \ 275 -n kb-system \ 276 --from-literal=accessKeyId=<ACCESS KEY> \ 277 --from-literal=secretAccessKey=<SECRET KEY> 278 279 # Create the BackupRepo resource 280 kubectl apply -f - <<-'EOF' 281 apiVersion: dataprotection.kubeblocks.io/v1alpha1 282 kind: BackupRepo 283 metadata: 284 name: my-repo 285 annotations: 286 dataprotection.kubeblocks.io/is-default-repo: "true" 287 spec: 288 storageProviderRef: s3 289 pvReclaimPolicy: Retain 290 volumeCapacity: 100Gi 291 config: 292 bucket: test-kb-backup 293 mountOptions: "" 294 endpoint: "" 295 region: cn-zhangjiakou 296 credential: 297 name: oss-credential-for-backuprepo 298 namespace: kb-system 299 EOF 300 ``` 301 302 </TabItem> 303 304 <TabItem value="MinIO" label="MinIO"> 305 306 ```bash 307 # Create a secret to save the access key for MinIO 308 kubectl create secret generic minio-credential-for-backuprepo \ 309 -n kb-system \ 310 --from-literal=accessKeyId=<ACCESS KEY> \ 311 --from-literal=secretAccessKey=<SECRET KEY> 312 313 # Create the BackupRepo resource 314 kubectl apply -f - <<-'EOF' 315 apiVersion: dataprotection.kubeblocks.io/v1alpha1 316 kind: BackupRepo 317 metadata: 318 name: my-repo 319 annotations: 320 dataprotection.kubeblocks.io/is-default-repo: "true" 321 spec: 322 storageProviderRef: minio 323 pvReclaimPolicy: Retain 324 volumeCapacity: 100Gi 325 config: 326 bucket: test-kb-backup 327 mountOptions: "" 328 endpoint: <ip:port> 329 credential: 330 name: minio-credential-for-backuprepo 331 namespace: kb-system 332 EOF 333 ``` 334 335 </TabItem> 336 337 </Tabs> 338 339 </TabItem> 340 341 </Tabs> 342 343 ## (Optional) Change the backup repository for a cluster 344 345 By default, all backups are stored in the global default repository when creating a cluster. You can specify another backup repository for this cluster by editing `BackupPolicy`. 346 347 <Tabs> 348 349 <TabItem value="kbcli" label="kbcli" default> 350 351 ```bash 352 kbcli cluster edit-backup-policy mysql-cluster --set="datafile.backupRepoName=my-repo" 353 ``` 354 355 </TabItem> 356 357 <TabItem value="kubectl" label="kubectl"> 358 359 ```bash 360 kubectl edit backuppolicy mysql-cluster-mysql-backup-policy 361 ... 362 spec: 363 datafile: 364 ... 365 # Edit the backup repository name 366 backupRepoName: my-repo 367 ``` 368 369 </TabItem> 370 371 </Tabs>