github.com/1aal/kubeblocks@v0.0.0-20231107070852-e1c03e598921/docs/developer_docs/integration/how-to-add-an-add-on.md (about) 1 --- 2 title: Add an add-on 3 description: Add an add-on to KubeBlocks 4 keywords: [add-on, add an add-on] 5 sidebar_position: 2 6 sidebar_label: Add an add-on 7 --- 8 9 # Add an add-on to KubeBlocks 10 11 This tutorial explains how to integrate an add-on to KubeBlocks, and takes Oracle MySQL as an example. You can also find the [PR here](https://github.com/apecloud/learn-kubeblocks-addon). 12 13 There are altogether 3 steps to integrate an add-on: 14 15 1. Design cluster blueprint. 16 2. Prepare cluster templates. 17 3. Add an `addon.yaml` file. 18 19 ## Step 1. Design a blueprint for cluster 20 21 Before getting started, make sure to design your cluster blueprint. Think about what you want your cluster to look like. For example: 22 23 - What components it has 24 - What format each component takes 25 - stateful/stateless 26 - Standalone/Replication/RaftGroup 27 28 In this tutorial you will learn how to deploy a cluster with one Stateful component which has only one node. The design configuration of the cluster is shown in the following table. 29 30 Cluster Format: Deploying a MySQL 8.0 Standalone. 31 32 :paperclip: Table 1. Blueprint for Oracle MySQL Cluster 33 34 | Term | Settings | 35 |-------------------|--------------------------------------------------------------------------------------------------------------| 36 | CLusterDefinition | Startup Scripts: Default Configuration Files: Default Service Port: 3306 Number of Components: 1, i.e. MySQL | 37 | ClusterVersion | Image: docker.io/mysql:8.0.34 | 38 | Cluster.yaml | Specified by the user during creation | 39 40 ## Step 2. Prepare cluster templates 41 42 ### 2.1 Create a Helm chart 43 44 Opt 1.`helm create oracle-mysql` 45 46 Opt 2. Directly create `mkdir oracle-mysql` 47 48 It should contain the following information: 49 50 ```bash 51 > tree oracle-mysql 52 . 53 ├── Chart.yaml # A YAML file containing information about the chart 54 ├── templates # A directory of templates that, when combined with values, will generate valid Kubernetes manifest files. 55 │ ├── NOTES.txt # OPTIONAL: A plain text file containing short usage notes 56 │ ├── _helpers.tpl # A place to put template helpers that you can re-use throughout the chart 57 │ ├── clusterdefinition.yaml 58 │ └── clusterversion.yaml 59 └── values.yaml # The default configuration values for this chart 60 61 2 directories, 6 files 62 ``` 63 64 There are two YAML files under `templates`, `clusterDefinition.yaml` and `clusterVersion.yaml`, which is about the component topology and version. 65 66 - `clusterDefinition.yaml` 67 68 This YAML file is very long, and each field is explained as follows. 69 70 - `ConnectionCredential` 71 72 ```yaml 73 connectionCredential: 74 username: root 75 password: "$(RANDOM_PASSWD)" 76 endpoint: "$(SVC_FQDN):$(SVC_PORT_mysql)" 77 host: "$(SVC_FQDN)" 78 port: "$(SVC_PORT_mysql)" 79 ``` 80 81 It generates a secret, whose naming convention is `{clusterName}-conn-credential`. 82 83 The field contains general information such as username, password, endpoint and port, and will be used when other services access the cluster (The secret is created before other resources, which can be used elsewhere). 84 85 `$(RANDOM_PASSWD)` will be replaced with a random password when created. 86 87 `$(SVC_PORT_mysql)` specifies the port number to be exposed by selecting the port name. Here the port name is mysql. 88 89 For more information, please refer to KubeBlocks Environment Variables (this doc is under developing and will be published soon). 90 91 - `ComponentDefs` 92 93 ```yaml 94 componentDefs: 95 - name: mysql-compdef 96 characterType: mysql 97 workloadType: Stateful 98 service: 99 ports: 100 - name: mysql 101 port: 3306 102 targetPort: mysql 103 podSpec: 104 containers: 105 ... 106 ``` 107 108 `componentDefs` (Component Definitions) defines the basic information required for each component, including startup scripts, configurations and ports. 109 110 Since there is only one MySQL component, you can just name it `mysql-compdef`, which stands for a component definition for MySQL. 111 112 - `name` [Required] 113 114 It is the name of the component. As there are no specific criteria, you can just choose a distinguishable and expressive one. 115 116 Remember the equation in the previous article? 117 118 $$Cluster = ClusterDefinition.yaml \Join ClusterVersion.yaml \Join ...$$ 119 120 `name` here is the join key. 121 122 Remember the `name`; it will be useful. 123 124 - `characterType` [Optional] 125 126 `characterType` is a string type used to identify the engine. For example, `mysql`, `postgresql` and `redis` are several predefined engine types used for database connection. When operating a database, it helps to quickly recognize the engine type and find the matching operation command. 127 128 It can be an arbitrary string, or a unique name as you define. The fact is that people seldom have engine-related operations in the early stage, so you can just leave it blank. 129 130 - `workloadType` [Required] 131 132 It is the type of workload. Kubernetes is equipped with several basic workload types, such as Deployment and StatefulSet. 133 134 On top of that, KubeBlocks makes abstractions and provides more choices, such as: 135 136 - Stateless, meaning it has no stateful services 137 - Stateful, meaning it has stateful services 138 - Consensus, meaning it has stateful services with self-election capabilities and roles. 139 140 A more in-depth introduction to workloads will be presented later (including design, implementation, usage, etc.). 141 142 For a MySQL Standalone, `Stateful` will do. 143 144 - `service` [Optional] 145 146 ```yaml 147 service: 148 ports: 149 - name: mysql #The port name is mysql, so connectionCredential will look for it to find the corresponding port 150 port: 3306 151 targetPort: mysql 152 ``` 153 154 It defines how to create a service for a component and which ports to expose. 155 156 Remember that in the `ConnectionCredential` section, it is mentioned that a cluster will expose ports and endpoints? 157 158 You can invoke `$(SVC_PORT_mysql)$` to select a port, where `mysql` is the `service.ports[0].name` here. 159 160 :::note 161 162 If the `connectionCredential` is filled with a port name, make sure the port name appears here. 163 164 ::: 165 166 - `podSpec` 167 168 The definition of podSpec is the same as that of the Kubernetes. 169 170 ```yaml 171 podSpec: 172 containers: 173 - name: mysql-container 174 imagePullPolicy: IfNotPresent 175 volumeMounts: 176 - mountPath: /var/lib/mysql 177 name: data 178 ports: 179 - containerPort: 3306 180 name: mysql 181 env: 182 - name: MYSQL_ROOT_HOST 183 value: {{ .Values.auth.rootHost | default "%" | quote }} 184 - name: MYSQL_ROOT_USER 185 valueFrom: 186 secretKeyRef: 187 name: $(CONN_CREDENTIAL_SECRET_NAME) 188 key: username 189 - name: MYSQL_ROOT_PASSWORD 190 valueFrom: 191 secretKeyRef: 192 name: $(CONN_CREDENTIAL_SECRET_NAME) 193 key: password 194 ``` 195 196 As is shown above, a pod is defined with a single container named `mysql-container`, along with other essential information, such as environment variables and ports. 197 198 Yet here is something worth noting: `$(CONN_CREDENTIAL_SECRET_NAME)`. 199 200 The username and password are obtained as pod environment variables from the secret in `$(CONN_CREDENTIAL_SECRET_NAME)`. 201 202 This is a placeholder for ConnectionCredential Secret mentioned earlier. 203 204 - ClusterVersion 205 206 All version-related information is configured in `ClusterVersion.yaml`. 207 208 Now you can add the required image information for each container needed for each component. 209 210 ```yaml 211 clusterDefinitionRef: oracle-mysql 212 componentVersions: 213 - componentDefRef: mysql-compdef 214 versionsContext: 215 containers: 216 - name: mysql-container 217 image: {{ .Values.image.registry | default "docker.io" }}/{{ .Values.image.repository }}:{{ .Values.image.tag }} 218 imagePullPolicy: {{ default .Values.image.pullPolicy "IfNotPresent" }} 219 ``` 220 221 Remember the ComponentDef Name used in ClusterDefinition? Yes, `mysql-compdef`, fill in the image information here. 222 223 :::note 224 225 Now you've finished with ClusterDefinition and ClusterVersion, try to do a quick test by installing them locally. 226 227 ::: 228 229 ### 2.2 Install Helm chart 230 231 Install Helm. 232 233 ```bash 234 helm install oracle-mysql ./oracle-mysql 235 ``` 236 237 After successful installation, you can see the following information: 238 239 ```yaml 240 NAME: oracle-mysql 241 LAST DEPLOYED: Wed Aug 2 20:50:33 2023 242 NAMESPACE: default 243 STATUS: deployed 244 REVISION: 1 245 TEST SUITE: None 246 ``` 247 248 ### 2.3 Create a cluster 249 250 Create a MySQL cluster with `kbcli cluster create`. 251 252 ```bash 253 kbcli cluster create mycluster --cluster-definition oracle-mysql 254 > 255 Info: --cluster-version is not specified, ClusterVersion oracle-mysql-8.0.34 is applied by default 256 Cluster mycluster created 257 ``` 258 259 You can specify the name of ClusterDefinition by using `--cluster-definition`. 260 261 :::note 262 263 If only one ClusterVersion object is associated with this ClusterDefinition, kbcli will use it when creating the cluster. 264 265 However, if there are multiple ClusterVersion objects associated, you will need to explicitly specify which one to use. 266 267 ::: 268 269 After the creating, you can: 270 271 **A. Check cluster status** 272 273 ```bash 274 kbcli cluster list mycluster 275 > 276 NAME NAMESPACE CLUSTER-DEFINITION VERSION TERMINATION-POLICY STATUS CREATED-TIME 277 mycluster default oracle-mysql oracle-mysql-8.0.34 Delete Running Aug 02,2023 20:52 UTC+0800 278 ``` 279 280 **B. Connect to the cluster** 281 282 ```bash 283 kbcli cluster connect mycluster 284 > 285 Connect to instance mycluster-mysql-compdef-0 286 mysql: [Warning] Using a password on the command line interface can be insecure. 287 Welcome to the MySQL monitor. Commands end with ; or \g. 288 Your MySQL connection id is 8 289 Server version: 8.0.34 MySQL Community Server - GPL 290 291 Copyright (c) 2000, 2023, Oracle and/or its affiliates. 292 293 Oracle is a registered trademark of Oracle Corporation and/or its 294 affiliates. Other names may be trademarks of their respective 295 owners. 296 297 Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. 298 299 mysql> 300 ``` 301 302 **C. Scale up a cluster** 303 304 ```bash 305 kbcli cluster vscale mycluster --components mysql-compdef --cpu='2' --memory=2Gi 306 ``` 307 308 **D. Stop a cluster** 309 310 Stopping the cluster releases all computing resources. 311 312 ```bash 313 kbcli cluster stop mycluster 314 ``` 315 316 ## Step 3. Add an addon.yaml file 317 318 This is the last step to integrate an add-on to KubeBlocks. After creating this addon.yaml file, this add-on is in the KubeBlocks add-on family. Please refer to `tutorial-1-create-an-addon/oracle-mysql-addon.yaml`. 319 320 ```bash 321 apiVersion: extensions.kubeblocks.io/v1alpha1 322 kind: Addon 323 metadata: 324 name: tutorial-mysql 325 spec: 326 description: 'MySQL is a widely used, open-source....' 327 type: Helm 328 helm: 329 chartsImage: registry-of-your-helm-chart 330 installable: 331 autoInstall: false 332 333 defaultInstallValues: 334 - enabled: true 335 ``` 336 337 And then configure your Helm chart remote repository address with `chartsImage`. 338 339 ## Step 4. (Optional) Publish to Kubeblocks community 340 341 You can contribute the Helm chart and `addon.yaml` to the [KubeBlocks community](https://github.com/apecloud/kubeblocks). 342 343 - Helm chart is in the `kubeblocks/deploy` directory. 344 - The `addon.yaml` file is in the `kubeblocks/deploy/helm/templates/addons directory`. 345 346 ## Appendix 347 348 ### A.1 How to configure multiple versions for the same engine? 349 350 To support multiple versions is one of the common problems in the daily production environment. And the problem can be solved by **associating multiple ClusterVersions with the same ClusterDefinition**. 351 352 Take MySQL as an example. 353 354 1. Modify `ClusterVersion.yaml` file to support multiple versions. 355 356 ```yaml 357 apiVersion: apps.kubeblocks.io/v1alpha1 358 kind: ClusterVersion 359 metadata: 360 name: oracle-mysql-8.0.32 361 spec: 362 clusterDefinitionRef: oracle-mysql ## Associate the same clusterdefinition: oracle-mysql 363 componentVersions: 364 - componentDefRef: mysql-compdef 365 versionsContext: 366 containers: 367 - name: mysql-container 368 image: <image-of-mysql-8.0.32> ## The mirror address is 8.0.32 369 --- 370 apiVersion: apps.kubeblocks.io/v1alpha1 371 kind: ClusterVersion 372 metadata: 373 name: oracle-mysql-8.0.18 374 spec: 375 clusterDefinitionRef: oracle-mysql ## Associate the same clusterdefinition: oracle-mysql 376 componentVersions: 377 - componentDefRef: mysql-compdef 378 versionsContext: 379 containers: 380 - name: mysql-container 381 image: <image-of-mysql-8.0.18> ## The mirror address is 8.0.18 382 ``` 383 384 2. Specify the version information when creating a cluster. 385 386 - Create a cluster with version 8.0.32 387 388 ```bash 389 kbcli cluster create mycluster --cluster-definition oracle-mysql --cluster-version oracle-mysql-8.0.32 390 ``` 391 392 - Create a cluster with version 8.0.18 393 394 ```bash 395 kbcli cluster create mycluster --cluster-definition oracle-mysql --cluster-version oracle-mysql-8.0.18 396 ``` 397 398 It allows you to quickly configure multiple versions for your engine. 399 400 ### A.2 What if kbcli cannot meet your needs? 401 402 While kbcli provides a convenient and generic way to create clusters, it may not meet the specific needs of every engine, especially when a cluster contains multiple components and needs to be used according to different requirements. 403 404 In that case, try to use a Helm chart to render the cluster, or create it through a cluster.yaml file. 405 406 ```yaml 407 apiVersion: apps.kubeblocks.io/v1alpha1 408 kind: Cluster 409 metadata: 410 name: mycluster 411 namespace: default 412 spec: 413 clusterDefinitionRef: oracle-mysql # Specify ClusterDefinition 414 clusterVersionRef: oracle-mysql-8.0.32 # Specify ClusterVersion 415 componentSpecs: # List required components 416 - componentDefRef: mysql-compdef # The type of the first component: mysql-compdef 417 name: mysql-comp # The name of the first component: mysql-comp 418 replicas: 1 419 resources: # Specify CPU and memory size 420 limits: 421 cpu: "1" 422 memory: 1Gi 423 requests: 424 cpu: "1" 425 memory: 1Gi 426 volumeClaimTemplates: # Set the PVC information, where the name must correspond to that of the Component Def. 427 - name: data 428 spec: 429 accessModes: 430 - ReadWriteOnce 431 resources: 432 requests: 433 storage: 20Gi 434 terminationPolicy: Delete 435 ```