github.com/theishshah/operator-sdk@v0.6.0/doc/ansible/user-guide.md (about) 1 # User Guide 2 3 This guide walks through an example of building a simple memcached-operator 4 powered by Ansible using tools and libraries provided by the Operator SDK. 5 6 ## Prerequisites 7 8 - [git][git_tool] 9 - [docker][docker_tool] version 17.03+. 10 - [kubectl][kubectl_tool] version v1.9.0+. 11 - [ansible][ansible_tool] version v2.6.0+ 12 - [ansible-runner][ansible_runner_tool] version v1.1.0+ 13 - [ansible-runner-http][ansible_runner_http_plugin] version v1.0.0+ 14 - [dep][dep_tool] version v0.5.0+. (Optional if you aren't installing from source) 15 - [go][go_tool] version v1.10+. (Optional if you aren't installing from source) 16 - Access to a Kubernetes v.1.9.0+ cluster. 17 18 **Note**: This guide uses [minikube][minikube_tool] version v0.25.0+ as the 19 local Kubernetes cluster and quay.io for the public registry. 20 21 ## Install the Operator SDK CLI 22 23 The Operator SDK has a CLI tool that helps the developer to create, build, and 24 deploy a new operator project. 25 26 Checkout the desired release tag and install the SDK CLI tool: 27 28 ```sh 29 $ mkdir -p $GOPATH/src/github.com/operator-framework 30 $ cd $GOPATH/src/github.com/operator-framework 31 $ git clone https://github.com/operator-framework/operator-sdk 32 $ cd operator-sdk 33 $ git checkout master 34 $ make dep 35 $ make install 36 ``` 37 38 This installs the CLI binary `operator-sdk` at `$GOPATH/bin`. 39 40 ## Create a new project 41 42 Use the CLI to create a new Ansible-based memcached-operator project: 43 44 ```sh 45 $ operator-sdk new memcached-operator --api-version=cache.example.com/v1alpha1 --kind=Memcached --type=ansible 46 $ cd memcached-operator 47 ``` 48 49 This creates the memcached-operator project specifically for watching the 50 Memcached resource with APIVersion `cache.example.com/v1apha1` and Kind 51 `Memcached`. 52 53 To learn more about the project directory structure, see [project 54 layout][layout_doc] doc. 55 56 #### Operator scope 57 58 A namespace-scoped operator (the default) watches and manages resources in a single namespace, whereas a cluster-scoped operator watches and manages resources cluster-wide. Namespace-scoped operators are preferred because of their flexibility. They enable decoupled upgrades, namespace isolation for failures and monitoring, and differing API definitions. However, there are use cases where a cluster-scoped operator may make sense. For example, the [cert-manager](https://github.com/jetstack/cert-manager) operator is often deployed with cluster-scoped permissions and watches so that it can manage issuing certificates for an entire cluster. 59 60 If you'd like to create your memcached-operator project to be cluster-scoped use the following `operator-sdk new` command instead: 61 ``` 62 $ operator-sdk new memcached-operator --cluster-scoped --api-version=cache.example.com/v1alpha1 --kind=Memcached --type=ansible 63 ``` 64 65 Using `--cluster-scoped` will scaffold the new operator with the following modifications: 66 * `deploy/operator.yaml` - Set `WATCH_NAMESPACE=""` instead of setting it to the pod's namespace 67 * `deploy/role.yaml` - Use `ClusterRole` instead of `Role` 68 * `deploy/role_binding.yaml`: 69 * Use `ClusterRoleBinding` instead of `RoleBinding` 70 * Use `ClusterRole` instead of `Role` for roleRef 71 * Set the subject namespace to `REPLACE_NAMESPACE`. This must be changed to the namespace in which the operator is deployed. 72 73 ### Watches file 74 75 The Watches file contains a list of mappings from custom resources, identified 76 by it's Group, Version, and Kind, to an Ansible Role or Playbook. The Operator 77 expects this mapping file in a predefined location: `/opt/ansible/watches.yaml` 78 79 * **group**: The group of the Custom Resource that you will be watching. 80 * **version**: The version of the Custom Resource that you will be watching. 81 * **kind**: The kind of the Custom Resource that you will be watching. 82 * **role** (default): This is the path to the role that you have added to the 83 container. For example if your roles directory is at `/opt/ansible/roles/` 84 and your role is named `busybox`, this value will be 85 `/opt/ansible/roles/busybox`. This field is mutually exclusive with the 86 "playbook" field. 87 * **playbook**: This is the path to the playbook that you have added to the 88 container. This playbook is expected to be simply a way to call roles. This 89 field is mutually exclusive with the "role" field. 90 * **reconcilePeriod** (optional): The reconciliation interval, how often the 91 role/playbook is run, for a given CR. 92 * **manageStatus** (optional): When true (default), the operator will manage 93 the status of the CR generically. Set to false, the status of the CR is 94 managed elsewhere, by the specified role/playbook or in a separate controller. 95 96 An example Watches file: 97 98 ```yaml 99 --- 100 # Simple example mapping Foo to the Foo role 101 - version: v1alpha1 102 group: foo.example.com 103 kind: Foo 104 role: /opt/ansible/roles/Foo 105 106 # Simple example mapping Bar to a playbook 107 - version: v1alpha1 108 group: bar.example.com 109 kind: Bar 110 playbook: /opt/ansible/playbook.yml 111 112 # More complex example for our Baz kind 113 # Here we will disable requeuing and be managing the CR status in the playbook 114 - version: v1alpha1 115 group: baz.example.com 116 kind: Baz 117 playbook: /opt/ansible/baz.yml 118 reconcilePeriod: 0 119 manageStatus: false 120 ``` 121 122 ## Customize the operator logic 123 124 For this example the memcached-operator will execute the following 125 reconciliation logic for each `Memcached` Custom Resource (CR): 126 - Create a memcached Deployment if it doesn't exist 127 - Ensure that the Deployment size is the same as specified by the `Memcached` 128 CR 129 130 ### Watch the Memcached CR 131 132 By default, the memcached-operator watches `Memcached` resource events as shown 133 in `watches.yaml` and executes Ansible Role `Memcached`: 134 135 ```yaml 136 --- 137 - version: v1alpha1 138 group: cache.example.com 139 kind: Memcached 140 ``` 141 142 #### Options 143 **Role** 144 Specifying a `role` option in `watches.yaml` will configure the operator to use 145 this specified path when launching `ansible-runner` with an Ansible Role. By 146 default, the `new` command will fill in an absolute path to where your role 147 should go. 148 ```yaml 149 --- 150 - version: v1alpha1 151 group: cache.example.com 152 kind: Memcached 153 role: /opt/ansible/roles/memcached 154 ``` 155 156 **Playbook** 157 Specifying a `playbook` option in `watches.yaml` will configure the operator to 158 use this specified path when launching `ansible-runner` with an Ansible 159 Playbook 160 ```yaml 161 --- 162 - version: v1alpha1 163 group: cache.example.com 164 kind: Memcached 165 playbook: /opt/ansible/playbook.yaml 166 ``` 167 168 ## Building the Memcached Ansible Role 169 170 The first thing to do is to modify the generated Ansible role under 171 `roles/memcached`. This Ansible Role controls the logic that is executed when a 172 resource is modified. 173 174 ### Define the Memcached spec 175 176 Defining the spec for an Ansible Operator can be done entirely in Ansible. The 177 Ansible Operator will simply pass all key value pairs listed in the Custom 178 Resource spec field along to Ansible as 179 [variables](https://docs.ansible.com/ansible/2.5/user_guide/playbooks_variables.html#passing-variables-on-the-command-line). 180 The names of all variables in the spec field are converted to snake_case 181 by the operator before running ansible. For example, `serviceAccount` in 182 the spec becomes `service_account` in ansible. 183 It is recommended that you perform some type validation in Ansible on the 184 variables to ensure that your application is receiving expected input. 185 186 First, set a default in case the user doesn't set the `spec` field by modifying 187 `roles/memcached/defaults/main.yml`: 188 ```yaml 189 size: 1 190 ``` 191 192 ### Defining the Memcached deployment 193 194 Now that we have the spec defined, we can define what Ansible is actually 195 executed on resource changes. Since this is an Ansible Role, the default 196 behavior will be to execute the tasks in `roles/memcached/tasks/main.yml`. We 197 want Ansible to create a deployment if it does not exist which runs the 198 `memcached:1.4.36-alpine` image. Ansible 2.5+ supports the [k8s Ansible 199 Module](https://docs.ansible.com/ansible/2.6/modules/k8s_module.html) which we 200 will leverage to control the deployment definition. 201 202 Modify `roles/memcached/tasks/main.yml` to look like the following: 203 ```yaml 204 --- 205 - name: start memcached 206 k8s: 207 definition: 208 kind: Deployment 209 apiVersion: apps/v1 210 metadata: 211 name: '{{ meta.name }}-memcached' 212 namespace: '{{ meta.namespace }}' 213 spec: 214 replicas: "{{size}}" 215 selector: 216 matchLabels: 217 app: memcached 218 template: 219 metadata: 220 labels: 221 app: memcached 222 spec: 223 containers: 224 - name: memcached 225 command: 226 - memcached 227 - -m=64 228 - -o 229 - modern 230 - -v 231 image: "docker.io/memcached:1.4.36-alpine" 232 ports: 233 - containerPort: 11211 234 235 ``` 236 237 It is important to note that we used the `size` variable to control how many 238 replicas of the Memcached deployment we want. We set the default to `1`, but 239 any user can create a Custom Resource that overwrites the default. 240 241 ### Build and run the operator 242 243 Before running the operator, Kubernetes needs to know about the new custom 244 resource definition the operator will be watching. 245 246 Deploy the CRD: 247 248 ```sh 249 $ kubectl create -f deploy/crds/cache_v1alpha1_memcached_crd.yaml 250 ``` 251 252 Once this is done, there are two ways to run the operator: 253 254 - As a pod inside a Kubernetes cluster 255 - As a go program outside the cluster using `operator-sdk` 256 257 #### 1. Run as a pod inside a Kubernetes cluster 258 259 Running as a pod inside a Kubernetes cluster is preferred for production use. 260 261 Build the memcached-operator image and push it to a registry: 262 ``` 263 $ operator-sdk build quay.io/example/memcached-operator:v0.0.1 264 $ docker push quay.io/example/memcached-operator:v0.0.1 265 ``` 266 267 Kubernetes deployment manifests are generated in `deploy/operator.yaml`. The 268 deployment image in this file needs to be modified from the placeholder 269 `REPLACE_IMAGE` to the previous built image. To do this run: 270 ``` 271 $ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml 272 ``` 273 274 If you created your operator using `--cluster-scoped=true`, update the service account namespace in the generated `ClusterRoleBinding` to match where you are deploying your operator. 275 ``` 276 $ export OPERATOR_NAMESPACE=$(kubectl config view --minify -o jsonpath='{.contexts[0].context.namespace}') 277 $ sed -i "s|REPLACE_NAMESPACE|$OPERATOR_NAMESPACE|g" deploy/role_binding.yaml 278 ``` 279 280 **Note** 281 If you are performing these steps on OSX, use the following commands instead: 282 ``` 283 $ sed -i "" 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml 284 $ sed -i "" "s|REPLACE_NAMESPACE|$OPERATOR_NAMESPACE|g" deploy/role_binding.yaml 285 ``` 286 287 Deploy the memcached-operator: 288 289 ```sh 290 $ kubectl create -f deploy/service_account.yaml 291 $ kubectl create -f deploy/role.yaml 292 $ kubectl create -f deploy/role_binding.yaml 293 $ kubectl create -f deploy/operator.yaml 294 ``` 295 296 Verify that the memcached-operator is up and running: 297 298 ```sh 299 $ kubectl get deployment 300 NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE 301 memcached-operator 1 1 1 1 1m 302 ``` 303 304 #### 2. Run outside the cluster 305 306 This method is preferred during the development cycle to speed up deployment and testing. 307 308 **Note**: Ensure that [Ansible Runner][ansible_runner_tool] and [Ansible Runner 309 HTTP Plugin][ansible_runner_http_plugin] is installed or else you will see 310 unexpected errors from Ansible Runner when a Custom Resource is created. 311 312 It is also important that the `role` path referenced in `watches.yaml` exists 313 on your machine. Since we are normally used to using a container where the Role 314 is put on disk for us, we need to manually copy our role to the configured 315 Ansible Roles path (e.g `/etc/ansible/roles`. 316 317 Run the operator locally with the default Kubernetes config file present at 318 `$HOME/.kube/config`: 319 320 ```sh 321 $ operator-sdk up local 322 INFO[0000] Go Version: go1.10 323 INFO[0000] Go OS/Arch: darwin/amd64 324 INFO[0000] operator-sdk Version: 0.0.5+git 325 ``` 326 327 Run the operator locally with a provided Kubernetes config file: 328 329 ```sh 330 $ operator-sdk up local --kubeconfig=config 331 INFO[0000] Go Version: go1.10 332 INFO[0000] Go OS/Arch: darwin/amd64 333 INFO[0000] operator-sdk Version: 0.0.5+git 334 ``` 335 336 ### Create a Memcached CR 337 338 Modify `deploy/crds/cache_v1alpha1_memcached_cr.yaml` as shown and create a `Memcached` custom resource: 339 340 ```sh 341 $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml 342 apiVersion: "cache.example.com/v1alpha1" 343 kind: "Memcached" 344 metadata: 345 name: "example-memcached" 346 spec: 347 size: 3 348 349 $ kubectl apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml 350 ``` 351 352 Ensure that the memcached-operator creates the deployment for the CR: 353 354 ```sh 355 $ kubectl get deployment 356 NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE 357 memcached-operator 1 1 1 1 2m 358 example-memcached 3 3 3 3 1m 359 ``` 360 361 Check the pods to confirm 3 replicas were created: 362 363 ```sh 364 $ kubectl get pods 365 NAME READY STATUS RESTARTS AGE 366 example-memcached-6fd7c98d8-7dqdr 1/1 Running 0 1m 367 example-memcached-6fd7c98d8-g5k7v 1/1 Running 0 1m 368 example-memcached-6fd7c98d8-m7vn7 1/1 Running 0 1m 369 memcached-operator-7cc7cfdf86-vvjqk 2/2 Running 0 2m 370 ``` 371 372 ### View the Ansible logs 373 374 The `memcached-operator` deployment creates a Pod with two containers, `operator` and `ansible`. 375 The `ansible` container exists only to expose the standard Ansible stdout logs that most Ansible 376 users will be familiar with. In order to see the logs from a particular container, you can run 377 378 ```sh 379 kubectl logs deployment/memcached-operator -c ansible 380 kubectl logs deployment/memcached-operator -c operator 381 ``` 382 383 The `ansible` logs contain all of the information about the Ansible run and will make it much easier to debug issues within your Ansible tasks, 384 whereas the `operator` logs will contain much more detailed information about the Ansible Operator's internals and interface with Kubernetes. 385 386 ### Update the size 387 388 Change the `spec.size` field in the memcached CR from 3 to 4 and apply the 389 change: 390 391 ```sh 392 $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml 393 apiVersion: "cache.example.com/v1alpha1" 394 kind: "Memcached" 395 metadata: 396 name: "example-memcached" 397 spec: 398 size: 4 399 400 $ kubectl apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml 401 ``` 402 403 Confirm that the operator changes the deployment size: 404 405 ```sh 406 $ kubectl get deployment 407 NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE 408 example-memcached 4 4 4 4 5m 409 ``` 410 411 ### Cleanup 412 413 Clean up the resources: 414 415 ```sh 416 $ kubectl delete -f deploy/crds/cache_v1alpha1_memcached_cr.yaml 417 $ kubectl delete -f deploy/operator.yaml 418 $ kubectl delete -f deploy/role_binding.yaml 419 $ kubectl delete -f deploy/role.yaml 420 $ kubectl delete -f deploy/service_account.yaml 421 $ kubectl delete -f deploy/crds/cache_v1alpha1_memcached_cr.yaml 422 ``` 423 424 [layout_doc]:./project_layout.md 425 [dep_tool]:https://golang.github.io/dep/docs/installation.html 426 [git_tool]:https://git-scm.com/downloads 427 [go_tool]:https://golang.org/dl/ 428 [docker_tool]:https://docs.docker.com/install/ 429 [kubectl_tool]:https://kubernetes.io/docs/tasks/tools/install-kubectl/ 430 [minikube_tool]:https://github.com/kubernetes/minikube#installation 431 [ansible_tool]:https://docs.ansible.com/ansible/latest/index.html 432 [ansible_runner_tool]:https://ansible-runner.readthedocs.io/en/latest/install.html 433 [ansible_runner_http_plugin]:https://github.com/ansible/ansible-runner-http