github.com/verrazzano/verrazzano@v1.7.0/tools/psr/README.md (about) 1 # PSR Testing Tool 2 The PSR testing tool is used to test Verrazzano performance, scalability, and 3 reliability. The tool works by doing some work on the Verrazzano cluster, then collecting 4 and analyzing results. The type of work that is done depends on the goal of the PSR test. 5 6 ## Concepts 7 At a high level, you run a test scenario using the `psrctl` command line interface to do some aspect of 8 PSR testing, whether it involves stressing a component, doing long term testing for 9 resource leaks, or other testing. 10 11 ### Worker 12 A worker is the code that performs a specific task, such as putting logs into OpenSearch. A worker 13 does a single task and runs continuously in a loop doing the same task repeatedly. Workers are 14 deployed in a pod and run in the target cluster that you are testing. You can have multiple worker 15 threads (doing the same work), along with multiple replicas. The type of work done would determine 16 how many workers of the same type should be running. The idea is that different types of worker are 17 combined into scenarios, which is described later. 18 19 ### Backend 20 Workers run in backend pods that are deployed using Helm charts. The backend consists of a single image that has 21 all the worker code for all workers. When a pod is started, the worker config is passed in as a set of Env vars. 22 The main.go code in the pod gets the worker type, creates an instance of the worker, then invokes it to run. 23 The pod only runs a single worker, which executes until the pod terminates. By default, the worker runs forever. 24 25 ### Use Case 26 The term `use case` describes what work is being done by a worker. All use cases are deployed using 27 the [worker Helm chart](./manifests/charts/worker). Each worker type has a Helm values override YAML file 28 that provides worker specific configuration, such as [usecases/opensearch/getlogs.yaml](./manifests/usecases/opensearch/getlogs.yaml). 29 To run a use case, just do a Helm install. For example, to deploy a use case to generate logs using 10 replicas, 30 you would run the following command: 31 ``` 32 helm install psr-writelogs manifests/charts/worker -f manifests/usecases/opensearch/writelogs.yaml --set replicas=10 33 ``` 34 35 ### Scenario 36 A scenario is a collection of use cases with a curated configuration, that are run concurrently. 37 An example is a scenario where one use case generates logs and another scales OpenSearch out and in, repeatedly. 38 The PSR command line interface, `psrctl`, is used to run scenarios. See [psrctl README](./psrctl/README.md) 39 40 ## Usage 41 Use the Makefile to build the backend image or execute other targets. If you 42 just want to try the example use case on a local Kind cluster, then run the following which 43 builds the code, builds the docker image, loads the docker image into the Kind cluster, and deploys the example use case. 44 ``` 45 make run-example-k8s 46 ``` 47 After you run the make command, run `helm list` to see the `psr` release. Then use kubectl to 48 get the backend pod logs in the default namespace to see that they are generating log messages. 49 50 **NOTE:** Any Kubernetes platform can be used, such as OKE. 51 52 The other Make targets include: 53 * go-build - build the code 54 * go-build-cli - build the psrctl CLI 55 * docker-clean - remove artifacts from a previous build 56 * docker-build - calls go-build, then build the image 57 * docker-push - calls docker-build, then push the image to ghcr.io. 58 * kind-load-image - calls docker-build, then load the image to a local Kind cluster 59 * run-example-oam - calls kind-load-image, then deploy the example worker as an OAM app 60 * run-example-k8s - calls kind-load-image, then deploy the example worker as a Kubernetes deployment 61 * install-cli - build and install the psrctl CLI to your go path 62 63 ### Running scenarios 64 Refer to [psrctl README](./psrctl/README.md) for information on running scenarios. 65 66 ### Example worker installs using Helm 67 68 Install the example worker as an OAM application with 10 replicas: 69 ``` 70 helm install psr-example manifests/charts/worker --set imageName=ghcr.io/verrazzano/psr-backend:local-582bfcfcf --replicas=10 71 ``` 72 73 Install the example worker as an OAM application using the default image in the helm chart, providing an imagePullSecret 74 ``` 75 helm install psr-example manifests/charts/worker --set imagePullSecrets[0].name=verrazzano-container-registry 76 ``` 77 78 Install the WriteLogs worker as a Kubernetes deployment using the default image in the helm chart. Note the appType must be supplied: 79 ``` 80 helm install psr-writelogs manifests/charts/worker --set appType=k8s -f manifests/usecases/opensearch/writelogs.yaml 81 ``` 82 83 Install the WriteLogs worker as an OAM application with 5 worker threads and the default 1 replica. 84 ``` 85 helm install psr-writelogs manifests/charts/worker --set global.envVars.PSR_WORKER_THREAD_COUNT=5 -f manifests/usecases/opensearch/writelogs.yaml 86 ``` 87 88 ## Workers 89 All of the workers are deployed using the same worker Helm chart. Worker configs are stored in global.envVars values. 90 There are common values applicable to each worker as follows: 91 92 ``` 93 global: 94 envVars: 95 PSR_WORKER_TYPE - type of worker 96 default: ops-postlogs 97 98 PSR_LOOP_SLEEP - duration to sleep between work iterations 99 default: 1s 100 101 PSR_NUM_LOOPS - number of iterations per worker thread 102 default: -1 (run forever) 103 104 PSR_WORKER_THREAD_COUNT - threads per worker 105 default: 1 106 ``` 107 108 The following section describes each worker. 109 110 ### Example 111 #### Description 112 The example worker periodically logs messages, it doesn't provide metrics. 113 114 #### Configuration 115 no configuration overrides 116 117 #### Run 118 ``` 119 helm install psr-example manifests/charts/worker 120 ``` 121 122 ### WriteLogs 123 #### Description 124 The WriteLogs worker periodically logs messages. The goal is to put a load on OpenSearch since fluentd collects the container logs and 125 sends them to OpenSearch. 126 127 #### Configuration 128 no configuration overrides 129 130 #### Run 131 ``` 132 helm install psr-writelogs manifests/charts/worker -f manifests/usecases/opensearch/writelogs.yaml 133 ``` 134 135 ### GetLogs 136 #### Description 137 The GetLogs worker periodically gets messages from OpenSearch, in-cluster. This worker must run in the mesh. 138 139 #### Configuration 140 no configuration overrides 141 142 #### Run 143 ``` 144 helm install psr-getlogs manifests/charts/worker -f manifests/usecases/opensearch/getlogs.yaml -n istioEnabledNamespace 145 ``` 146 147 ### PostLogs 148 #### Description 149 The PostLogs worker makes batch http post requests to OpenSearch, in-cluster. This worker must run in the mesh. 150 151 #### Configuration 152 ``` 153 global: 154 envVars: 155 LOG_ENTRIES - number of log messages per post request 156 default: 1 157 158 LOG_LENGTH - number of characters per log message 159 default: 1 160 ``` 161 162 #### Run 163 ``` 164 helm install psr-postlogs manifests/charts/worker -f manifests/usecases/opensearch/postlogs.yaml -n istioEnabledNamespace 165 ``` 166 167 ### Scale 168 #### Description 169 The Scale worker continuously scales one tier of OpenSearch out and in. This worker must run in the mesh. 170 171 #### Configuration 172 ``` 173 global: 174 envVars: 175 OPEN_SEARCH_TIER - OpenSearch tier 176 default: master 177 178 MIN_REPLICA_COUNT - number of replicas to scale in to 179 default: 3 180 181 MAX_REPLICA_COUNT - number of replicas to scale out to 182 default: 5 183 ``` 184 185 #### Run 186 ``` 187 helm install psr-scale manifests/charts/worker -f manifests/usecases/opensearch/scale.yaml -n istioEnabledNamespace 188 ``` 189 190 ### Restart 191 #### Description 192 The restart worker continuously restarts pods one tier of OpenSearch. This worker must run in the mesh. 193 194 #### Configuration 195 ``` 196 global: 197 envVars: 198 OPEN_SEARCH_TIER - OpenSearch tier 199 default: none 200 ``` 201 202 #### Run 203 ``` 204 helm install psr-scale manifests/charts/worker -f manifests/usecases/opensearch/restart.yaml -n istioEnabledNamespace 205 ``` 206 207 ### HTTPGet 208 #### Description 209 The HTTPGet worker makes http requests to the endpoint for a specified service. 210 211 #### Configuration 212 ``` 213 global: 214 envVars: 215 SERVICE_NAME - service name 216 default: "" 217 218 SERVICE_NAMESPACE - service namespace 219 default: "" 220 221 SERVICE_PORT - service port 222 default: "" 223 224 PATH - service path 225 default: "" 226 ``` 227 228 #### Run 229 ``` 230 helm install psr-httpget manifests/charts/worker -f manifests/usecases/http/get.yaml 231 ``` 232 233 ## Developing scenarios and use cases 234 Refer to the [PSR developer guide](./DEVELOPER.md) to learn how to develop new use cases and scenarios.