github.com/kubeshop/testkube@v1.17.23/contrib/executor/jmeterd/README.md (about)

     1  ![Testkube Logo](https://raw.githubusercontent.com/kubeshop/testkube/main/assets/testkube-color-gray.png)
     2  
     3  [![Go Report Card](https://goreportcard.com/badge/github.com/kubeshop/testkube-executor-jmeter)](https://goreportcard.com/report/github.com/kubeshop/testkube-executor-jmeter)
     4  [![Go Reference](https://pkg.go.dev/badge/github.com/kubeshop/testkube-executor-jmeter.svg)](https://pkg.go.dev/github.com/kubeshop/testkube-executor-jmeter)
     5  [![License](https://img.shields.io/github/license/kubeshop/testkube-executor-jmeter)]()
     6  
     7  # Distributed JMeter Executor
     8  An extension of JMeter Executor which can run the JMeter Tests in distributed mode by creating slave pods and distributing the test among them.
     9  
    10  ## What is an Executor?
    11  
    12  Executor is nothing more than a program wrapped into Docker container which gets JSON (testube.Execution) OpenAPI based document as an input and returns a stream of JSON output lines (testkube.ExecutorOutput),
    13  where each output line is simply wrapped in this JSON, similar to the structured logging idea.
    14  
    15  ## Features
    16  This executor is an extension of JMeter executor and has all the features of JMeter executor. In addition to that, it has the following features:
    17  * Can run JMeter tests in distributed mode by creating slave pods and distributing the test among them.
    18  * Supports defining plugins for a test in a git repo by placing plugins in a directory named `plugins` in the test folder.
    19  * Supports overriding JMeter `user.properties` file by placing a custom `user.properties` file in the test folder.
    20  
    21  ## Usage
    22  
    23  ### Supported Environment Variables
    24  
    25  1. **MASTER_OVERRIDE_JVM_ARGS / SLAVES_OVERRIDE_JVM_ARGS**: Used to override default memory options for JMeter master/slaves. Example: `MASTER_OVERRIDE_JVM_ARGS=-Xmn256m -Xms512m -Xmx512m`.
    26  
    27  2. **SLAVES_COUNT**: Specifies the number of slave pods required for Distributed JMeter tests. Example: `SLAVES_COUNT=3`. If `SLAVES_COUNT` is not provided or is equal to 1, then the executor will run the test in standalone mode.
    28  
    29  3. **MASTER_ADDITIONAL_JVM_ARGS / SLAVES_ADDITIONAL_JMETER_ARGS**: Allows exporting additional JVM arguments for slaves/master. Example: `MASTER_ADDITIONAL_JVM_ARGS=-Xmx1024m -Xms512m -XX:MaxMetaspaceSize=256m`.
    30  
    31  4. **SLAVES_ADDITIONAL_JMETER_ARGS**: Provides additional JVM arguments for JMeter server / slaves. Example: `SLAVES_ADDITIONAL_JMETER_ARGS=jmeter-server -Jserver.rmi.ssl.disable=true -Dserver_port=60000`.
    32  
    33  ### Guide
    34  
    35  Below guide will provide you the details about how to run a Jmeter test in distributed environment.
    36  
    37  1. File option:
    38  
    39     When you provide a test (.jmx) file to `Distributed JMeter ( JMeter in distributed mode )`,
    40     the executor of `Distributed JMeter` will spawn number of slaves pods specified by user through `SLAVES_COUNT` environment variable as described above and run the test on all the slave pods.
    41  
    42  2. Git Option:
    43     Using Git flow of the executor we can have use advanced features of `Distributed JMeter` executor which is not possible with JMeter executor:
    44  
    45      - Additional files required by a particular test like a CSV or JSON file can be provided through git repo.
    46        There is an example of using a CSV file by test (.jmx) file in the `example` folder of `Distributed JMeter`.
    47      - Dynamic plugins required for a test by keeping the plugins inside the test folder in a directory named `plugins` in the git repo.
    48      - Overriding the JMeter `user.properties` can be provided by using custom `user.properties` file in the git repo.
    49  
    50  
    51  For using the Git option and to avail all the above features, user should have the following directory structure in the git repo:
    52  
    53     ```
    54        github.com/`<username>/<reponame>`/---
    55  
    56                                            |-test1/---
    57                                                     |- testfile1.jmx
    58                                                     |- userdata.csv ( or any other additional file )
    59                                                     |- user.properties
    60                                                     |- plugins/---
    61                                                                 |- plugin-manager.jar
    62                                                                 |- <jar file of any other required plugins to run test1>
    63  
    64                                            |-test2/---
    65                                                     |- testfile2.jmx
    66                                                     |- userdata.json ( or any other additional file )
    67                                                     |- user.properties
    68                                                     |- plugins/---
    69                                                                 |- plugin-manager.jar
    70                                                                 |- <jar file of any other required plugins to run test2>
    71  
    72     ```
    73  
    74  For additional info, see the [GitFlow Example test for Distributed JMeter](./examples/gitflow/README.md).
    75  
    76  ## Local development
    77  
    78  ### Prerequisites
    79  
    80  Make sure the following tools are installed on your machine and available in your PATH:
    81  * [JMeter](https://jmeter.apache.org/download_jmeter.cgi) - pure Java application designed to load test functional behavior and measure performance
    82  
    83  ### Setup
    84  1. Create a directory called `data/` where JMeter will run and store results (best practice is to create it in the project root because it is git-ignored)
    85  2. Create a JMeter XML project file and save it as a file named `test-content` in the newly created `data/` directory
    86  3. Create an execution JSON file and save it as a file named `execution.json` based on the template below (best practice is to save it in the `temp/` folder in the project root because it is git-ignored)
    87      ```json
    88      {
    89        "id": "jmeterd-test",
    90        "args": [],
    91        "variables": {},
    92        "content": {
    93          "type": "string"
    94        }
    95      }
    96      ```
    97  4. You need to provide the `RUNNER_SCRAPPERENABLED`, `RUNNER_SSL` and `RUNNER_DATADIR` environment variables and run the Executor using the `make run run_args="-f|--file <path>"` make command where `-f|--file <path>` argument is the path to the `execution.json` file you created in step 3.
    98      ```bash
    99      RUNNER_SCRAPPERENABLED=false RUNNER_SSL=false RUNNER_DATADIR="./data" make run run_args="-f temp/execution.json"
   100      ```
   101  
   102  #### Execution JSON
   103  
   104  Execution JSON stores information required for an Executor to run the configured tests.
   105  
   106  Breakdown of the Execution JSON:
   107  ```json
   108  {
   109     "args": ["-n", "-t", "test.jmx"],
   110     "variables": {
   111        "example": {
   112           "type": "basic", 
   113           "name": "example", 
   114           "value": "some-value"
   115       }
   116     },
   117     "content": {
   118        "type": "string"
   119     }
   120  }
   121  ```
   122  * **args** - array of strings which will be passed to JMeter as arguments
   123    * example: `["-n", "-t", "test.jmx"]`
   124  * **variables** - map of variables which will be passed to JMeter as arguments
   125    * example: `{"example": {"type": "basic", "name": "example", "value": "some-value"}}` 
   126  * **content.type** - used to specify that JMeter XML is provided as a text file
   127  
   128  #### Environment Variables
   129  ```bash
   130  RUNNER_SSL=false                  # used if storage backend is behind HTTPS, should be set to false for local development
   131  RUNNER_SCRAPPERENABLED=false      # used to enable/disable scrapper, should be set to false for local development
   132  RUNNER_DATADIR=<path-to-data-dir> # path to the data/ directory where JMeter will run and store results
   133  ```
   134  
   135  ## Testing in Kubernetes
   136  
   137  ### Prerequisites
   138  * Kubernetes cluster with Testkube installed (best practice is to install it in the `testkube` namespace)
   139  
   140  ### Guide
   141  
   142  After validating locally that the Executor changes work as expected, next step is to test whether Testkube can successfully schedule a Test using the new Executor image.
   143  
   144  NOTE: The following commands assume that Testkube is installed in the `testkube` namespace, if you have it installed in a different namespace, please adjust the `--namespace` flag accordingly.
   145  
   146  The following steps need to be executed in order for Testkube to use the new Executor image:
   147  1. Build the new Executor image using the `make docker-build-local` command. By default, the image will be tagged as `kubeshop/testkube-executor-jmeter:999.0.0` unless a `LOCAL_TAG` environment variable is provided before the command.
   148  2. Now you need to make the image accessible in Kubernetes, there are a couple of approaches:
   149     * *kind* - `kind load docker-image <image-name> --name <kind cluster name>` (e.g. `kind load docker-image testkube-executor-jmeter:999.0.0 --name testkube-k8s-cluster`)
   150     * *minikube* - `minikube image load <image-name> --profile <minikube profile>` (e.g. `minikube image load testkube-executor-jmeter:999.0.0 --profile k8s-cluster-test`)
   151     * *Docker Desktop* - just by building the image locally, it becomes accessible in the Docker Desktop Kubernetes cluster
   152     * *other* - you can push the image to a registry and then Testkube will pull it in Kubernetes (assuming it has credentials for it if needed)
   153  3. Edit the Job Template and change the `imagePullPolicy` to `IfNotPresent`
   154     * Edit the ConfigMap `testkube-api-server` either by running `kubectl edit configmap testkube-api-server --namespace testkube` or by using a tool like Monokle
   155     * Find the `job-template.yml` key and change the `imagePullPolicy` field in the `containers` section to `IfNotPresent`
   156  4. Edit the Executors configuration and change the base image to use the newly created image:
   157     * Edit the ConfigMap `testkube-api-server` either by running `kubectl edit configmap testkube-api-server --namespace testkube` or by using a tool like Monokle
   158     * Find the `executors.json` key and change the `executor.image` field to use the newly created image for the JMeter Executor (`name` field is `jmeter-executor`)
   159  5. Restart the API Server by running `kubectl rollout restart deployment testkube-api-server --namespace testkube`
   160  
   161  Testkube should now use the new image for the Executor and you can schedule a Test with your preferred method.