github.com/smartcontractkit/chainlink-testing-framework/libs@v0.0.0-20240227141906-ec710b4eb1a3/docs/writing-a-test/soak.md (about)

     1  ---
     2  layout: default
     3  title: Soak Tests
     4  nav_order: 2
     5  parent: Writing a Test
     6  ---
     7  
     8  # Soak Tests
     9  
    10  Soak tests refer to running longer tests, that can take anywhere from hours to days to see how the application fares over long stretches of time. See some examples in our [soak test suite](https://github.com/smartcontractkit/integrations-framework/tree/main/suite/soak).
    11  
    12  The test framework is designed around you launching a test environment to a K8s cluster, then running the test from your personal machine. Your personal machine coordinates the chainlink nodes, reads the blockchain, etc. This works fine for running tests that take < 5 minutes, but soak tests often last days or weeks. So the tests become dependent on your local machine maintaining power and network connection for that time frame. This quickly becomes untenable. So the solution is to launch a `remote-test-runner` container along with the test environment.
    13  
    14  ## Writing the Test
    15  
    16  Since the test is being run from a `remote-test-runner` instead of your local machine, setting up and tearing down the test environment is a little different. So is connecting to things like the blockchain networks, chainlink nodes, and the mock adapter. Most other interactions should be as normal though.
    17  
    18  ```go
    19  // Connects to the soak test resources from the `remote-test-runner`
    20  env, err := environment.DeployOrLoadEnvironmentFromConfigFile(
    21    tools.ChartsRoot,      // Default location of helm charts to look for
    22    "/root/test-env.json", // Default location for the soak-test-runner container
    23  )
    24  log.Info().Str("Namespace", env.Namespace).Msg("Connected to Soak Environment")
    25  
    26  // Run test logic
    27  
    28  // Teardown remote suite
    29  if err := actions.TeardownRemoteSuite(keeperBlockTimeTest.TearDownVals()); err != nil {
    30    log.Error().Err(err).Msg("Error tearing down environment")
    31  }
    32  log.Info().Msg("Soak Test Concluded")
    33  ```
    34  
    35  ## Running the Test
    36  
    37  The soak tests are triggered by the [soak_runner_test.go](https://github.com/smartcontractkit/integrations-framework/blob/main/suite/soak/soak_runner_test.go) tests, or with `make test_soak`. When running, the test will check for a local config file: `remote_runner_config.yaml`. If it's not already created, it will generate one with some default values, and then inform you that you should modify those values.
    38  
    39  ```yaml
    40  test_regex: '@soak-ocr' # The regex of the test name to run
    41  test_directory: /Users/adam/Projects/integrations-framework/suite/soak/tests # The directory where the go tests you want the remote runner to run
    42  # Slack values are covered below
    43  ```
    44  
    45  Modify these values that make sense for the tests you want to run. Once the values are modified, you can run the test again. The soak runner test will then compile the tests that you pointed to by the `test_directory` into a `remote.test` executable. This executable, including your local `framework.yaml` and `networks.yaml` configs are uploaded to the remote test runner. Make sure to read the section below to take advantage of slack integration.
    46  
    47  ## Watching the Test
    48  
    49  The rest of the `remote_runner_config.yaml` file holds various Slack bot params to notify you when the test finishes.
    50  
    51  ```yaml
    52  slack_api_key: abcdefg # A Slack API key to upload test results with. This should be the `Bot User OAuth Token` 
    53  slack_channel: C01xxxxx # The Slack Channel ID (open your Slack channel details and copy the ID there)
    54  slack_user_id: U01xxxxx # Your Slack member ID https://zapier.com/help/doc/common-problems-slack
    55  ```
    56  
    57  The Slack Bot will need to have the following:
    58  
    59  * Permission for [files:write](https://api.slack.com/scopes/files:write)
    60  * Permission for [chat:write](https://api.slack.com/scopes/chat:write)
    61  * The bot must be invited into the channel you want it to notify in: `/invite @botname` HINT: If you get the error `not_in_channel` this is likely what you need to set up.
    62  
    63  ## After the Test
    64  
    65  The test environment **will stay active until you manually delete it from your Kubernetes cluster**. This keeps the test env alive so you can view the logs when the test is done. You can do so by [using kubectl](https://www.dnsstuff.com/how-to-tail-kubernetes-and-kubectl-logs), something like [Lens](https://k8slens.dev/), or use the [chainlink-testing-framework](https://github.com/smartcontractkit/chainlink-testing-framework/k8s/examples/dump/env.go) `dump` command.