github.com/MetalBlockchain/metalgo@v1.11.9/tests/antithesis/README.md (about)

     1  # Antithesis Testing
     2  
     3  This package supports testing with
     4  [Antithesis](https://antithesis.com/docs/introduction/introduction.html),
     5  a SaaS offering that enables deployment of distributed systems (such
     6  as Avalanche) to a deterministic and simulated environment that
     7  enables discovery and reproduction of anomalous behavior.
     8  
     9  ## Package details
    10  
    11  | Filename       | Purpose                                                                            |
    12  |:---------------|:-----------------------------------------------------------------------------------|
    13  | compose.go     | Generates Docker Compose project file and initial database for antithesis testing. |
    14  | config.go      | Defines common flags for the workload binary.                                      |
    15  | init_db.go     | Initializes initial db state for subnet testing.                                   |
    16  | node_health.go | Helper to check node health.                                                       |
    17  | avalanchego/   | Defines an antithesis test setup for avalanchego's primary chains.                 |
    18  | xsvm/          | Defines an antithesis test setup for the xsvm VM.                                  |
    19  
    20  ## Instrumentation
    21  
    22  Software running in Antithesis's environment must be
    23  [instrumented](https://antithesis.com/docs/instrumentation/overview.html)
    24  to take full advantage of the supported traceability. Since the
    25  Antithesis Go SDK only supports the amd64/x86_64 architecture as of this
    26  writing, running of instrumented binaries on Macs (arm64) is not possible
    27  without emulation (which would be very slow). To support test development
    28  on Macs, a local build will not be instrumented.
    29  
    30  ## Defining a new test setup
    31  
    32  When defining a new test setup - whether in the avalanchego repo or
    33  for a VM in another repo - following the example of an existing test
    34  setup is suggested. The following table enumerates the files defining
    35  a test setup:
    36  
    37  | Filename                                               | Purpose                                                |
    38  |:-------------------------------------------------------|:-------------------------------------------------------|
    39  | scripts/build_antithesis_images.sh                     | Builds the test images to deploy to antithesis         |
    40  | scripts/build_antithesis_[test setup]_workload.sh      | Builds the workload binary                             |
    41  | scripts/tests.build_antithesis_images.sh               | Validates the build of the test images                 |
    42  | tests/antithesis/[test setup]/main.go                  | The entrypoint for the workload binary                 |
    43  | tests/antithesis/[test setup]/Dockerfile.config        | Defines how to build the config image                  |
    44  | tests/antithesis/[test setup]/Dockerfile.node          | Defines how to build the instrumented node image       |
    45  | tests/antithesis/[test setup]/Dockerfile.workload      | Defines how to build the workload image                |
    46  | tests/antithesis/[test setup]/gencomposeconfig/main.go | Generates the compose configuration for the test setup |
    47  
    48  In addition, github workflows are suggested to ensure
    49  `scripts/tests.build_antithesis_images.sh` runs against PRs and
    50  `scripts/build_antithesis_images.sh` runs against pushes.
    51  
    52  ### Use of a builder image
    53  
    54  To simplify building instrumented (for running in CI) and
    55  non-instrumented (for running locally) versions of the workload and
    56  node images, a common builder image is used. If on an amd64 host,
    57  `tests/antithesis/avalanchego/Dockerfile.builder-instrumented` is used
    58  to create an instrumented builder. On an arm64 host,
    59  `tests/antithesis/avalanchego/Dockerfile.builder-uninstrumented` is
    60  used to create an uninstrumented builder. In both cases, the builder
    61  image is based on the default golang image and will include the source
    62  code necessary to build the node and workload binaries. The
    63  alternative would require duplicating builder setup for instrumented
    64  and non-instrumented builds for the workload and node images of each
    65  test setup.
    66  
    67  ## Troubleshooting a test setup
    68  
    69  ### Running a workload directly
    70  
    71  The workload of the 'avalanchego' test setup can be invoked against an
    72  arbitrary network:
    73  
    74  ```bash
    75  $ AVAWL_URIS="http://10.0.20.3:9650 http://10.0.20.4:9650" go run ./tests/antithesis/avalanchego
    76  ```
    77  
    78  The workload of a subnet test setup like 'xsvm' additionally requires
    79  a network with a configured chain for the xsvm VM and the ID for that
    80  chain needs to be provided to the workload:
    81  
    82  ```bash
    83  $ AVAWL_URIS=... CHAIN_IDS="2S9ypz...AzMj9" go run ./tests/antithesis/xsvm
    84  ```
    85  
    86  ### Running a workload with docker-compose
    87  
    88  Running the test script for a given test setup with the `DEBUG` flag
    89  set will avoid cleaning up the the temporary directory where the
    90  docker-compose setup is written to. This will allow manual invocation of
    91  docker-compose to see the log output of the workload.
    92  
    93  ```bash
    94  $ DEBUG=1 ./scripts/tests.build_antithesis_images.sh
    95  ```
    96  
    97  After the test script has terminated, the name of the temporary
    98  directory will appear in the output of the script:
    99  
   100  ```
   101  ...
   102  using temporary directory /tmp/tmp.E6eHdDr4ln as the docker-compose path"
   103  ...
   104  ```
   105  
   106  Running compose from the temporary directory will ensure the workload
   107  output appears on stdout for inspection:
   108  
   109  ```bash
   110  $ cd [temporary directory]
   111  
   112  # Start the compose project
   113  $ docker-compose up
   114  
   115  # Cleanup the compose project
   116  $ docker-compose down --volumes
   117  ```
   118  
   119  ## Manually triggering an Antithesis test run
   120  
   121  When making changes to a test setup, it may be useful to manually
   122  trigger an Antithesis test run outside of the normal schedule. This
   123  can be performed against master or an arbitrary branch:
   124  
   125   - Navigate to the ['Actions' tab of the avalanchego
   126     repo](https://github.com/ava-labs/avalanchego/actions).
   127   - Select the [Publish Antithesis
   128     Images](https://github.com/ava-labs/avalanchego/actions/workflows/publish_antithesis_images.yml)
   129     workflow on the left.
   130   - Find the 'Run workflow' drop-down on the right and trigger the
   131     workflow against the desired branch. The default value for
   132     `image_tag` (`latest`) is used by scheduled test runs, so consider
   133     supplying a different value to avoid interferring with the results
   134     of the scheduled runs.
   135   - Wait for the publication job to complete successfully so that the
   136     images are available to be tested against.
   137   - Select the [Trigger Antithesis Test Runs](https://github.com/ava-labs/avalanchego/actions/workflows/trigger-antithesis-runs.yml)
   138     workflow on the left.
   139   - Find the 'Run workflow' drop-down on the right and trigger the
   140     workflow against the desired branch. The branch only determines the
   141     CI configuration (the images have already been built), so master is
   142     probably fine. Make sure to supply the same `image_tag` that was
   143     provided to the publishing workflow and consider setting
   144     `recipients` to your own email rather than sending the test report
   145     to everyone on the regular distribution list.