github.com/facebookincubator/ttpforge@v1.0.13-0.20240405153150-5ae801628835/docs/foundations/tests.md (about)

     1  # Tests for TTPs
     2  
     3  You can write tests for your TTPs using the `tests:` section of a TTPForge YAML
     4  file. These tests serve two purposes:
     5  
     6  - They act as continuously-validated documentation for how users should run your
     7    TTP.
     8  - They help ensure that the TTPForge engine will remain compatible with your TTP
     9    and provide warning if this compatibility is broken for any reason.
    10  
    11  ## Basic Test Cases
    12  
    13  The simplest-possible test case that you can define for a TTP is shown below:
    14  
    15  https://github.com/facebookincubator/TTPForge/blob/7634dc65879ec43a108a4b2d44d7eb2105a2a4b1/example-ttps/tests/minimal-test-case.yaml#L1-L14
    16  
    17  When you run the test cases for this TTP via the command
    18  `ttpforge test examples//tests/minimal-test-case.yaml`, TTPForge will call
    19  `ttpforge run` and pass the absolute path to your TTP file as an argument. In
    20  this instance, the `tests` syntax may seem superfluous, but even in this simple
    21  case it plays a very important role: **by declaring a test case, you are telling
    22  TTPForge that your TTP is safe to run as an automated test.**
    23  
    24  ## Test Cases with Arguments
    25  
    26  The `tests` feature really starts to show its value when used for TTPs that
    27  expect command-line arguments. An example of such a TTP, with two associated
    28  test cases, is shown below:
    29  
    30  https://github.com/facebookincubator/TTPForge/blob/7634dc65879ec43a108a4b2d44d7eb2105a2a4b1/example-ttps/tests/with-args.yaml#L1-L46
    31  
    32  When you test this TTP via `ttpforge test examples//tests/with-args.yaml`, both
    33  of the test cases in the above file will be run sequentially. TTPForge will
    34  parse the provided `args` list, encode each entry in the string format
    35  `--arg foo=bar`, and then append each resulting string to a dynamically
    36  generated `ttpforge run` command. The subsequent execution of that command
    37  verifies that the TTP functions correctly for that test case.
    38  
    39  ## Dry-Run Test Cases
    40  
    41  Some TTPs can only be executed except under very specific conditions - for
    42  example, Active Directory exploits that target domain controllers. It may not be
    43  feasible to test execution of such a TTP in an automated setting; however, it is
    44  still possible to verify that the TTP parses its arguments correctly and that
    45  all TTPForge validation phases _prior to actual execution_ complete
    46  successfully. To perform "validation without execution" in this manner, add
    47  `dry_run: true` to your test case, as shown below:
    48  
    49  https://github.com/facebookincubator/TTPForge/blob/7634dc65879ec43a108a4b2d44d7eb2105a2a4b1/example-ttps/tests/dry-run.yaml#L1-L30