github.com/tiagovtristao/plz@v13.4.0+incompatible/src/test/README.md (about)

     1  Please Tests
     2  ============
     3  
     4  Reporting
     5  ---------
     6  
     7  Tests are reported in much the same output format as the Maven Surefire Plugin.
     8  This makes it easy to consume the results in popular CI tools with a minimum
     9  of effort.
    10  
    11  After executing `plz test` against one or more targets, the test results are
    12  written to `plz-out/log/test_results.xml`. 
    13  
    14  We no longer support the idea of "expected failures" as people who write tests
    15  they expect to fail will find they are first against the wall when the revolution
    16  comes.
    17  
    18  Executing
    19  ---------
    20  
    21  Executing `plz test` against a target produces a set of results that are
    22  collated into one `core.TestSuite` with the package and name matching the
    23  target's label (this is a slight departure from the normal package/classname
    24  decomposition as it does not particularly translate well to non-Java languages).
    25  
    26  If any tests fail, and the target is marked as `flaky`, it will be run again
    27  (up to `flaky` times). Tests can also be explicitly run multiple times by
    28  passing `--num-runs n` on the command line. We no longer accept a default of `0`
    29  as running a test 0 times is (a) not useful and (b) not what actually happened.
    30  
    31  Each executed test during this target becomes a `core.TestExecution` under a
    32  `core.TestCase` in this test suite - so that multiple executions of a test
    33  are put together. We ignore multiple successes or skips as they are not interesting.
    34  In the case of a flaky test where some runs succeed and some fail, we store at
    35  most one success but also all of the failures and errors. 
    36  
    37  The `time` of the testsuite is the total time it took to execute all of the
    38  tests (including any re-runs for flakes).
    39  
    40  Once all of the targets have been executed, these `core.TestSuite`s are
    41  collected into one `core.TestSuites` object. This is then rendered as XML
    42  to the output file mentioned above.