github.com/kchristidis/fabric@v1.0.4-0.20171028114726-837acd08cde1/test/regression/daily/README.rst.orig (about)

     1  # Daily Test Suite
     2  Click here for the latest daily status report (WIP: link TBD)
     3  
     4  ## Running tests
     5  The entire suite of tests may be executed from script [runDailyTestSuite.sh](runDailyTestSuite.sh).
     6  Refer to that script for more details about how to invoke each test.
     7  
     8  ## Adding new tests
     9  Contributors may add a new test to an existing related test tool group, or create a new one.
    10  Some examples:
    11  
    12  1. To add another test to an existing test suite subgroup, such as
    13     the Performance Traffic Engine (PTE) tool,
    14     add a test inside the existing python wrapper test_pte.py.
    15     The header comment section inside that script contains
    16     detailed steps explaining how to do so. In brief, it is as simple as
    17     copying a block of about nine lines and modify three things:
    18  
    19  ```
    20     (A) edit the testcase comments
    21     (B) edit the line which specifies the command and arguments to execute
    22     (C) edit the line that specifies the test result to be matched
    23  ```
    24  
    25  2. To add a new test with a new tool, it involves a few more steps:
    26  
    27  ```
    28     (A) Create and merge a new tool such as .../fabric/test/tools/NewTool/newTool.sh
    29     (B) create a new file .../fabric/test/regression/daily/test_newTool.py
    30         and define a python wrapper to invoke the new tool.
    31         Model it after others like test_example.py; the file should
    32         contain a testcase that looks something like this:
    33  
    34         def test_TLS(self):
    35            '''
    36            FAB-2032,FAB-3593
    37            Network: 2 Ord, 5 KB, 3 ZK, 2 Org, 4 Peers, 10 Chan, 10 CC
    38            Launch network, use NewTool to wreak havoc on the network by
    39            doing something crazy, and ensure the network handles it gracefully.
    40            Then remove network and cleanup.
    41            '''
    42            result = subprocess.check_output("../../tools/NewTool/newTool.sh arg1 arg2", shell=True)
    43            self.assertIn("A STRING from stdout of NewTool that indicates PASS", result)
    44  
    45     (C) add lines at the bottom of runDailyTestSuite.sh to
    46         invoke the new testcase(s) using the new tool:
    47  
    48         py.test -v --junitxml results.xml ./test_example.py
    49  ```
    50  
    51  ### Test Output: formatting requirements
    52  The Jenkins automation tool that runs the test suite expects
    53  to receive xml output to display. For this reason, we execute
    54  tests in one of the following ways:
    55  
    56  Option 1. (Useful for any test language including bash, tool binaries, etc):
    57     Invoke the test from within a python wrapper script, which allows
    58     searching the stdout for a user-defined test result string.
    59     Using the python wrapper makes it easy to provide the desired
    60     junitxml output format. For example:
    61  
    62  ```
    63        py.test -v --junitxml results.xml ./test_example.py
    64  ```
    65  
    66  Option 2. (Useful for GO tests):
    67     Execute "go" tests, and pipe output through a tool such as
    68     github.com/jstemmer/go-junit-report to convert to xml. e.g.:
    69  
    70  ```
    71        cd ../../tools/OTE
    72        go get github.com/jstemmer/go-junit-report
    73        go test -run ORD7 -v | go-junit-report >> results.xml
    74  ```
    75  
    76  ## Test Descriptions
    77  
    78  [Test Descriptions](README_testdescriptions.rst)
    79  
    80  [ChainCode Tests descriptions and how-to](chaincodeTests/README.rst)
    81  
    82  .. Licensed under Creative Commons Attribution 4.0 International License
    83     https://creativecommons.org/licenses/by/4.0/