github.com/kchristidis/fabric@v1.0.4-0.20171028114726-837acd08cde1/bddtests/regression/README.md (about)

     1  ## regression
     2  Regression Test Suites scripts, and folders for: execution results for
     3  significant releases, and
     4  configuration scripts and supporting files in a variety of languages:
     5  
     6  - [GO](https://github.com/hyperledger/fabric/bddtests/regression/go)
     7    language test scripts
     8  - [NODE](https://github.com/hyperledger/fabric/bddtests/regression/node)
     9    language test scripts
    10    Note: a flexible-use performance engine, which can be configured and used
    11    for a variety of tests, is located in the
    12    [node/performance](https://github.com/hyperledger/fabric/bddtests/regression/node/performance)
    13    folder.  For examples how to use it, check there, or look for examples in the
    14    daily test suite and long-run test suite scripts.
    15  - [results](https://github.com/hyperledger/fabric/bddtests/regression/results)
    16    folder containing logfiles and results of running the Test Suites on
    17    significant releases, for reference
    18  
    19  
    20  ### Continous Integration Setup
    21  Continuous Integration team will execute the *daily test suite* each day when a
    22  merge commit has been pushed to fabric repository.
    23  We have configured Jenkins in vLaunch to execute the Daily Test Suite:
    24  the jobs listed below are downstream jobs which are run after successfully
    25  executing the upstream job.
    26  After the build is successfully executed, Jenkins will post build results
    27  back to **rel-criteria-build** slack channel,
    28  and generate a test summary report, to be available for viewing for 30 days.
    29  For significant releases, the results will be stored
    30  in **bddtests/regression/results** folder.
    31  
    32  
    33  ### Daily Test Suite - daily_test_suite.sh
    34  Expected total time duration is between 13 - 16 hours.
    35  
    36  * Consensus Acceptance Tests (CAT) - using chaincode example02, GO, gRPC
    37    - Objective of CAT tests is to ensure the stability and resiliency of the
    38      BFT Batch design.
    39  * Large Networks Basic API And Consensus Tests - using chaincode example02,
    40    GO, gRPC
    41    - Objective of Basic gRPC API test is to ensure basic gRPC API functions
    42      are working as expected
    43  * Ledger Stress Tests (LST) for API and 20K runs with concurrency and
    44    1K payload - using chaincode addrecs, GO, gRPC
    45  * Speed Tests for measuring turnaround performance of the communication path
    46    and blockchain network processing - using chaincode addrecs, GO, gRPC
    47  * Concurrency Tests - using chaincode addrecs, GO, gRPC
    48    - Objective of Concurrency test is to ensure system is able to accept
    49      multiple threads concurrently for specified number of mins.
    50  * Complex Transactions Tests - using chaincode auction, node, gRPC
    51    - Objective of this test is to send complex transactions using
    52      auction chaincode
    53  * Performance Tests, random sized payloads - using chaincodes example02
    54    and auction, node, gRPC
    55    - CI performs below tests in Performance testing
    56       * Invoke on chaincode_example02 using 4 peers and
    57         4 Threads for 180 secs
    58       * Query on chaincode_example02 using 4 peers and
    59         4 Threads for 180 secs
    60       * Invoke on auction chaincode using 4 peers and
    61         1000 Tx for each of the 4 Threads
    62       * Query on auction chaincode using 4 peers and
    63         1000 Tx for each of the 4 Threads
    64       * Invoke on auction chaincode using 4 peers and 4 Threads.
    65         Each Invoke is followed by a Query on every Thread
    66       * Invoke on example02 chaincode using 4 peers and 4 Threads.
    67  Each Invoke is followed by a Query on every Thread
    68  
    69  
    70  ### LongRun Test Suite - longrun_test_suite.sh
    71  Expected total time duration is 72 hours when executed from
    72  Jenkins automation scripts, which run several parallel jobs.
    73  
    74  * Consensus peers restarts tests, 2 tests, each approximately 10 hours -
    75    using chaincode example02, GO, gRPC
    76  * Ledger Stress Test 1 Million transactions, approximately 36 hours -
    77    using chaincode addrecs, GO, gRPC
    78  * Ledger Stress Test, 72 hours - using chaincode addrecs, GO, gRPC
    79    - Send parallel Invoke requests on all 4 peers
    80  * Performance Tests, 72 hours, random sized payloads - using chaincode auction,
    81    node, gRPC (includes variable traffic, concurrency, complex transactions)
    82    - Invoke 1 Tx per sec on 1 peer for 72 hrs (1 Thread)
    83  
    84  
    85  ### Full Regression Test Suite
    86  Execute both the Daily Test Suite AND the Long Run Test Suite.
    87  
    88  
    89  <a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.
    90  s