github.com/kchristidis/fabric@v1.0.4-0.20171028114726-837acd08cde1/test/tools/LTE/README.md (about)

     1  # Ledger Traffic Engine
     2  
     3  This readme explains the working and the usage of Ledger Traffic Engine (LTE
     4  test tool.
     5  
     6  
     7  - What is the LTE Test Tool
     8  - How to Run the Tests
     9  - Where to View the Results
    10  
    11  ## What is the LTE Test Tool
    12  
    13  The LTE test tool is test harness that engages the Ledger APIs and benchmarks
    14  the performance of the Ledger component. It contains the framework for creating
    15  and managing chains, for transaction submission, block creation, and block
    16  commit. It conducts benchmark tests for insert and read-write transactions
    17  (transactions per second) and contains scripts for launching the benchmarks. An
    18  insert benchmark followed by a readwrite benchmark on the same dataset is
    19  considered to constitute a single test-run.
    20  
    21  
    22  ## How to Run The tests
    23  
    24  To run all the available tests, run:
    25  ```
    26  cd fabric/test/tools/LTE/scripts
    27  ./runbenchmark.sh -f parameters_daily_CI.sh all
    28  ```
    29  where the file `parameters_daily_CI.sh` has all the necessary test parameters.
    30  
    31  
    32  You can run individual tests without running all the available tests by giving
    33  the name of the test as parameter, instead of `all`. You can get the available
    34  test names by:
    35  ```
    36  ./runbenchmark.sh help
    37  ```
    38  
    39  ### What the Tests Do
    40  
    41  Each test reads test parameters from the provided parameter file and
    42  conducts several test-runs by varying one or two of the parameters. The
    43  parameters are:
    44  * number of chains (ledger),
    45  * number of parallel transactions in each chain,
    46  * number of Key-value pairs,
    47  * number of transactions,
    48  * number of keys in each transaction,
    49  * size of batch for ledger,
    50  * size of Key-value
    51  
    52  For example, the *varyNumChains* test reads the parameters and varies the
    53  number of chains for each test-run while keeping the other parameters constant,
    54  and generate result. Varying a specific parameter for each test-runs gives
    55  insight into the parameter's influence on the Ledger performance.
    56  
    57  Each test-run consists of two phases: benchmarking of ledger insert phase and
    58  benchmarking of ledger read-write phase.
    59  
    60  #### Insert Phase
    61  
    62  The insert benchmark starts fresh chains and inserts the Key-values by
    63  simulating writes-only transactions For each of the chains. It launches the
    64  parallel clients (based on the configuration) and the clients simulate and
    65  commit write-only transactions. The keys are divided among clients such that
    66  one key is written only once and all the keys are inserted. For instance, if
    67  this benchmark is invoked with the following parameters:
    68  ```
    69  Number of chains=2,
    70  Number of parallel transactions per chain=2,
    71  Number of key-value pairs=100
    72  ```
    73  then client_1 on chain_1 will insert Key_1 to key_25 and client_2 on chain_1 will
    74  insert Key_26 to key_50. Similarly client_3 on chain_2 will insert Key_1 to
    75  key_25 and client_4 on chain_2 will insert Key_26 to key_50 where, client_1 and
    76  client_2 run in parallel on chain_1 and client_3 and client_4 run in parallel
    77  on chain_2.
    78  
    79  #### Read-write Phase
    80  
    81  Subsequently, the Read-write benchmark step opens the existing chains and
    82  modifies the Key-values by simulating read-write transactions For each of the
    83  chains. It launches the parallel clients (based on the configuration) and the
    84  clients simulate and commit read-write transactions. This test assumes the
    85  pre-populated chain by previously running insert benchmark step (described
    86  above). Each transaction simulated by this benchmark randomly selects a
    87  configurable number of keys and modifies their values. For instance, if this
    88  benchmark is invoked with the following test parameters:
    89  ```
    90  Number of chains=2,
    91  Number of parallel transactions per chain=2,
    92  Number of key-value pairs=100,
    93  Number of total transactions=200
    94  ```
    95  then client_1, and client_2 both execute 50 transactions on
    96  chain_1 in parallel.  Similarly, client_3, and client_4 both execute 50
    97  transactions on chain_2 in parallel In each of the transactions executed by any
    98  client, the transaction expects and modifies any key(s) between Key_1 to key_50
    99  (because, total keys are to be 100 across two chains).
   100  
   101  ### Running with Custom Parameters
   102  
   103  The tests can be run with user-defined parameters by creating a new file that
   104  has all the necessary parameters to run and using that file as the input (see
   105  the section on how to run the tests) . The names of necessary parameters can be
   106  found in the file `parameters_daily_CI.sh`.
   107  
   108  ## How to View the Test Results
   109  
   110  The test results can be viewed as in the stdout where it shows how long each
   111  single operation took to complete in a test. These results are also saved in a
   112  .csv file in the following directory: `/tmp/experiments`