github.com/prysmaticlabs/prysm@v1.4.4/shared/benchutil/README.md (about)

     1  # Benchmarks for Prysm State Transition
     2  This package contains the functionality needed for benchmarking Prysms state transitions, this includes its block processing (with and without caching) and epoch processing functions. There is also a benchmark for HashTreeRoot on a large beacon state.
     3  
     4  ## Benchmark Configuration
     5  The following configs are in `config.go`:
     6  * `ValidatorCount`: Sets the amount of active validators to perform the benchmarks with. Default is 16384.
     7  * `AttestationsPerEpoch`: Sets the amount of attestations per epoch for the benchmarks to perform with, this affects the amount of attestations in a full block and the amount of attestations per epoch in the state for the `ProcessEpoch` and `HashTreeRoot` benchmark. Default is 128.
     8  
     9  ## Generating new SSZ files
    10  Due to the sheer size of the benchmarking configurations (16384 validators), the files used for benchmarking are pregenerated so there's no wasted computations on generating a genesis state with 16384 validators. This should only be needed if there is a breaking spec change and the tests fail from SSZ issues.
    11  
    12  To generate new files to use for benchmarking, run the below command in the root of Prysm.
    13  ```
    14  bazel run //tools/benchmark-files-gen -- --output-dir $PRYSMPATH/shared/benchutil/benchmark_files/ --overwrite
    15  ```
    16  
    17  ## Running the benchmarks
    18  To run the ExecuteStateTransition benchmark:
    19  
    20  ```bazel test //beacon-chain/core/state:go_default_test --test_filter=BenchmarkExecuteStateTransition_FullBlock --test_arg=-test.bench=BenchmarkExecuteStateTransition_FullBlock```
    21  
    22  To run the ExecuteStateTransition (with cache) benchmark:
    23  
    24  ```bazel test //beacon-chain/core/state:go_default_test --test_filter=BenchmarkExecuteStateTransition_WithCache --test_arg=-test.bench=BenchmarkExecuteStateTransition_WithCache```
    25  
    26  To run the ProcessEpoch benchmark:
    27  
    28  ```bazel test //beacon-chain/core/state:go_default_test --test_filter=BenchmarkProcessEpoch_2FullEpochs --test_arg=-test.bench=BenchmarkProcessEpoch_2FullEpochs```
    29  
    30  To run the HashTreeRoot benchmark:
    31  
    32  ```bazel test //beacon-chain/core/state:go_default_test --test_filter=BenchmarkHashTreeRoot_FullState --test_arg=-test.bench=BenchmarkHashTreeRoot_FullState```
    33  
    34  To run the HashTreeRootState benchmark:
    35  
    36  ```bazel test //beacon-chain/core/state:go_default_test --test_filter=BenchmarkHashTreeRootState_FullState --test_arg=-test.bench=BenchmarkHashTreeRootState_FullState```
    37  
    38  Extra flags needed to benchmark properly:
    39  
    40  ```--nocache_test_results --test_arg=-test.v --test_timeout=2000 --test_arg=-test.cpuprofile=/tmp/cpu.profile --test_arg=-test.memprofile=/tmp/mem.profile --test_output=streamed```
    41  
    42  ## Current Results as of January 2020
    43  ```
    44  BenchmarkExecuteStateTransition_FullBlock-4           20	  2031438030 ns/op
    45  BenchmarkExecuteStateTransition_WithCache-4   	      20	  1857290454 ns/op
    46  BenchmarkHashTreeRoot_FullState-4   	              50	   297655834 ns/op
    47  BenchmarkHashTreeRootState_FullState-4                50           155535883 ns/op
    48  ```