github.com/aakash4dev/cometbft@v0.38.2/scripts/qa/reporting/README.md (about)

     1  # Reporting Scripts
     2  
     3  This directory contains some utility scripts used in the reporting/QA.
     4  
     5  * [`latency_throughput.py`](./latency_throughput.py) is a Python script that uses
     6      [matplotlib] to plot a graph of transaction latency vs throughput rate based on
     7      the CSV output generated by the [loadtime reporting
     8      tool](../../../test/loadtime/cmd/report/).
     9  
    10  * [`latency_plotter.py`](./latency_plotter.py) is a Python script that uses
    11      [matplotlib] and [pandas] to plot graph of transaction latency vs throughput rate based on
    12      the CSV output generated by the [loadtime reporting
    13      tool](../../../test/loadtime/cmd/report/), for multiple experiments and configurations.
    14  
    15  * [`prometheus_plotter.py`](./prometheus_plotter.py) is a Python script that uses
    16      [matplotlib] and [pandas] to plot graphs of several metrics from Prometheus.
    17  
    18  ## Setup
    19  
    20  Execute the following within this directory (the same directory as the
    21  `latency_throughput.py` file).
    22  
    23  ```bash
    24  # Create a virtual environment into which to install your dependencies
    25  python3 -m venv .venv
    26  
    27  # Activate the virtual environment
    28  source .venv/bin/activate
    29  
    30  # Install dependencies listed in requirements.txt
    31  pip install -r requirements.txt
    32  ```
    33  
    34  ## Latency vs Throughput Plotting
    35  To show the instructions and parameter options, execute 
    36  
    37  ```bash
    38  ./latency_throughput.py --help
    39  ```
    40  
    41  Example:
    42  
    43  ```bash
    44  # Do the following while ensuring that the virtual environment is activated (see
    45  # the Setup steps).
    46  #
    47  # This will generate a plot in a PNG file called 'tm034.png' in the current
    48  # directory based on the reporting tool CSV output in the "raw.csv" file. The
    49  # '-t' flag overrides the default title at the top of the plot.
    50  
    51  ./latency_throughput.py \
    52      -t 'CometBFT v0.34.x Latency vs Throughput' \
    53      ./tm034.png \
    54      /path/to/csv/files/raw.csv
    55  ```
    56  
    57  ## Latency vs Throughput Plotting (version 2)
    58  Example:
    59  
    60  ```bash
    61  # Do the following while ensuring that the virtual environment is activated (see
    62  # the Setup steps).
    63  #
    64  # This will generate a series of plots in the `imgs` folder.
    65  # Plots include combined experiment plots and experiments as subplots.
    66  # - all_experiments - plots of all experiments as individual subplots.
    67  # - all_configs - plots of all experiments, grouped by configuration (r,c).
    68  # cXrY.png - Independent plot of experiments of configuration (c=X,r=Y) as different curves.
    69  # cXrY_merged.png - Independent plot of experiments of configuration (c=X,r=Y) combined as single curve.
    70  # e_ID.png - independent plot with just experiment with id ID as a single curve.
    71  
    72  mkdir -p imgs
    73  python3 latency_plotter.py /path/to/csv/files/raw.csv
    74  ```
    75  
    76  ## Prometheus metrics
    77  
    78  1. Ensure that Prometheus is running locally and listening on port 9090. 
    79  2. Tweak the script to your needs
    80     1. Adjust the time window
    81     2. Select the right fork 
    82     3. Select the right test case
    83     4. Tweak/add/remove metrics
    84  3. Run the script as follows
    85     ```bash
    86     # Do the following while ensuring that the virtual environment is activated (see
    87     # the Setup steps).
    88     #
    89     # This will generate a series of plots in the folder `imgs` of the current folder.
    90  
    91     mkdir imgs
    92     python3 prometheus_plotter.py
    93     ```
    94  4. Plots are saved in the `imgs` folder.
    95  
    96  [matplotlib]: https://matplotlib.org/
    97  [pandas]: https://pandas.pydata.org