github.com/kata-containers/tests@v0.0.0-20240307153542-772105b56064/metrics/report/README.md (about)

     1  # Kata Containers metrics report generator
     2  
     3  The files within this directory can be used to generate a "metrics report"
     4  for Kata Containers.
     5  
     6  The primary workflow consists of two stages:
     7  
     8  1) Run the provided report metrics data gathering scripts on the system(s) you wish
     9  to analyze.
    10  2) Run the provided report generation script to analyze the data and generate a
    11  report file.
    12  
    13  ## Data gathering
    14  
    15  Data gathering is provided by the `grabdata.sh` script. When run, this script
    16  executes a set of tests from the `tests/metrics` directory. The JSON results files
    17  will be placed into the `tests/metrics/results` directory.
    18  
    19  Once the results are generated, create a suitably named subdirectory of
    20  `tests/metrics/results`, and move the JSON files into it.
    21  
    22  Repeat this process if you want to compare multiple sets of results. Note, the
    23  report generation scripts process all subdirectories of `tests/metrics/results` when
    24  generating the report.
    25  
    26  > **Note:** By default, the `grabdata.sh` script tries to launch some moderately
    27  > large containers (i.e. 8Gbyte RAM) and may fail to produce some results on a memory
    28  > constrained system.
    29  
    30  You can restrict the subset of tests run by `grabdata.sh` via its commandline parameters:
    31  
    32  | Option | Description |
    33  | ------ | ----------- |
    34  | -a | Run all tests (default) |
    35  | -d | Run the density tests |
    36  | -h | Print this help |
    37  | -s | Run the storage tests |
    38  | -t | Run the time tests |
    39  
    40  ## Report generation
    41  
    42  Report generation is provided by the `makereport.sh` script. By default this script 
    43  processes all subdirectories of the `tests/metrics/results` directory to generate the report.
    44  To run in the default mode, execute the following:
    45  
    46  ```sh
    47  $ ./makereport.sh
    48  ```
    49  
    50  The report generation tool uses [`Rmarkdown`](https://github.com/rstudio/rmarkdown),
    51  [R](https://www.r-project.org/about.html) and [Pandoc](https://pandoc.org/) to produce
    52  a PDF report. To avoid the need for all users to set up a working environment
    53  with all the necessary tooling, the `makereport.sh` script utilises a `Dockerfile` with
    54  the environment pre-defined in order to produce the report. Thus, you need to
    55  have Docker installed on your system in order to run the report generation.
    56  
    57  The resulting `metrics_report.pdf` is generated into the `output` subdirectory of the `report`
    58  directory.
    59  
    60  ## Debugging and development
    61  
    62  To aid in script development and debugging, the `makereport.sh` script offers a debug
    63  facility via the `-d` command line option. Using this option will place you into a `bash`
    64  shell within the running `Dockerfile` image used to generate the report, whilst also
    65  mapping your host side `R` scripts from the `report_dockerfile` subdirectory into the
    66  container, thus facilitating a "live" edit/reload/run development cycle.
    67  From there you can examine the Docker image environment, and execute the generation scripts.
    68  E.g., to test the `scaling.R` script, you can execute:
    69  
    70  ```sh
    71  $ makereport.sh -d
    72  # R
    73  > source('/inputdir/Env.R')
    74  > source('/scripts/lifecycle-time.R')
    75  ```
    76  
    77  You can then edit the `report_dockerfile/lifecycle-time.R` file on the host, and re-run
    78  the `source('/scripts/lifecycle-time.R')` command inside the still running `R` container
    79  to re-execute the newly edited script.