github.com/NVIDIA/aistore@v1.3.23-0.20240517131212-7df6609be51d/docs/howto_benchmark.md (about)

     1  ---
     2  layout: post
     3  title: HOW TO BENCHMARK
     4  permalink: /docs/howto-benchmark
     5  redirect_from:
     6   - /howto_benchmark.md/
     7   - /docs/howto_benchmark.md/
     8  ---
     9  
    10  ## How to benchmark with AIStore
    11  
    12  The tool we have developed is called AIS Loader (`aisloader`) - a load generator to benchmark and stress-test AIStore, or any S3-compatible backend. AIS Loader generates arbitrary read/write workloads including those (synthetic) ones that mimic training and inference workloads. The latter allows to run benchmarks in isolation and avoid compute-side bottlenecks, if any.
    13  
    14  For usage, run: `aisloader`, or `aisloader usage`, or `aisloader --help`.
    15  
    16  To get started, go to the `aistore` root directory and run:
    17  
    18  ```console
    19  $ make aisloader
    20  $ $GOPATH/bin/aisloader
    21  ```
    22  
    23  Following in this document are a few *easy* usage examples and dry-run instructions. For detailed description and more examples, please see [aisloader readme](/docs/aisloader.md).
    24  
    25  ### Basic Examples
    26  
    27  For the most recently updated command-line options and examples, please run `aisloader` or `aisloader usage`.
    28  
    29  1. Destroy existing ais bucket - the first example. Delete all objects in a given Cloud-based bucket - the second example:
    30  
    31      ```console
    32      $ aisloader -bucket=nvais -duration 0s -totalputsize=0
    33      $ aisloader -bucket=aws://nvais -cleanup=true -duration 0s -totalputsize=0
    34      ```
    35  
    36  2. Time-based 100% PUT into ais bucket. Upon exit the bucket is emptied (by default):
    37  
    38      ```console
    39      $ aisloader -bucket=nvais -duration 10s -numworkers=3 -minsize=1K -maxsize=1K -pctput=100 -provider=ais
    40      ```
    41  
    42  3. 100% GET from an ais bucket:
    43  
    44      ```console
    45      $ aisloader -bucket=nvais -duration 5s -numworkers=3 -pctput=0 -provider=ais
    46      ```
    47  
    48  4. Mixed 30%/70% PUT and GET of variable-size objects to/from a Cloud bucket. PUT will generate random object names and is limited by the 10GB total size. Cleanup is not disabled, which means that upon completion all generated objects will be deleted:
    49  
    50      ```console
    51      $ aisloader -bucket=nvaws -duration 0s -numworkers=3 -minsize=1024 -maxsize=1MB -pctput=30 -provider=cloud -totalputsize=10G
    52      ```
    53  
    54  5. PUT 2000 objects named as `aisloader/hex({0..2000}{loaderid})`:
    55  
    56      ```console
    57      $ aisloader -bucket=nvais -duration 10s -numworkers=3 -loaderid=11 -loadernum=20 -maxputs=2000 -objNamePrefix="aisloader"
    58      ```
    59  
    60  6. PUT 30% of random TAR files and 70% GET of transformed TARs to TFRecords with tar2tf transformation.
    61  
    62      ```console
    63      $ aisloader -bucket=nvaws -duration=5m -numworkers=3 -pctput=30 -readertype=tar -transformation=tar2tf
    64      ```
    65  
    66  ### AIS Loader full documentation
    67  
    68  Full aisloader documentation, including more examples and full options list, is available [here](/docs/aisloader.md).