github.com/NVIDIA/aistore@v1.3.23-0.20240517131212-7df6609be51d/python/tests/README.md (about)

     1  # Python Tests
     2  
     3  This directory contains unit tests and integration tests for each of the python packages we provide.
     4  It also contains tests for verifying s3 compatibility.
     5  
     6  ---
     7  
     8  ## Prerequisites
     9  
    10  Before running the tests, install Python and set up your environment (venv, pyenv, conda, etc.). 
    11  The below instructions assume `python` will execute the correct version of Python. 
    12  
    13  Next, with your environment active, from the `aistore/python` directory install the dependencies for the tests you want 
    14  to run:
    15  
    16  For all packages:
    17  
    18  `pip install -r aistore/common_requirements`
    19  
    20  For botocore patch tests:
    21  
    22  `pip install -r aistore/botocore_patch/botocore_requirements`
    23  
    24  For pytorch tests:
    25  
    26  `pip install -r aistore/pytorch/dev_requirements`
    27  
    28  ---
    29  
    30  ## Integration tests
    31  
    32  ### Cluster
    33  
    34  Integration tests require a running AIS cluster. 
    35  This can be specified with the environment variable `AIS ENDPOINT` or will default to http://localhost:8080. 
    36  
    37  ### ETL
    38  
    39  Note that any tests of etl functionality require the AIS cluster to be running in kubernetes. 
    40  These tests can be skipped using the pytest ignore option, e.g. `--ignore=tests/integration/sdk/test_etl_ops.py`
    41  
    42  ### Remote buckets
    43  
    44  To run tests that expect a remote bucket, e.g. testing eviction of an object stored in s3, the `BUCKET`
    45  environment variable must be set; otherwise these tests will be skipped. 
    46  
    47  ---
    48  
    49  ## Running the tests
    50  
    51  All tests should support running on all operating systems, given access to a working AIS cluster. 
    52  
    53  Because of the way our botocore patch works, simply running pytest on the entire test directory will fail those tests. See the [botocore test README](python/tests/unit/botocore_patch/README.md) for more info
    54  
    55  Below are the recommended commands for running the test suites. Run these commands from the aistore/python directory. 
    56  
    57  ---
    58  
    59  ### All SDK tests:
    60  
    61  `python -m pytest tests/unit/sdk tests/integration/sdk`
    62  
    63  
    64  #### Integration only, excluding ETL (ETL requires AIS cluster running on k8s)
    65  
    66  `python -m pytest tests/integration/sdk -m "not etl"`
    67  
    68  #### Integration only, with ETL
    69  
    70  `python -m pytest tests/integration/sdk -m etl`
    71  
    72  
    73  #### Unit only
    74  
    75  `python -m pytest tests/unit/sdk`
    76  
    77  ---
    78  
    79  ### Botocore patch
    80  
    81  #### Unit tests
    82  1.  Set the variable for number of tests
    83      ##### Windows (Powershell): 
    84          $BOTO_UNIT_TEST_COUNT = Get-ChildItem -Path "tests/unit/botocore_patch" -Filter "test*.py" | Measure-Object | Select-Object -ExpandProperty Count
    85      ##### Unix:
    86          BOTO_UNIT_TEST_COUNT=$(ls -1 tests/unit/botocore_patch/test*py | wc -l)
    87  2.  Run pytest with the xdist plugin to ensure each test runs separately
    88  
    89      `python -m pytest -v -n $BOTO_UNIT_TEST_COUNT --dist loadfile tests/unit/botocore_patch/`
    90  
    91  #### Integration tests
    92  
    93      python -m pytest tests/integration/botocore_patch
    94  
    95  ---
    96  
    97  ### PyTorch
    98  
    99      python -m pytest tests/integration/pytorch
   100  
   101  ---
   102  
   103  ### S3 compatibility
   104  
   105  Currently, we have 2 separate tests for AIS S3 compatibility.
   106  
   107  #### Custom s3 tests
   108  
   109  There is our own suite of tests, located in tests/integration/boto3. It can be run with
   110  
   111      python -m pytest -v tests/integration/boto3 
   112  
   113  
   114  #### Minio s3 tests
   115  
   116  `s3compat` contains a modified version of the s3 tests from the Minio Python SDK, see the [README](s3compat/README.md) for details.
   117  
   118  ### References
   119  
   120  * [Botocore Test README](unit/botocore_patch/README.md)
   121  * [S3 Compatibility Test README](s3compat/README.md)
   122