github.com/NVIDIA/aistore@v1.3.23-0.20240517131212-7df6609be51d/python/tests/unit/botocore_patch/README.md (about)

     1  # A note on botocore monkey patch testing
     2  
     3  These tests verify the behavior of aistore.botocore_patch.botocore, a monkey patch
     4  for the Amazon botocore library that modifies it to respect the HTTP 
     5  `location` header when receiving a HTTP 301, 302 or 307.
     6  
     7  There are two degrees of freedom to test here:
     8  
     9   - We've either monkey patched botocore, or we haven't ("patched" or "unpatched")
    10   - The upstream service either sends redirects (like aistore), or it doesn't ("redirects" or "noredirects").
    11  
    12  We have four test sets which each run the same base fixture:
    13  
    14   - `test_botocore_noredirect_unpatched.py`: A control case for the default botocore behavior
    15   - `test_botocore_noredirect_patched.py`: A passthrough test to check the monkey patch doesn't break botocores normal behavior
    16   - `test_botocore_redirects_unpatched.py`: Another control test: unpatched botocore should throw errors when redirects happen
    17   - `test_botocore_redirects_patched.py`: Positive tests for the monkey patch itself
    18  
    19  We use [moto](https://github.com/spulec/moto) to mock an S3 like endpoint, and monkey patch it to send redirects when wanted.
    20  
    21  Since both the monkey patch and our moto patches rely on runtime imports, running tests like this:
    22  
    23  ```
    24  pytest -v tests/unit
    25  ```
    26  
    27  ...won't work. It's hard to "unimport" a module in python, and so previous tests will contaminate each other's state.
    28  
    29  To run these tests, you need to start a new forked process each time, scoped per test file.
    30  To do this inline we use the pytest-xdist plugin like so:
    31  
    32  ```
    33  pytest -v -n $(MP_TESTCOUNT) --dist loadfile tests/unit/botocore_patch
    34  ```
    35  
    36  ...which is one of the reasons why this test set is kept separate from those for the aistore SDK proper.
    37  
    38  ## Testing different boto3 and botocore versions
    39  By default we pull in the latest `boto3` and `botocore` to test against (see `botocore_requirements.dev.txt`).
    40  
    41  You can alter this behavior by exporting `BOTO3_VERSION` and / or `BOTOCORE_VERSION` prior to running tests:
    42  
    43  ```
    44  export BOTO3_VERSION=1.26.24
    45  export BOTOCORE_VERSION=1.29.24
    46  make python_botocore_tests
    47  ```