github.com/treeverse/lakefs@v1.24.1-0.20240520134607-95648127bfb0/esti/docs/README.md (about)

     1  # System and Integration Testing
     2  
     3  When a pull request is being committed / merged, we run several test suites to ensure nothing got broken due to the introduced changes.
     4  Since these are system level tests, we created an infrastructure which deploys lakeFS on a VM and executes the tests using different clients on various storage adapters.
     5  
     6  The following guide explains how to set up and execute the system tests infrastructure in order to optimize the development process, and enable debugging of the tests in a local environment.
     7  
     8  ## Running system tests
     9  
    10  A couple of prerequisites before we start:
    11  
    12  * Go over our [contributing](https://docs.lakefs.io/contributing.html) page and make sure you have a working lakeFS environment.
    13  * [Docker](https://www.docker.com/)
    14  * [Curl](https://curl.se/)
    15  
    16  ### Running lakeFS
    17  
    18  *Before running the tests we need to run a lakeFS instance configured for local execution of system tests*
    19  
    20  Under _**esti/scripts**_ you will find the following
    21  
    22  1. **lakefs.yaml**
    23     1. Contains the lakeFS configuration required for local test execution
    24     2. We use local storage type to run the tests
    25     3. The data will be stored under the filesystem's _/tmp_ folder (can be configured to any folder with the required permissions)
    26  2. **runner.sh**
    27     1. Script to execute lakeFS and tests
    28  3. **set_env_vars.sh**
    29     1. Defines and sets all environment variables required for both tests and lakeFS execution
    30     2. Adding env vars or modifying existing variables should be done on this file
    31     3. Used as a reference for debug env as well (see next)
    32     
    33  
    34  To run lakeFS for testing simply execute the following command in a shell console:
    35  ```shell
    36     esti/scripts/runner.sh -r lakefs 
    37  ```
    38  The lakeFS server will run in the console and logs will be piped to both stdout and in a temporary file in the form of lakefs_<XXX>.log
    39  
    40  ### Executing tests
    41  
    42  To run all the tests under the system tests folder, execute the following command in a new shell console:
    43  ```shell
    44     esti/scripts/runner.sh -r test
    45  ```
    46  The tests will run, with the output piped to both stdout and in a temporary file in the form of lakefs_tests_<XXX>.log
    47  
    48  The system test infrastructure allows additional flags and parameters to be passed for execution.
    49  For example to run a specific test (by regex) we can use:
    50  ```shell
    51     esti/scripts/runner.sh -r test -test.run TestHooksSuccess
    52  ```
    53  This command will execute all tests which match the given string regex (in this case it will run a single test by that name)  
    54  For a full list of arguments run: 
    55  ```shell
    56     esti/scripts/runner.sh -r test --help
    57  ```
    58  
    59  ---
    60  
    61  Notes:
    62  1. By default, tests are performing lakeFS setup using predefined credentials. As consequence:
    63     1. Tests that validate creation of access key id and secret access key will not test this functionality
    64     2. This will require stopping and re-running the lakeFS server on each test execution
    65  2. To run tests on an already existing lakeFS instance (after initial setup has occurred):
    66     1. Modify the ESTI_SETUP_LAKEFS environment variable from 'true' to 'false'
    67     2. Use cautiously as some test preconditions will cause tests to fail on existing lakeFS environments
    68  
    69  ---
    70  
    71  ## Debugging lakeFS and the system tests using IntelliJ
    72  
    73  To debug the tests and server code we must create run configurations for both lakeFS and the system tests.
    74  
    75  ### LakeFS run configuration
    76  
    77  1. From the '**Run**' menu select '**Edit Configurations...**', then '**Add New Configuration**' (the plus sign) and create a new '**go build**' configuration  
    78      ![](img/go_build.png)
    79  2. Copy the configuration from the following screenshot
    80     1. Add basic environment variables from the _set_env_vars.sh_ script's lakeFS section
    81     2. Add additional / modify environment variables as needed
    82     
    83     ![](img/lakefs_config.png)
    84  3. Create a before launch, external tool to load the postgres docker image
    85     ![](img/postgres.png)
    86  4. Press 'Apply', make sure the configuration is saved (it is not grayed out on the left side menu) and press 'OK'
    87  5. Try to execute the target (either by run / debug) and verify lakeFS server is running
    88  
    89  ### System tests run configuration
    90  
    91  1. From the '**Run**' menu select '**Edit Configurations...**', and then '**Add New Configuration**' (the plus sign) and create a new '**go test**' configuration
    92  
    93     ![](img/go_test.png)
    94  2. Copy the configuration from the following screenshot
    95     1. Add basic environment variables from the _set_env_vars.sh_ script's System tests section
    96     2. Add additional / modify environment variables as needed
    97     3. To filter tests, you can provide a regular expression in the '**Pattern**' text box or alternatively provide the proper command line argument in the '**Prgram arguments**' text box
    98     
    99     ![](img/esti_config.png)
   100  3. Press 'Apply', make sure the configuration is saved (it is not grayed out on the left side menu) and press 'OK'
   101  4. Try to execute the target (either by run / debug) and verify tests are running
   102  
   103  ---
   104  
   105  Don't know where to start? Experiencing difficulties? Reach out on the #dev channel on [our Slack](https://lakefs.io/slack), and we will help you get started.
   106  
   107  ---
   108