github.com/GoogleContainerTools/skaffold/v2@v2.13.2/docs-v2/design_proposals/integrationtest-command.md (about)

     1  # integrationtest command
     2  
     3  * Author(s): Dominic Werner (@daddz)
     4  * Design Shepherd: Tejal Desai (@tejal29) 
     5  * Date: 2019-08-16
     6  * Status: Cancelled
     7  * Reason: 
     8    
     9    We discussed this proposal in Skaffold Community Hours and decided at this
    10    moment, the design proposal is just an entry point for running a script.
    11    It is not built up on Skaffold's run knowledge so far.
    12    With that said, the core team agrees Skaffold does not a solution
    13    for [#2561](https://github.com/GoogleContainerTools/skaffold/issues/2561)
    14    and [#992](https://github.com/GoogleContainerTools/skaffold/issues/992) and we want to
    15    explore this space.
    16            
    17  ## Background
    18  
    19  Currently, skaffold has no support for running integration/unit tests 
    20  that are part of a built artifact. Since this is a crucial part of a CI/CD
    21  pipeline it would be good to support this feature so skaffold can be used
    22  in every step of a pipeline.
    23  
    24  Proof-of-concept:
    25  - [Integration test command (\#2594)](https://github.com/GoogleContainerTools/skaffold/pull/2594)
    26  
    27  Related issues:
    28  - [Integrationtest phase (\#2561)](https://github.com/GoogleContainerTools/skaffold/issues/2561)
    29  - [Add \`test\` phase to skaffold runner (\#992)](https://github.com/GoogleContainerTools/skaffold/issues/992)
    30  ___
    31  
    32  ## Design
    33  
    34  #### New configuration options in skaffold.yaml
    35  
    36  ```yaml
    37  integrationtest:
    38      podSelector: 'app=skaffold-integration'
    39      testCommand: 'pytest tests/pass.py'
    40  ```
    41  
    42  A top-level key called `integrationtest` holds the configuration for running
    43  integration tests with skaffold.
    44  
    45  `podSelector`: Define in which pod the tests shall be executed.
    46  The pod will be looked up via the defined label across all namespaces.
    47  
    48  `testCommand`: Define the test command that will be executed in the pod.
    49  
    50  ##### Backwards compatibility
    51  
    52  The configuration is not required and thus should not impact older versions.
    53  
    54  #### New skaffold CLI command
    55  
    56  A new command called `skaffold integrationtest` is implemented that takes
    57  the values from `skaffold.yaml` to execute the tests and report the output
    58  and outcome.
    59  
    60  This command can then be used locally or within the CI pipeline.
    61  
    62  #### Usage in CI pipeline
    63  
    64  The usage within a CI pipeline could look like this:
    65  
    66  * build: `skaffold build --file-output build.json`
    67  * test: 
    68    * `skaffold deploy -n $(pipeline_id) -a build.json`
    69    * `skaffold integrationtest`
    70    * `skaffold delete -n $(pipeline_id)`
    71  * deploy: `skaffold deploy -a build.json`
    72  
    73  ### Open Issues/Questions
    74  
    75  **Does this approach make sense for different languages/frameworks?**
    76  
    77  Resolution: __Not Yet Resolved__
    78  
    79  **Should the command look for the pod in all namespaces or shall it require it to be defined since
    80  usually one would deploy to a fresh namespace and execute the tests there explicitly?**
    81  
    82  Resolution: __Not Yet Resolved__
    83  
    84  ## Implementation plan
    85  
    86  1. Add new top-level config key `integrationtest` and test schema validation
    87  2. Add new config keys `podSelector` and `testCommand` to `integrationtest` and test schema validation
    88  3. Add new command `integrationtest`
    89  
    90  
    91  ## Integration test plan
    92  
    93  1. Test handling of different config variations (non-existent, empty, wrong/typo'd podSelector/testCommand)
    94  2. Test proper return values (log output, return code) of `integrationtest` command