github.com/devcamcar/cli@v0.0.0-20181107134215-706a05759d18/test/README.md (about)

     1  # CLI End-To-End tests
     2  
     3  
     4  These are end-to-end CLI tests that operate on a compiled CLI binary  - use the [Test Harness](../testharness/harness.go) for more details:
     5  
     6  
     7  ## Example:
     8  
     9  The following test makes a function directory (`h.Mkdir`)  runs a few Fn commands `h.Fn(...)` and verifies the results on the returned commands `.AssertSuccess`
    10  
    11  ```
    12  func TestPythonCall(t *testing.T) {
    13  	t.Parallel()
    14  
    15  	h := testharness.Create(t)
    16  	defer h.Cleanup()
    17  
    18  	appName := h.NewAppName()
    19  	funcName := h.NewFuncName(appName)
    20  
    21  	h.MkDir(funcName)
    22  	h.Cd(funcName)
    23  	h.Fn("init", "--name", funcName, "--runtime", "python3.6").AssertSuccess()
    24  	appName := h.NewAppName()
    25  	h.Fn("deploy", "--local", appName).AssertSuccess()
    26  	h.Fn("invoke", appName, funcName).AssertSuccess()
    27  	h.FnWithInput(`{"name": "John"}`, "call", appName, funcName).AssertSuccess()
    28  
    29  }
    30  ```
    31  
    32  The test harness runs a specified CLI  binary(either "../fn" or "TEST_CLI_BINARY" from env) in a dynamically created test directory, $HOME is also mocked out to allow testing of configuration files.
    33  
    34  ## Hints for writing good tests:
    35  
    36  * Don't write lots of tests for features: CLI end-to-end tests are primarily there to detect regressions in users' expectations about command behaviour - they are somewhat expensive (typically some seconds per test) so shouldn't be used as the only means to test changes - a good rule of thumb is to test the use cases that you would demonstrate to somebody when showing them the feature.
    37  * Don't be spammy : You shouldn't log excessively in tests as this will impact diagnosability when a test fails.  Instead, always log the `CmdResult` you got from the last command that failed - this should include enough diagnostic history to work out what went wrong (including previous commands)
    38  * Write parallelizable tests: Tests are slow so sequencing them will make the test package slow - the harness includes tools to help make tests isolated (e.g. any app names created with `h.NewFuncName()` will be deleted after a test is done )  - remember to defer `h.Cleanup()` to ensure test state is cleaned up
    39  * Watch out for the Environment: The CLI package will pass on the surrounding environment to the CLI when its called - (primarily to allow easily overriding local  configuration like FN_API_URL and other env vars)