github.com/containers/libpod@v1.9.4-0.20220419124438-4284fd425507/test/README.md (about) 1  2 # Test utils 3 Test utils provide common functions and structs for testing. It includes two structs: 4 * `PodmanTest`: Handle the *podman* command and other global resources like temporary 5 directory. It provides basic methods, like checking podman image and pod status. Test 6 suites should create their owner test *struct* as a composite of `PodmanTest`, and their 7 owner PodmanMakeOptions(). 8 9 * `PodmanSession`: Store execution session data and related *methods*. Such like get command 10 output and so on. It can be used directly in the test suite, only embed it to your owner 11 session struct if you need expend it. 12 13 ## Unittest for test/utils 14 To ensure neither *tests* nor *utils* break, There are unit-tests for each *functions* and 15 *structs* in `test/utils`. When you adding functions or structs to this *package*, please 16 update both unit-tests for it and this documentation. 17 18 ### Run unit test for test/utils 19 Run unit test for test/utils. 20 21 ``` 22 make localunit 23 ``` 24 25 ## Structure of the test utils and test suites 26 The test *utils* package is at the same level of test suites. Each test suites also have their 27 owner common functions and structs stored in `libpod_suite_test.go`. 28 29 # Ginkgo test framework 30 [Ginkgo](https://github.com/onsi/ginkgo) is a BDD testing framework. This allows 31 us to use native Golang to perform our tests and there is a strong affiliation 32 between Ginkgo and the Go test framework. 33 34 ## Installing dependencies 35 The dependencies for integration really consists of three things: 36 * ginkgo binary 37 38 The following instructions assume your GOPATH is ~/go. Adjust as needed for your 39 environment. 40 41 ### Installing ginkgo 42 Build ginkgo and install it under $GOPATH/bin with the following commands: 43 ``` 44 export GOCACHE="$(mktemp -d)" 45 GOPATH=~/go make .install.ginkgo 46 ``` 47 If your PATH does not include $GOPATH/bin, you might consider adding it. 48 49 ``` 50 PATH=$PATH:$GOPATH/bin 51 ``` 52 53 # Integration Tests 54 Test suite for integration test for podman command line. It has its own structs: 55 * `PodmanTestIntegration`: Integration test *struct* as a composite of `PodmanTest`. It 56 set up the global options for *podman* command to ignore the environment influence from 57 different test system. 58 59 * `PodmanSessionIntegration`: This *struct* has it own *methods* for checking command 60 output with given format JSON by using *structs* defined in inspect package. 61 62 ## Running the integration tests 63 You can run the entire suite of integration tests with the following command: 64 65 ``` 66 GOPATH=~/go ginkgo -v test/e2e/. 67 ``` 68 69 Note the trailing period on the command above. Also, **-v** invokes verbose mode. That 70 switch is optional. 71 72 73 ### Running a single file of integration tests 74 You can run a single file of integration tests using the go test command: 75 76 ``` 77 GOPATH=~/go go test -v test/e2e/libpod_suite_test.go test/e2e/common_test.go test/e2e/config.go test/e2e/config_amd64.go test/e2e/your_test.go 78 ``` 79 80 ### Running a single integration test 81 Before running the test suite, you have to declare which test you want run in the test 82 file itself. Consider the following actual test: 83 ``` 84 It("podman inspect bogus pod", func() { 85 session := podmanTest.Podman([]string{"pod", "inspect", "foobar"}) 86 session.WaitWithDefaultTimeout() 87 Expect(session.ExitCode()).To(Not(Equal(0))) 88 }) 89 ``` 90 91 To mark this as the test you want run, you simply change the *It* description to *FIt*. Please note how 92 both the `F` and `I` are capitalized. 93 94 You can run a single integration test using the same command we used to run all the tests in a single 95 file. 96 97 ``` 98 GOPATH=~/go go test -v test/e2e/libpod_suite_test.go test/e2e/common_test.go test/e2e/config.go test/e2e/config_amd64.go test/e2e/your_test.go 99 ``` 100 101 *Note*: Be sure you remove the `F` from the tests before committing your changes or you will skip all tests 102 in that file except the one with the `FIt` denotation. 103 104 105 ### Run tests in a container 106 In case you have issue running the tests locally on your machine, you can run 107 them in a container: 108 ``` 109 make shell 110 ``` 111 112 This will run a container and give you a shell and you can follow the instructions above. 113 114 # System tests 115 System tests are used for testing the *podman* CLI in the context of a complete system. It 116 requires that *podman*, all dependencies, and configurations are in place. The intention of 117 system testing is to match as closely as possible with real-world user/developer use-cases 118 and environments. The orchestration of the environments and tests is left to external 119 tooling. 120 121 System tests use Bash Automated Testing System (`bats`) as a testing framework. 122 Install it via your package manager or get latest stable version 123 [directly from the repository](https://github.com/bats-core/bats-core), e.g.: 124 125 ``` 126 mkdir -p ~/tools/bats 127 git clone --single-branch --branch v1.1.0 https://github.com/bats-core/bats-core.git ~/tools/bats 128 ``` 129 130 Make sure that `bats` binary (`bin/bats` in the repository) is in your `PATH`, if not - add it: 131 132 ``` 133 PATH=$PATH:~/tools/bats/bin 134 ``` 135 136 ## Running system tests 137 When `bats` is installed and is in your `PATH`, you can run the test suite with following command: 138 139 ``` 140 make localsystem 141 ``` 142 143 ## Contributing to system tests 144 145 Please see [the TODO list of needed workflows/tests](system/TODO.md).