github.com/mheon/docker@v0.11.2-0.20150922122814-44f47903a831/docs/project/test-and-docs.md (about) 1 <!--[metadata]> 2 +++ 3 title = "Run tests and test documentation" 4 description = "Describes Docker's testing infrastructure" 5 keywords = ["make test, make docs, Go tests, gofmt, contributing, running tests"] 6 [menu.main] 7 parent = "smn_develop" 8 weight=6 9 +++ 10 <![end-metadata]--> 11 12 # Run tests and test documentation 13 14 Contributing includes testing your changes. If you change the Docker code, you 15 may need to add a new test or modify an existing one. Your contribution could 16 even be adding tests to Docker. For this reason, you need to know a little 17 about Docker's test infrastructure. 18 19 Many contributors contribute documentation only. Or, a contributor makes a code 20 contribution that changes how Docker behaves and that change needs 21 documentation. For these reasons, you also need to know how to build, view, and 22 test the Docker documentation. 23 24 In this section, you run tests in the `dry-run-test` branch of your Docker 25 fork. If you have followed along in this guide, you already have this branch. 26 If you don't have this branch, you can create it or simply use another of your 27 branches. 28 29 ## Understand testing at Docker 30 31 Docker tests use the Go language's test framework. In this framework, files 32 whose names end in `_test.go` contain test code; you'll find test files like 33 this throughout the Docker repo. Use these files for inspiration when writing 34 your own tests. For information on Go's test framework, see <a 35 href="http://golang.org/pkg/testing/" target="_blank">Go's testing package 36 documentation</a> and the <a href="http://golang.org/cmd/go/#hdr-Test_packages" 37 target="_blank">go test help</a>. 38 39 You are responsible for _unit testing_ your contribution when you add new or 40 change existing Docker code. A unit test is a piece of code that invokes a 41 single, small piece of code ( _unit of work_ ) to verify the unit works as 42 expected. 43 44 Depending on your contribution, you may need to add _integration tests_. These 45 are tests that combine two or more work units into one component. These work 46 units each have unit tests and then, together, integration tests that test the 47 interface between the components. The `integration` and `integration-cli` 48 directories in the Docker repository contain integration test code. 49 50 Testing is its own specialty. If you aren't familiar with testing techniques, 51 there is a lot of information available to you on the Web. For now, you should 52 understand that, the Docker maintainers may ask you to write a new test or 53 change an existing one. 54 55 ### Run tests on your local host 56 57 Before submitting any code change, you should run the entire Docker test suite. 58 The `Makefile` contains a target for the entire test suite. The target's name 59 is simply `test`. The `Makefile` contains several targets for testing: 60 61 <style type="text/css"> 62 .monospaced {font-family: Monaco, Consolas, "Lucida Console", monospace !important;} 63 </style> 64 <table> 65 <tr> 66 <th>Target</th> 67 <th>What this target does</th> 68 </tr> 69 <tr> 70 <td class="monospaced">test</td> 71 <td>Run all the tests.</td> 72 </tr> 73 <tr> 74 <td class="monospaced">test-unit</td> 75 <td>Run just the unit tests.</td> 76 </tr> 77 <tr> 78 <td class="monospaced">test-integration-cli</td> 79 <td>Run the test for the integration command line interface.</td> 80 </tr> 81 <tr> 82 <td class="monospaced">test-docker-py</td> 83 <td>Run the tests for Docker API client.</td> 84 </tr> 85 </table> 86 87 Run the entire test suite on your current repository: 88 89 1. Open a terminal on your local host. 90 91 2. Change to the root your Docker repository. 92 93 $ cd docker-fork 94 95 3. Make sure you are in your development branch. 96 97 $ git checkout dry-run-test 98 99 4. Run the `make test` command. 100 101 $ make test 102 103 This command does several things, it creates a container temporarily for 104 testing. Inside that container, the `make`: 105 106 * creates a new binary 107 * cross-compiles all the binaries for the various operating systems 108 * runs all the tests in the system 109 110 It can take approximate one hour to run all the tests. The time depends 111 on your host performance. The default timeout is 60 minutes, which is 112 defined in hack/make.sh(${TIMEOUT:=60m}). You can modify the timeout 113 value on the basis of your host performance. When they complete 114 successfully, you see the output concludes with something like this: 115 116 117 [PASSED]: top - sleep process should be listed in privileged mode 118 [PASSED]: version - verify that it works and that the output is properly formatted 119 PASS 120 coverage: 70.8% of statements 121 ---> Making bundle: test-docker-py (in bundles/1.5.0-dev/test-docker-py) 122 +++ exec docker daemon --debug --host unix:///go/src/github.com/docker/docker/bundles/1.5.0-dev/test-docker-py/docker.sock --storage-driver vfs --exec-driver native --pidfile /go/src/github.com/docker/docker/bundles/1.5.0-dev/test-docker-py/docker.pid 123 ................................................................. 124 ---------------------------------------------------------------------- 125 Ran 65 tests in 89.266s 126 127 128 ### Run test targets inside the development container 129 130 If you are working inside a Docker development container, you use the 131 `hack/make.sh` script to run tests. The `hack/make.sh` script doesn't 132 have a single target that runs all the tests. Instead, you provide a single 133 command line with multiple targets that does the same thing. 134 135 Try this now. 136 137 1. Open a terminal and change to the `docker-fork` root. 138 139 2. Start a Docker development image. 140 141 If you are following along with this guide, you should have a 142 `dry-run-test` image. 143 144 $ docker run --privileged --rm -ti -v `pwd`:/go/src/github.com/docker/docker dry-run-test /bin/bash 145 146 3. Run the tests using the `hack/make.sh` script. 147 148 root@5f8630b873fe:/go/src/github.com/docker/docker# hack/make.sh dynbinary binary cross test-unit test-integration-cli test-docker-py 149 150 The tests run just as they did within your local host. 151 152 153 Of course, you can also run a subset of these targets too. For example, to run 154 just the unit tests: 155 156 root@5f8630b873fe:/go/src/github.com/docker/docker# hack/make.sh dynbinary binary cross test-unit 157 158 Most test targets require that you build these precursor targets first: 159 `dynbinary binary cross` 160 161 162 ## Running individual or multiple named tests 163 164 We use [gocheck](https://labix.org/gocheck) for our integration-cli tests. 165 You can use the `TESTFLAGS` environment variable to run a single test. The 166 flag's value is passed as arguments to the `go test` command. For example, from 167 your local host you can run the `TestBuild` test with this command: 168 169 $ TESTFLAGS='-check.f DockerSuite.TestBuild*' make test-integration-cli 170 171 To run the same test inside your Docker development container, you do this: 172 173 root@5f8630b873fe:/go/src/github.com/docker/docker# TESTFLAGS='-check.f TestBuild*' hack/make.sh binary test-integration-cli 174 175 ## Testing the Windows binary against a Linux daemon 176 177 This explains how to test the Windows binary on a Windows machine set up as a 178 development environment. The tests will be run against a docker daemon 179 running on a remote Linux machine. You'll use **Git Bash** that came with the 180 Git for Windows installation. **Git Bash**, just as it sounds, allows you to 181 run a Bash terminal on Windows. 182 183 1. If you don't have one open already, start a Git Bash terminal. 184 185  186 187 2. Change to the `docker` source directory. 188 189 $ cd /c/gopath/src/github.com/docker/docker 190 191 3. Set `DOCKER_REMOTE_DAEMON` as follows: 192 193 $ export DOCKER_REMOTE_DAEMON=1 194 195 4. Set `DOCKER_TEST_HOST` to the `tcp://IP_ADDRESS:2376` value; substitute your 196 Linux machines actual IP address. For example: 197 198 $ export DOCKER_TEST_HOST=tcp://213.124.23.200:2376 199 200 5. Make the binary and run the tests: 201 202 $ hack/make.sh binary test-integration-cli 203 204 Some tests are skipped on Windows for various reasons. You can see which 205 tests were skipped by re-running the make and passing in the 206 `TESTFLAGS='-test.v'` value. For example 207 208 $ TESTFLAGS='-test.v' hack/make.sh binary test-integration-cli 209 210 Should you wish to run a single test such as one with the name 211 'TestExample', you can pass in `TESTFLAGS='-check.f TestExample'`. For 212 example 213 214 $TESTFLAGS='-check.f TestExample' hack/make.sh binary test-integration-cli 215 216 You can now choose to make changes to the Docker source or the tests. If you 217 make any changes just run these commands again. 218 219 220 ## Build and test the documentation 221 222 The Docker documentation source files are under `docs`. The content is 223 written using extended Markdown. We use the static generator <a 224 href="http://www.mkdocs.org/" target="_blank">MkDocs</a> to build Docker's 225 documentation. Of course, you don't need to install this generator 226 to build the documentation, it is included with container. 227 228 You should always check your documentation for grammar and spelling. The best 229 way to do this is with <a href="http://www.hemingwayapp.com/" 230 target="_blank">an online grammar checker</a>. 231 232 When you change a documentation source file, you should test your change 233 locally to make sure your content is there and any links work correctly. You 234 can build the documentation from the local host. The build starts a container 235 and loads the documentation into a server. As long as this container runs, you 236 can browse the docs. 237 238 1. In a terminal, change to the root of your `docker-fork` repository. 239 240 $ cd ~/repos/docker-fork 241 242 2. Make sure you are in your feature branch. 243 244 $ git status 245 On branch dry-run-test 246 Your branch is up-to-date with 'origin/dry-run-test'. 247 nothing to commit, working directory clean 248 249 3. Build the documentation. 250 251 $ make docs 252 253 When the build completes, you'll see a final output message similar to the 254 following: 255 256 Successfully built ee7fe7553123 257 docker run --rm -it -e AWS_S3_BUCKET -e NOCACHE -p 8000:8000 "docker-docs:dry-run-test" mkdocs serve 258 Running at: http://0.0.0.0:8000/ 259 Live reload enabled. 260 Hold ctrl+c to quit. 261 262 4. Enter the URL in your browser. 263 264 If you are using Docker Machine, replace the default localhost address 265 (0.0.0.0) with your DOCKERHOST value. You can get this value at any time by 266 entering `docker-machine ip <machine-name>` at the command line. 267 268 5. Once in the documentation, look for the red notice to verify you are seeing the correct build. 269 270  271 272 6. Navigate to your new or changed document. 273 274 7. Review both the content and the links. 275 276 8. Return to your terminal and exit out of the running documentation container. 277 278 279 ## Where to go next 280 281 Congratulations, you have successfully completed the basics you need to 282 understand the Docker test framework. In the next steps, you use what you have 283 learned so far to [contribute to Docker by working on an 284 issue](/project/make-a-contribution/).