github.com/mheon/docker@v0.11.2-0.20150922122814-44f47903a831/docs/project/test-and-docs.md (about)

     1  <!--[metadata]>
     2  +++
     3  title = "Run tests and test documentation"
     4  description = "Describes Docker's testing infrastructure"
     5  keywords = ["make test, make docs, Go tests, gofmt, contributing,  running tests"]
     6  [menu.main]
     7  parent = "smn_develop"
     8  weight=6
     9  +++
    10  <![end-metadata]-->
    11  
    12  # Run tests and test documentation
    13  
    14  Contributing includes testing your changes. If you change the Docker code, you
    15  may need to add a new test or modify an existing one. Your contribution could
    16  even be adding tests to Docker. For this reason, you need to know a little
    17  about Docker's test infrastructure.
    18  
    19  Many contributors contribute documentation only. Or, a contributor makes a code
    20  contribution that changes how Docker behaves and that change needs
    21  documentation. For these reasons, you also need to know how to build, view, and
    22  test the Docker documentation.
    23  
    24  In this section, you run tests in the `dry-run-test` branch of your Docker
    25  fork. If you have followed along in this guide, you already have this branch.
    26  If you don't have this branch, you can create it or simply use another of your
    27  branches.
    28  
    29  ## Understand testing at Docker
    30  
    31  Docker tests use the Go language's test framework. In this framework, files
    32  whose names end in `_test.go` contain test code; you'll find test files like
    33  this throughout the Docker repo. Use these files for inspiration when writing
    34  your own tests. For information on Go's test framework, see <a
    35  href="http://golang.org/pkg/testing/" target="_blank">Go's testing package
    36  documentation</a> and the <a href="http://golang.org/cmd/go/#hdr-Test_packages"
    37  target="_blank">go test help</a>. 
    38  
    39  You are responsible for _unit testing_ your contribution when you add new or
    40  change existing Docker code. A unit test is a piece of code that invokes a
    41  single, small piece of code ( _unit of work_ ) to verify the unit works as
    42  expected.
    43  
    44  Depending on your contribution, you may need to add _integration tests_. These
    45  are tests that combine two or more work units into one component. These work
    46  units each have unit tests and then, together, integration tests that test the
    47  interface between the components. The `integration` and `integration-cli`
    48  directories in the Docker repository contain integration test code.
    49  
    50  Testing is its own specialty. If you aren't familiar with testing techniques,
    51  there is a lot of information available to you on the Web. For now, you should
    52  understand that, the Docker maintainers may ask you to write a new test or
    53  change an existing one.
    54  
    55  ### Run tests on your local host
    56  
    57  Before submitting any code change, you should run the entire Docker test suite.
    58  The `Makefile` contains a target for the entire test suite. The target's name
    59  is simply `test`. The `Makefile` contains several targets for testing:
    60  
    61  <style type="text/css">
    62  .monospaced {font-family: Monaco, Consolas, "Lucida Console", monospace !important;}
    63  </style>
    64  <table>
    65    <tr>
    66      <th>Target</th>
    67      <th>What this target does</th>
    68    </tr>
    69    <tr>
    70      <td class="monospaced">test</td>
    71      <td>Run all the tests.</td>
    72    </tr>
    73    <tr>
    74      <td class="monospaced">test-unit</td>
    75      <td>Run just the unit tests.</td>
    76    </tr>
    77    <tr>
    78      <td class="monospaced">test-integration-cli</td>
    79      <td>Run the test for the integration command line interface.</td>
    80    </tr>
    81    <tr>
    82      <td class="monospaced">test-docker-py</td>
    83      <td>Run the tests for Docker API client.</td>
    84    </tr>
    85  </table>
    86  
    87  Run the entire test suite on your current repository:
    88  
    89  1. Open a terminal on your local host.
    90  
    91  2. Change to the root your Docker repository.
    92  
    93          $ cd docker-fork
    94  
    95  3. Make sure you are in your development branch.
    96  
    97          $ git checkout dry-run-test
    98  
    99  4. Run the `make test` command.
   100  
   101          $ make test
   102  
   103      This command does several things, it creates a container temporarily for
   104      testing. Inside that container, the `make`:
   105  
   106      * creates a new binary
   107      * cross-compiles all the binaries for the various operating systems
   108      * runs all the tests in the system
   109  
   110      It can take approximate one hour to run all the tests. The time depends
   111      on your host performance. The default timeout is 60 minutes, which is
   112      defined in hack/make.sh(${TIMEOUT:=60m}). You can modify the timeout
   113      value on the basis of your host performance. When they complete
   114      successfully, you see the output concludes with something like this:
   115  
   116  
   117          [PASSED]: top - sleep process should be listed in privileged mode
   118          [PASSED]: version - verify that it works and that the output is properly formatted
   119          PASS
   120          coverage: 70.8% of statements
   121          ---> Making bundle: test-docker-py (in bundles/1.5.0-dev/test-docker-py)
   122          +++ exec docker daemon --debug --host unix:///go/src/github.com/docker/docker/bundles/1.5.0-dev/test-docker-py/docker.sock --storage-driver vfs --exec-driver native --pidfile /go/src/github.com/docker/docker/bundles/1.5.0-dev/test-docker-py/docker.pid
   123          .................................................................
   124          ----------------------------------------------------------------------
   125          Ran 65 tests in 89.266s
   126   
   127  
   128  ### Run test targets inside the development container
   129  
   130  If you are working inside a Docker development container, you use the
   131  `hack/make.sh` script to run tests. The `hack/make.sh` script doesn't
   132  have a single target that runs all the tests. Instead, you provide a single
   133  command line with multiple targets that does the same thing.
   134  
   135  Try this now.
   136  
   137  1. Open a terminal and change to the `docker-fork` root.
   138  
   139  2. Start a Docker development image.
   140  
   141      If you are following along with this guide, you should have a
   142      `dry-run-test` image.
   143  
   144          $ docker run --privileged --rm -ti -v `pwd`:/go/src/github.com/docker/docker dry-run-test /bin/bash
   145  
   146  3. Run the tests using the `hack/make.sh` script.
   147  
   148          root@5f8630b873fe:/go/src/github.com/docker/docker# hack/make.sh dynbinary binary cross test-unit test-integration-cli test-docker-py
   149  
   150      The tests run just as they did within your local host.
   151  
   152  
   153  Of course, you can also run a subset of these targets too. For example, to run
   154  just the unit tests:
   155  
   156      root@5f8630b873fe:/go/src/github.com/docker/docker# hack/make.sh dynbinary binary cross test-unit
   157  
   158  Most test targets require that you build these precursor targets first:
   159  `dynbinary binary cross`
   160  
   161  
   162  ## Running individual or multiple named tests 
   163  
   164  We use [gocheck](https://labix.org/gocheck) for our integration-cli tests. 
   165  You can use the `TESTFLAGS` environment variable to run a single test. The
   166  flag's value is passed as arguments to the `go test` command. For example, from
   167  your local host you can run the `TestBuild` test with this command:
   168  
   169      $ TESTFLAGS='-check.f DockerSuite.TestBuild*' make test-integration-cli
   170  
   171  To run the same test inside your Docker development container, you do this:
   172  
   173      root@5f8630b873fe:/go/src/github.com/docker/docker# TESTFLAGS='-check.f TestBuild*' hack/make.sh binary test-integration-cli
   174  
   175  ## Testing the Windows binary against a Linux daemon
   176  
   177  This explains how to test the Windows binary on a Windows machine set up as a
   178  development environment.  The tests will be run against a docker daemon 
   179  running on a remote Linux machine. You'll use  **Git Bash** that came with the 
   180  Git for Windows installation.  **Git Bash**, just as it sounds, allows you to
   181  run a Bash terminal on Windows. 
   182  
   183  1.  If you don't have one open already, start a Git Bash terminal.
   184  
   185  	 ![Git Bash](/project/images/git_bash.png)
   186  
   187  2. Change to the `docker` source directory.
   188  
   189  		$ cd /c/gopath/src/github.com/docker/docker
   190      
   191  3. Set `DOCKER_REMOTE_DAEMON` as follows:
   192  
   193  		$ export DOCKER_REMOTE_DAEMON=1
   194  
   195  4. Set `DOCKER_TEST_HOST` to the `tcp://IP_ADDRESS:2376` value; substitute your
   196  Linux machines actual IP address. For example:
   197  
   198  		$ export DOCKER_TEST_HOST=tcp://213.124.23.200:2376
   199  
   200  5. Make the binary and run the tests:
   201  
   202  		$ hack/make.sh binary test-integration-cli
   203    	
   204     Some tests are skipped on Windows for various reasons. You can see which
   205     tests were skipped by re-running the make and passing in the 
   206     `TESTFLAGS='-test.v'` value. For example 
   207  
   208  		$ TESTFLAGS='-test.v' hack/make.sh binary test-integration-cli
   209  		
   210  	Should you wish to run a single test such as one with the name 
   211  	'TestExample', you can pass in `TESTFLAGS='-check.f TestExample'`. For
   212  	example 
   213  	
   214  		$TESTFLAGS='-check.f TestExample' hack/make.sh binary test-integration-cli
   215          
   216  You can now choose to make changes to the Docker source or the tests. If you
   217  make any changes just run these commands again.
   218  
   219  
   220  ## Build and test the documentation
   221  
   222  The Docker documentation source files are under `docs`. The content is
   223  written using extended Markdown. We use the static generator <a
   224  href="http://www.mkdocs.org/" target="_blank">MkDocs</a> to build Docker's
   225  documentation. Of course, you don't need to install this generator
   226  to build the documentation, it is included with container.
   227  
   228  You should always check your documentation for grammar and spelling. The best
   229  way to do this is with <a href="http://www.hemingwayapp.com/"
   230  target="_blank">an online grammar checker</a>.
   231  
   232  When you change a documentation source file, you should test your change
   233  locally to make sure your content is there and any links work correctly. You
   234  can build the documentation from the local host. The build starts a container
   235  and loads the documentation into a server. As long as this container runs, you
   236  can browse the docs.
   237  
   238  1. In a terminal, change to the root of your `docker-fork` repository.
   239  
   240          $ cd ~/repos/docker-fork
   241  
   242  2. Make sure you are in your feature branch.
   243  
   244          $ git status
   245          On branch dry-run-test
   246          Your branch is up-to-date with 'origin/dry-run-test'.
   247          nothing to commit, working directory clean
   248  
   249  3. Build the documentation.
   250  
   251          $ make docs
   252  
   253      When the build completes, you'll see a final output message similar to the
   254      following:
   255  
   256          Successfully built ee7fe7553123
   257          docker run --rm -it  -e AWS_S3_BUCKET -e NOCACHE -p 8000:8000 "docker-docs:dry-run-test" mkdocs serve
   258          Running at: http://0.0.0.0:8000/
   259          Live reload enabled.
   260          Hold ctrl+c to quit.
   261  
   262  4. Enter the URL in your browser.
   263  
   264      If you are using Docker Machine, replace the default localhost address
   265      (0.0.0.0) with your DOCKERHOST value. You can get this value at any time by
   266      entering `docker-machine ip <machine-name>` at the command line.
   267  
   268  5. Once in the documentation, look for the red notice to verify you are seeing the correct build.
   269  
   270      ![Beta documentation](/project/images/red_notice.png)
   271  
   272  6. Navigate to your new or changed document.
   273  
   274  7. Review both the content and the links.
   275  
   276  8. Return to your terminal and exit out of the running documentation container.
   277  
   278  
   279  ## Where to go next
   280  
   281  Congratulations, you have successfully completed the basics you need to
   282  understand the Docker test framework. In the next steps, you use what you have
   283  learned so far to [contribute to Docker by working on an
   284  issue](/project/make-a-contribution/).