github.com/demonoid81/moby@v0.0.0-20200517203328-62dd8e17c460/docs/contributing/test.md (about)

     1  ### Run tests
     2  
     3  Contributing includes testing your changes. If you change the Moby code, you
     4  may need to add a new test or modify an existing test. Your contribution could
     5  even be adding tests to Moby. For this reason, you need to know a little
     6  about Moby's test infrastructure.
     7  
     8  This section describes tests you can run in the `dry-run-test` branch of your Docker
     9  fork. If you have followed along in this guide, you already have this branch.
    10  If you don't have this branch, you can create it or simply use another of your
    11  branches.
    12  
    13  ## Understand how to test Moby
    14  
    15  Moby tests use the Go language's test framework. In this framework, files
    16  whose names end in `_test.go` contain test code; you'll find test files like
    17  this throughout the Moby repo. Use these files for inspiration when writing
    18  your own tests. For information on Go's test framework, see <a
    19  href="http://golang.org/pkg/testing/" target="_blank">Go's testing package
    20  documentation</a> and the <a href="http://golang.org/cmd/go/#hdr-Test_packages"
    21  target="_blank">go test help</a>.
    22  
    23  You are responsible for _unit testing_ your contribution when you add new or
    24  change existing Moby code. A unit test is a piece of code that invokes a
    25  single, small piece of code (_unit of work_) to verify the unit works as
    26  expected.
    27  
    28  Depending on your contribution, you may need to add _integration tests_. These
    29  are tests that combine two or more work units into one component. These work
    30  units each have unit tests and then, together, integration tests that test the
    31  interface between the components. The `integration` and `integration-cli`
    32  directories in the Docker repository contain integration test code.  Note that
    33  `integration-cli` tests are now deprecated in the Moby project, and new tests
    34  cannot be added to this suite - add `integration` tests instead using the API
    35  client.
    36  
    37  Testing is its own specialty. If you aren't familiar with testing techniques,
    38  there is a lot of information available to you on the Web. For now, you should
    39  understand that, the Docker maintainers may ask you to write a new test or
    40  change an existing one.
    41  
    42  ## Run tests on your local host
    43  
    44  Before submitting a pull request with a code change, you should run the entire
    45  Moby Engine test suite. The `Makefile` contains a target for the entire test
    46  suite, named `test`. Also, it contains several targets for
    47  testing:
    48  
    49  | Target                 | What this target does                          |
    50  | ---------------------- | ---------------------------------------------- |
    51  | `test`                 | Run the unit, integration, and docker-py tests |
    52  | `test-unit`            | Run just the unit tests                        |
    53  | `test-integration`     | Run the integration tests                      |
    54  | `test-docker-py`       | Run the tests for the Docker API client        |
    55  
    56  Running the entire test suite on your current repository can take over half an
    57  hour. To run the test suite, do the following:
    58  
    59  1.  Open a terminal on your local host.
    60  
    61  2.  Change to the root of your Docker repository.
    62  
    63      ```bash
    64      $ cd moby-fork
    65      ```
    66  
    67  3.  Make sure you are in your development branch.
    68  
    69      ```bash
    70      $ git checkout dry-run-test
    71      ```
    72  
    73  4.  Run the `make test` command.
    74  
    75      ```bash
    76      $ make test
    77      ```
    78  
    79      This command does several things, it creates a container temporarily for
    80      testing. Inside that container, the `make`:
    81  
    82      * creates a new binary
    83      * cross-compiles all the binaries for the various operating systems
    84      * runs all the tests in the system
    85  
    86      It can take approximate one hour to run all the tests. The time depends
    87      on your host performance. The default timeout is 60 minutes, which is
    88      defined in `hack/make.sh` (`${TIMEOUT:=60m}`). You can modify the timeout
    89      value on the basis of your host performance. When they complete
    90      successfully, you see the output concludes with something like this:
    91  
    92      ```none
    93      Ran 68 tests in 79.135s
    94      ```
    95  
    96  ## Run targets inside a development container
    97  
    98  If you are working inside a development container, you use the
    99  `hack/test/unit` script to run unit-tests, and `hack/make.sh` script to run
   100  integration and other tests. The `hack/make.sh` script doesn't
   101  have a single target that runs all the tests. Instead, you provide a single
   102  command line with multiple targets that does the same thing.
   103  
   104  Try this now.
   105  
   106  1.  Open a terminal and change to the `moby-fork` root.
   107  
   108  2.  Start a Moby development image.
   109  
   110      If you are following along with this guide, you should have a
   111      `dry-run-test` image.
   112  
   113      ```bash
   114      $ docker run --privileged --rm -ti -v `pwd`:/go/src/github.com/demonoid81/moby dry-run-test /bin/bash
   115      ```
   116  
   117  3.  Run the unit tests using the `hack/test/unit` script.
   118  
   119      ```bash
   120      # hack/test/unit
   121      ```
   122  
   123  4.  Run the tests using the `hack/make.sh` script.
   124  
   125      ```bash
   126      # hack/make.sh dynbinary binary cross test-integration test-docker-py
   127      ```
   128  
   129      The tests run just as they did within your local host.
   130  
   131      Of course, you can also run a subset of these targets too. For example, to run
   132      just the integration tests:
   133  
   134      ```bash
   135      # hack/make.sh dynbinary binary cross test-integration
   136      ```
   137  
   138      Most test targets require that you build these precursor targets first:
   139      `dynbinary binary cross`
   140  
   141  
   142  ## Run unit tests
   143  
   144  We use golang standard [testing](https://golang.org/pkg/testing/)
   145  package or [gocheck](https://labix.org/gocheck) for our unit tests.
   146  
   147  You can use the `TESTDIRS` environment variable to run unit tests for
   148  a single package.
   149  
   150  ```bash
   151  $ TESTDIRS='github.com/demonoid81/moby/opts' make test-unit
   152  ```
   153  
   154  You can also use the `TESTFLAGS` environment variable to run a single test. The
   155  flag's value is passed as arguments to the `go test` command. For example, from
   156  your local host you can run the `TestValidateIPAddress` test with this command:
   157  
   158  ```bash
   159  $ TESTFLAGS='-test.run ^TestValidateIPAddress$' make test-unit
   160  ```
   161  
   162  On unit tests, it's better to use `TESTFLAGS` in combination with
   163  `TESTDIRS` to make it quicker to run a specific test.
   164  
   165  ```bash
   166  $ TESTDIRS='github.com/demonoid81/moby/opts' TESTFLAGS='-test.run ^TestValidateIPAddress$' make test-unit
   167  ```
   168  
   169  ## Run integration tests
   170  
   171  We use [gocheck](https://labix.org/gocheck) for our integration-cli tests.
   172  You can use the `TESTFLAGS` environment variable to run a single test. The
   173  flag's value is passed as arguments to the `go test` command. For example, from
   174  your local host you can run the `TestBuild` test with this command:
   175  
   176  ```bash
   177  $ TESTFLAGS='-test.run TestDockerSuite/TestBuild*' make test-integration
   178  ```
   179  
   180  To run the same test inside your Docker development container, you do this:
   181  
   182  ```bash
   183  # TESTFLAGS='-test.run TestDockerSuite/TestBuild*' hack/make.sh binary test-integration
   184  ```
   185  
   186  ## Test the Windows binary against a Linux daemon
   187  
   188  This explains how to test the Windows binary on a Windows machine set up as a
   189  development environment. The tests will be run against a daemon
   190  running on a remote Linux machine. You'll use **Git Bash** that came with the
   191  Git for Windows installation. **Git Bash**, just as it sounds, allows you to
   192  run a Bash terminal on Windows.
   193  
   194  1.  If you don't have one open already, start a Git Bash terminal.
   195  
   196      ![Git Bash](images/git_bash.png)
   197  
   198  2.  Change to the `moby` source directory.
   199  
   200      ```bash
   201      $ cd /c/gopath/src/github.com/demonoid81/moby
   202      ```
   203  
   204  3.  Set `DOCKER_REMOTE_DAEMON` as follows:
   205  
   206      ```bash
   207      $ export DOCKER_REMOTE_DAEMON=1
   208      ```
   209  
   210  4.  Set `DOCKER_TEST_HOST` to the `tcp://IP_ADDRESS:2376` value; substitute your
   211      Linux machines actual IP address. For example:
   212  
   213      ```bash
   214      $ export DOCKER_TEST_HOST=tcp://213.124.23.200:2376
   215      ```
   216  
   217  5.  Make the binary and run the tests:
   218  
   219      ```bash
   220      $ hack/make.sh binary test-integration
   221      ```
   222      Some tests are skipped on Windows for various reasons. You can see which
   223      tests were skipped by re-running the make and passing in the
   224     `TESTFLAGS='-test.v'` value. For example
   225  
   226      ```bash
   227      $ TESTFLAGS='-test.v' hack/make.sh binary test-integration
   228      ```
   229  
   230      Should you wish to run a single test such as one with the name
   231      'TestExample', you can pass in `TESTFLAGS='-test.run /TestExample'`. For
   232      example
   233  
   234      ```bash
   235      $ TESTFLAGS='-test.run /TestExample' hack/make.sh binary test-integration
   236      ```
   237  
   238  You can now choose to make changes to the Moby source or the tests. If you
   239  make any changes, just run these commands again.
   240  
   241  ## [Public CI infrastructure](ci.docker.com/public)
   242  
   243  The current infrastructure is maintained here: [Moby ci job](https://ci.docker.com/public/job/moby).  The Jenkins infrastructure is for the Moby project is maintained and
   244  managed by Docker Inc.  All contributions against the Jenkinsfile are
   245  appreciated and welcomed!  However we might not be able to fully provide the
   246  infrastructure to test against various architectures in our CI pipelines.  All
   247  jobs can be triggered and re-ran by the Moby maintainers
   248  
   249  ## Where to go next
   250  
   251  Congratulations, you have successfully completed the basics you need to
   252  understand the Moby test framework.