github.com/inflatablewoman/deis@v1.0.1-0.20141111034523-a4511c46a6ce/docs/contributing/testing.rst (about)

     1  :title: Testing Deis
     2  :description: How to test Deis
     3  
     4  .. _testing:
     5  
     6  Testing Deis
     7  ============
     8  
     9  Deis is a distributed system with many moving parts, which makes it of paramount
    10  importance to test every change thoroughly.
    11  
    12  Deis is also a set of components that correspond to directories in the source
    13  code repository. Most components are Docker containers, two are command-line
    14  clients, and one contains the documentation. Components have source-code level
    15  `unit tests`_ and black-box type `functional tests`_. `integration tests`_
    16  verify the behavior of the components together as a system.
    17  
    18  GitHub pull requests for Deis are tested automatically by a Jenkins
    19  `continuous integration`_ (CI) system at http://ci.deis.io. Contributors should
    20  run the same tests locally before proposing any changes to the Deis codebase.
    21  
    22  
    23  Set Up the Environment
    24  ----------------------
    25  
    26  To run all tests, you will need:
    27  
    28  - Vagrant 1.6.5 or later
    29  - VirtualBox 4.3 or later
    30  - Docker 1.3.0
    31  - PostgreSQL server
    32  
    33  The tests assume that you have Deis' `source code`_ in your ``$GOPATH``:
    34  
    35  .. code-block:: console
    36  
    37      $ go get -u -v github.com/deis/deis
    38      $ cd $GOPATH/src/github.com/deis/deis
    39  
    40  Start a Docker Registry
    41  ^^^^^^^^^^^^^^^^^^^^^^^
    42  
    43  Deis' functional tests build Docker images and test them locally. The images are
    44  then pushed to a `Docker registry`_ so that integration tests can test them as
    45  binary artifacts--just as a real-world provisioning of Deis pulls images from
    46  the Docker Hub.
    47  
    48  If you don't have a Docker registry already accessible for your testing or for
    49  continuous deployment, start one locally:
    50  
    51  .. code-block:: console
    52  
    53      $ make dev-registry
    54      registry
    55  
    56      To use local boot2docker registry for Deis development:
    57          export DEV_REGISTRY=192.168.59.103:5000
    58  
    59  
    60  Run the Tests
    61  -------------
    62  
    63  The unit and functional tests for each component are in their respective
    64  directories. The integration tests, scripts, and supporting go packages are in
    65  the ``tests/`` directory in the project root.
    66  
    67  Scripts in the ``tests/bin`` directory are the best place to start. These test
    68  individual pieces of Deis, then bring up a Vagrant cluster and test all of them
    69  as a system. They call ``tests/bin/test-setup.sh`` to test for important
    70  environment variables and will exit with a helpful message if any are missing.
    71  
    72  test-integration.sh
    73  ^^^^^^^^^^^^^^^^^^^
    74  
    75  - runs documentation tests
    76  - builds Docker images tagged with ``$BUILD_TAG``
    77  - runs unit and functional tests
    78  - creates a 3-node Vagrant CoreOS cluster
    79  - pushes the Docker images to a registry
    80  - provisions the cluster for Deis with the registry images
    81  - runs all integration tests
    82  - takes roughly an hour
    83  
    84  .. code-block:: console
    85  
    86      $ ./tests/bin/test-integration.sh
    87  
    88      >>> Preparing test environment <<<
    89  
    90      DEIS_ROOT=/Users/matt/Projects/src/github.com/deis/deis
    91      DEIS_TEST_APP=example-go
    92      ...
    93      >>> Running integration suite <<<
    94  
    95      make -C tests/ test-full
    96      ...
    97      >>> Test run complete <<<
    98  
    99  test-smoke.sh
   100  ^^^^^^^^^^^^^
   101  
   102  - runs documentation tests
   103  - builds Docker images tagged with ``$BUILD_TAG``
   104  - runs unit and functional tests
   105  - creates a 3-node Vagrant CoreOS cluster
   106  - pushes the Docker images to a registry
   107  - provisions the cluster for Deis with the registry images
   108  - runs a "smoke test" that pushes and scales an app
   109  - takes roughly 45 minutes
   110  
   111  test-latest.sh
   112  ^^^^^^^^^^^^^^
   113  
   114  - installs the latest ``deis`` and ``deisctl`` client releases
   115  - creates a 3-node Vagrant CoreOS cluster
   116  - provisions the cluster for Deis with latest release images
   117  - runs a "smoke test" that pushes and scales an app
   118  - takes roughly 30 minutes
   119  
   120  Run Specific Tests
   121  ^^^^^^^^^^^^^^^^^^
   122  
   123  Run the tests for a single component this way:
   124  
   125  .. code-block:: console
   126  
   127      $ make -C logger test             # unit + functional
   128      $ make -C controller test-unit
   129      $ make -C router test-functional
   130  
   131  
   132  Customize Test Runs
   133  -------------------
   134  
   135  The file ``tests/bin/test-setup.sh`` is the best reference to environment
   136  variables that can affect the tests' behavior. Here are some important ones:
   137  
   138  - ``HOST_IPADDR`` - address on which Docker containers can communicate for the
   139    functional tests, probably the host's IP or the one assigned to boot2docker_.
   140  - ``DEIS_TEST_APP`` - name of the `Deis example app`_ to use, which is cloned
   141    from GitHub (default: ``example-go``)
   142  - ``DEIS_TEST_AUTH_KEY`` - SSH key used to register with the Deis controller
   143    (default: ``~/.ssh/deis``)
   144  - ``DEIS_TEST_SSH_KEY`` - SSH key used to login to the controller machine
   145    (default: ``~/.vagrant.d/insecure_private_key``)
   146  - ``DEIS_TEST_DOMAIN`` - the domain to use for testing
   147    (default: ``local3.deisapp.com``)
   148  
   149  
   150  .. _`unit tests`: http://en.wikipedia.org/wiki/Unit_testing
   151  .. _`functional tests`: http://en.wikipedia.org/wiki/Functional_testing
   152  .. _`integration tests`: http://en.wikipedia.org/wiki/Integration_testing
   153  .. _`continuous integration`: http://en.wikipedia.org/wiki/Continuous_integration
   154  .. _boot2docker: http://boot2docker.io/
   155  .. _`source code`: https://github.com/deis/deis
   156  .. _`Docker registry`: https://github.com/docker/docker-registry
   157  .. _`Deis example app`: https://github.com/deis?query=example-