github.com/dmvolod/operator-sdk@v0.8.2/doc/dev/testing/running-the-tests.md (about)

     1  # Running the SDK Tests
     2  
     3  The operator-sdk is automatically tested with a variety of tests anytime
     4  a pull request is made. The E2E tests ensure that the operator-sdk acts as intended by
     5  simulating how a typical user might use the SDK. The automated tests test each PR and run in
     6  Travis CI, and Travis CI has a couple of features to simplify the E2E tests that we run. For
     7  a more in depth description of the tests that Travis runs, please read the [Travis Build][travis] doc.
     8  This doc will talk about how to run the tests locally as well.
     9  
    10  ## Running the Tests Locally
    11  
    12  To run the tests locally, the tests either need access to a remote Kubernetes cluster or a
    13  local Kubernetes instance running on the machine.
    14  
    15  ### Remote Kubernetes Instance
    16  
    17  To run the tests on a remote cluster, the tests need access to a remote Kubernetes cluster
    18  running Kubernetes 1.11.3 or higher as well as a docker image repo to push the operator image to,
    19  such as [`quay.io`][quay]. Your kubeconfig must be located at `$HOME/.kube/config` and certain
    20  tests will not run on remote clusters. See [Running the Tests](#running-the-tests) for more details.
    21  
    22  ### Local OpenShift Cluster
    23  
    24  One way to run the tests is with an OpenShift 3.11 cluster and `oc cluster up` on a Linux system.
    25  
    26  For the first run configuration, either you can either run the `hack/ci/setup-openshift.sh` script, or download
    27  and install the [oc binary][oc-binary] and run these commands:
    28  
    29  ```sh
    30  $ sudo service docker stop
    31  $ sudo sed -i 's/DOCKER_OPTS=\"/DOCKER_OPTS=\"--insecure-registry 172.30.0.0\/16 /' /etc/default/docker
    32  $ sudo service docker start
    33  ```
    34  
    35  Depending on the system you are on, you may also need to configure your firewall before running the cluster.
    36  Refer to the [official docs][oc-docs] for more information.
    37  
    38  After the initial configuration, you can use this command to start the cluster and login as admin:
    39  
    40  ```sh
    41  $ oc cluster up --base-dir=$HOME/oscluster
    42  $ oc login -u system:admin
    43  ```
    44  
    45  You can use this command to stop the cluster:
    46  
    47  ```sh
    48  $ oc cluster down
    49  ```
    50  
    51  To fully delete the cluster, you must run these commands:
    52  
    53  ```sh
    54  $ oc cluster down
    55  $ sudo umount $(grep $HOME/oscluster /proc/mounts | cut -f2 -d" " | sort -r)
    56  $ sudo rm -rf $HOME/oscluster
    57  ```
    58  
    59  **NOTE**: Starting with openshift 4.0, the `oc cluster up` command will not be available. This document
    60  and the tests will be updated in the future to support openshift 4.0.
    61  
    62  
    63  ### Local Minikube
    64  
    65  Another option for testing is using minikube. This is not advised as it uses vanilla Kubernetes, which has less
    66  strict security and may allow some tests to pass when they would not under openshift. Minikube is faster than
    67  openshift and uses less RAM though. To start the minikube cluster, download and install the proper [binary][minikube-binary]
    68  for your system and run these commands.
    69  
    70  ```sh
    71  # The latest version of minikube at the time of writing (v0.31.0) defaults to k8s v1.10.0, so we must explicitly specify the latest k8s v1.11
    72  $ minikube start --kubernetes-version v1.11.6
    73  $ eval $(minikube docker-env)
    74  ```
    75  
    76  ## Running the tests
    77  
    78  All the tests are run through the [`Makefile`][makefile]. This is a brief description of all makefile test instructions:
    79  
    80  - `test` - Installs `operator-sdk` by running `dep ensure` and `make install`, and then runs all tests. This is intended as a full test for developers.
    81  - `test/ci-go` - Runs all the tests that the Go job runs in CI (`sanity`, `unit`, `subcommand`, `e2e/go`).
    82  - `test/ci-ansible` - Runs all the tests that the Ansible job runs in CI (`e2e/ansible`).
    83  - `test/ci-helm` - Runs all the tests that the Helm job runs in CI (`e2e/helm`).
    84  - `test/sanity` - Runs sanity checks.
    85  - `test/unit` - Runs unit tests.
    86  - `test/subcommand` - Runs subcommand tests.
    87  - `test/e2e` - Runs all E2E tests (`e2e/go`, `e2e/ansible`, and `e2e/helm`).
    88  - `test/e2e/go` - Runs the go E2E test.
    89  - `test/e2e/ansible` - Runs the ansible E2E test.
    90  - `test/e2e/helm` - Runs the helm E2E test.
    91  - `test/markdown` - Runs the markdown checks
    92  
    93  For more info on what these tests actually do, please see the [Travis Build][travis] doc.
    94  
    95  Some of the tests will run using the kube config in `$HOME/.kube/config` (others may check the `KUBECONFIG` env var first)
    96  and the operator images will be built and stored in you local docker registry.
    97  
    98  **NOTE**: Some of these tests, specifically the ansible (`test/e2e/ansible` and `test/ci-ansible`), helm
    99  (`test/e2e/helm` and `test/ci-helm`), and CI Go (`test/e2e/ci-go`) tests, only work when the cluster shares the local docker
   100  registry, as is the case with `oc cluster up` and `minikube` after running `eval $(minikube docker-env)`.
   101  
   102  The E2E go test (`test/e2e/go`) can be run on a remote cluster by specifying an image name using a repo that you are logged into and
   103  have permission to push to as such:
   104  
   105  ```sh
   106  $ make test/e2e/go ARGS="-image=quay.io/example/memcached:e2e-test"
   107  ```
   108  
   109  All other tests will run correctly on a remote cluster if `$HOME/.kube/config` points to the remote cluster and your
   110  `KUBECONFIG` env var is either empty or is set to the path of a kubeconfig for the remote cluster.
   111  
   112  ## Cleanup of the Go E2E Tests
   113  
   114  The E2E tests create a new project using the operator-sdk to run in the provided
   115  cluster. The tests are designed to cleanup everything that gets created, but some errors
   116  during the go tests can cause these cleanups to fail (the ansible and helm E2E tests should
   117  always clean up correctly). For example, if a segfault occurs or a user kills the
   118  testing process, the cleanup functions for the go tests will not run. To manually clean up a test:
   119  
   120  1. Delete the CRD (`kubectl delete -f $GOPATH/src/github.com/example-inc/memcached-operator/deploy/crds/cache_v1alpha1_memcached_crd.yaml`).
   121  2. Delete the created project in `$GOPATH/src/github.com/example-inc/memcached-operator`
   122  3. Delete the namespaces that the tests run in, which also deletes any resources created within the namespaces. The namespaces start with `memcached-memcached-group` or `main` and are appended with a unix timestamp (seconds since Jan 1 1970). The kubectl command can be used to delete namespaces: `kubectl delete namespace $NAMESPACE`.
   123  
   124  [travis]: ./travis-build.md
   125  [quay]: https://quay.io
   126  [oc-docs]: https://github.com/openshift/origin/blob/v3.11.0/docs/cluster_up_down.md
   127  [oc-binary]: https://github.com/openshift/origin/releases/v3.11.0
   128  [minikube-binary]: https://github.com/kubernetes/minikube/releases
   129  [makefile]: ../../../Makefile