github.com/shashidharatd/test-infra@v0.0.0-20171006011030-71304e1ca560/README.md (about) 1 # Kubernetes Test Infrastructure 2 3 [![Build Status](https://travis-ci.org/kubernetes/test-infra.svg?branch=master)](https://travis-ci.org/kubernetes/test-infra) [![Go Report Card](https://goreportcard.com/badge/github.com/kubernetes/test-infra)](https://goreportcard.com/report/github.com/kubernetes/test-infra) [![GoDoc](https://godoc.org/github.com/kubernetes/test-infra?status.svg)](https://godoc.org/github.com/kubernetes/test-infra) 4 5 The test-infra repository contains a collection of tools for testing Kubernetes 6 and displaying Kubernetes tests results. See also [CONTRIBUTING.md](CONTRIBUTING.md). 7 8 See the [architecture diagram](docs/architecture.svg) for an overview of how 9 the different services interact. 10 11 ## Viewing test results 12 13 * The [Kubernetes TestGrid](https://k8s-testgrid.appspot.com/) shows historical test results 14 - Configure your own testgrid dashboard at [testgrid/config/config.yaml](testgrid/config/config.yaml) 15 - [Gubernator](https://k8s-gubernator.appspot.com/) formats the output of each run 16 * [PR Dashboard](https://k8s-gubernator.appspot.com/pr) finds PRs that need your attention 17 * [Prow](https://prow.k8s.io) schedules testing and updates issues 18 - Prow responds to GitHub events, timers and manual commands 19 - The [prow dashboard](https://prow.k8s.io/) shows what it is currently testing 20 - Configure prow to run new tests at [prow/config.yaml](prow/config.yaml) 21 * [Triage Dashboard](https://go.k8s.io/triage) aggregates failures 22 - Triage clusters together similar failures 23 - Search for test failures across jobs 24 - Filter down failures in a specific regex of tests and/or jobs 25 * [Test history](https://go.k8s.io/test-history) is a deprecated tool 26 - Use the triage dashboard instead 27 - Summarizes the last 24 hours of testing 28 - See [Kettle](kettle) and the corresponding [bigquery metrics](metrics) that largely supplement this information 29 30 31 ## Automated testing 32 33 Test anything with the following pattern: 34 35 ``` 36 git clone https://github.com/kubernetes/test-infra 37 test-infra/jenkins/bootstrap.py --job=J --repo=R --service-account=S.json --upload=gs://B 38 ``` 39 40 The `--job=J` flag specifies what test job to run. 41 The `--repo=R` (or `--bare`) flag controls what we check out from git. 42 43 Anyone can reconfigure our CI system with a test-infra PR that updates the 44 appropriate files. Detailed instructions follow: 45 46 ### E2E Testing 47 48 Our e2e testing uses [kubetest](/kubetest) to build/deploy/test kubernetes 49 clusters on various providers. Please see those documents for additional details 50 about this tool as well as e2e testing generally. 51 52 ### Create a new job 53 54 Create a PR in this repo to add/update/remove a job or suite. Specifically 55 you'll need to do the following: 56 * Create an entry in [`jobs/config.json`] for the job 57 - If this is a kubetest job create the corresponding `jobs/env/FOO.env` file 58 - It will pick a free project from [boskos](/boskos) pool by default, or 59 - You can also set --gcp-project=foo in [`jobs/config.json`] for a dedicated project, make sure the project has the right [IAM grants](jenkins/check_projects.py) 60 * Add the job name to the `test_groups` list in [`testgrid/config/config.yaml`](https://github.com/kubernetes/test-infra/blob/master/testgrid/config/config.yaml) 61 - Also the group to at least one `dashboard_tab` 62 * Add the job to the appropriate section in [`prow/config.yaml`](https://github.com/kubernetes/test-infra/blob/master/prow/config.yaml) 63 - Presubmit jobs run on unmerged code in PRs 64 - Postsubmit jobs run after merging code 65 - Periodic job run on a timed basis 66 * (Deprecated!) Some old jobs still run on jenkins 67 - Please do not add new jobs to jenkins 68 - Jenkins configuration is defined at `jenkins/job-configs` 69 - More deprecated details at [jenkins/README.md](jenkins/README.md) 70 71 NOTE: `kubernetes/kubernetes` and `kubernetes-security/kubernetes` must have matching presubmits. 72 73 Please test the job on your local workstation before creating a PR: 74 ``` 75 mkdir /tmp/whatever && cd /tmp/whatever 76 $GOPATH/src/k8s.io/test-infra/jenkins/bootstrap.py \ 77 --job=J \ # aka your new job 78 --repo=R1 --repo=R2 \ # what repos to check out 79 --service-account ~/S.json # the service account to use to launch GCE/GKE clusters 80 # Note: create a service account at the cloud console for the project J uses 81 ``` 82 83 Presubmit will tell you if you forget to do any of this correctly. 84 85 Merge your PR and the bot will deploy your change automatically. 86 87 ### Update an existing job 88 89 Largely similar to creating a new job, except you can just modify the existing 90 entries rather than adding new ones. 91 92 Update what a job does by editing its definition in [`jobs/config.json`]. For 93 the kubetest jobs this typically means editing the `jobs/FOO.env` files it uses. 94 95 Update when a job runs by changing its definition in [`prow/config.yaml`]. 96 The [test-infra oncall] must push prow changes (`make -C prow update-config`). 97 98 Update where the job appears on testgrid by changing [`testgrid/config/config.yaml`]. 99 100 ### Delete a job 101 102 The reverse of creating a new job: delete the appropriate entries in 103 [`jobs/config.json`], [`prow/config.yaml`] and [`testgrid/config/config.yaml`]. 104 105 The [test-infra oncall] must push prow changes (`make -C prow update-config`). 106 107 ## Building and testing the test-infra 108 109 We use [Bazel](https://www.bazel.io/) to build and test the code in this repo. 110 The commands `bazel build //...` and `bazel test //...` should be all you need 111 for most cases. If you modify Go code, run `./verify/update-bazel.sh` to keep 112 `BUILD` files up-to-date. 113 114 ## Federated Testing 115 116 The Kubernetes project encourages organizations to contribute execution of e2e 117 test jobs for a variety of platforms (e.g., Azure, rktnetes). The test-history 118 scripts gather e2e results from these federated jobs. For information about 119 how to contribute test results, see [Federated Testing](docs/federated_testing.md). 120 121 122 [`jobs/config.json`]: https://github.com/kubernetes/test-infra/blob/master/jobs/config.json 123 [`prow/config.yaml`]: https://github.com/kubernetes/test-infra/blob/master/prow/config.yaml 124 [`testgrid/config/config.yaml`]: https://github.com/kubernetes/test-infra/blob/master/testgrid/config/config.yaml 125 [test-infra oncall]: https://go.k8s.io/oncall