github.com/jenkins-x/test-infra@v0.0.7/README.md (about) 1 # Kubernetes Test Infrastructure 2 3 [![Go Report Card](https://goreportcard.com/badge/github.com/kubernetes/test-infra)](https://goreportcard.com/report/github.com/kubernetes/test-infra) [![GoDoc](https://godoc.org/github.com/kubernetes/test-infra?status.svg)](https://godoc.org/github.com/kubernetes/test-infra) 4 5 The test-infra repository contains a collection of tools for testing Kubernetes 6 and displaying Kubernetes tests results. See also [CONTRIBUTING.md](CONTRIBUTING.md). 7 8 See the [architecture diagram](docs/architecture.svg) for an overview of how 9 the different services interact. 10 11 ## Viewing test results 12 13 * The [Kubernetes TestGrid](https://testgrid.k8s.io/) shows historical test results 14 - Configure your own testgrid dashboard at [testgrid/config.yaml](testgrid/config.yaml) 15 - [Gubernator](https://gubernator.k8s.io/) formats the output of each run 16 * [PR Dashboard](https://gubernator.k8s.io/pr) finds PRs that need your attention 17 * [Prow](https://prow.k8s.io) schedules testing and updates issues 18 - Prow responds to GitHub events, timers and [manual commands](https://go.k8s.io/bot-commands) 19 given in GitHub comments. 20 - The [prow dashboard](https://prow.k8s.io/) shows what it is currently testing 21 - Configure prow to run new tests at [config/jobs](config/jobs) 22 * [Triage Dashboard](https://go.k8s.io/triage) aggregates failures 23 - Triage clusters together similar failures 24 - Search for test failures across jobs 25 - Filter down failures in a specific regex of tests and/or jobs 26 * [Velodrome metrics](http://velodrome.k8s.io/dashboard/db/bigquery-metrics?orgId=1) track job and test health. 27 - [Kettle](kettle) does collection, [metrics](metrics) does reporting, and [velodrome](velodrome) is the frontend. 28 29 30 ## E2E Testing 31 32 Our e2e testing uses [kubetest](/kubetest) to build/deploy/test kubernetes 33 clusters on various providers. Please see those documents for additional details 34 about this tool as well as e2e testing generally. 35 36 Anyone can reconfigure our CI system with a test-infra PR that updates the 37 appropriate files. Detailed instructions follow: 38 39 ## CI Job management 40 41 ### Create a new job 42 43 Bootstrap is deprecated, Please utilize the [podutils](prow/pod-utilities.md#how-to-configure) to create new prowjobs. 44 45 Create a PR in this repo to add/update/remove a job or suite. Specifically 46 you'll need to do the following: 47 * Add the job to the appropriate section in [`config/jobs`](config/jobs) 48 - Directory Structure: 49 - In general for jobs for github.com/org/repo use config/jobs/org/repo/filename.yaml 50 - For Kubernetes repos we also allow config/jobs/kubernetes/sig-foo/filename.yaml 51 - We use basename of the config name as a key in the prow configmap, so the name of your config file need to be unique across the config subdir 52 - Type of jobs: 53 - Presubmit jobs run on unmerged code in PRs 54 - Postsubmit jobs run after merging code 55 - Periodic job run on a timed basis 56 - You can find more prowjob definitions at [how-to-add-new-jobs](prow/jobs.md#how-to-configure-new-jobs) 57 - A simple sample job uses podutil looks like: 58 ```yaml 59 - name: foo-repo-presubmit-test 60 decorate: true 61 spec: 62 containers: 63 - image: gcr.io/k8s-testimages/kubekins-e2e:latest-master 64 command: 65 - /path/to/cmd 66 args: 67 - positional 68 - --and 69 - flags 70 ``` 71 72 * Add the job name to the `test_groups` list in [`testgrid/config.yaml`](testgrid/config.yaml) 73 - Also the group to at least one `dashboard_tab` 74 75 The configs need to be sorted and kubernetes must be in sync with the security repo, or else presubmit will fail. 76 You can run the script below to keep them valid: 77 ``` 78 hack/update-config.sh 79 ``` 80 81 #### Local testing 82 83 `docker run` your image locally, and mount in the repos you depend on. 84 85 <!-- TODO: We are working on have a utility to run the job locally - https://github.com/kubernetes/test-infra/issues/6590 --> 86 87 88 ### Release branch jobs & Image validation jobs 89 90 Release branch jobs and image validation jobs are defined in [test_config.yaml](experiment/test_config.yaml). 91 We test different master/node image versions against multiple k8s branches on different features. 92 93 Those jobs are using channel based versions, current supported testing map is: 94 - k8s-dev : master 95 - k8s-beta : release-1.13 96 - k8s-stable1 : release-1.12 97 - k8s-stable2 : release-1.11 98 - k8s-stable3 : release-1.10 99 100 Our build job will generate a ci/(channel-name) file pointer in gcs. 101 102 After you update [test_config.yaml](experiment/test_config.yaml), please run 103 104 ``` 105 bazel run //experiment:generate_tests -- --yaml-config-path=experiment/test_config.yaml 106 ``` 107 108 to regenerate the job configs. 109 110 We are moving towards making more jobs to fit into the generated config. 111 112 113 Presubmit will tell you if you forget to do any of this correctly. 114 115 Merge your PR and [@k8s-ci-robot] will deploy your change automatically. 116 117 ### Update an existing job 118 119 Largely similar to creating a new job, except you can just modify the existing 120 entries rather than adding new ones. 121 122 Update what a job does by editing its definition in [`config/jobs`](config/jobs). 123 124 Update where the job appears on testgrid by changing [`testgrid/config.yaml`]. 125 126 ### Delete a job 127 128 The reverse of creating a new job: delete the appropriate entries in 129 [`config/jobs`] and [`testgrid/config.yaml`]. 130 131 Merge your PR and [@k8s-ci-robot] will deploy your change automatically. 132 133 ## Building and testing the test-infra 134 135 We use [Bazel](https://www.bazel.io/) to build and test the code in this repo. 136 The commands `bazel build //...` and `bazel test //...` should be all you need 137 for most cases. If you modify Go code, run `./hack/update-bazel.sh` to keep 138 `BUILD.bazel` files up-to-date. 139 140 ## Contributing Test Results 141 142 The Kubernetes project encourages organizations to contribute execution of e2e 143 test jobs for a variety of platforms (e.g., Azure, rktnetes). For information about 144 how to contribute test results, see [Contributing Test Results](docs/contributing-test-results.md). 145 146 ## Other Docs 147 148 * [kubernetes/test-infra dependency management](docs/dep.md) 149 150 151 [`config/jobs`]: /config/jobs 152 [`testgrid/config.yaml`]: /testgrid/config.yaml 153 [test-infra oncall]: https://go.k8s.io/oncall 154 [@k8s-ci-robot]: (https://github.com/k8s-ci-robot)