github.com/shashidharatd/test-infra@v0.0.0-20171006011030-71304e1ca560/README.md (about)

     1  # Kubernetes Test Infrastructure
     2  
     3  [![Build Status](https://travis-ci.org/kubernetes/test-infra.svg?branch=master)](https://travis-ci.org/kubernetes/test-infra)  [![Go Report Card](https://goreportcard.com/badge/github.com/kubernetes/test-infra)](https://goreportcard.com/report/github.com/kubernetes/test-infra)  [![GoDoc](https://godoc.org/github.com/kubernetes/test-infra?status.svg)](https://godoc.org/github.com/kubernetes/test-infra)
     4  
     5  The test-infra repository contains a collection of tools for testing Kubernetes
     6  and displaying Kubernetes tests results. See also [CONTRIBUTING.md](CONTRIBUTING.md).
     7  
     8  See the [architecture diagram](docs/architecture.svg) for an overview of how
     9  the different services interact.
    10  
    11  ## Viewing test results
    12  
    13  * The [Kubernetes TestGrid](https://k8s-testgrid.appspot.com/) shows historical test results
    14    - Configure your own testgrid dashboard at [testgrid/config/config.yaml](testgrid/config/config.yaml)
    15    - [Gubernator](https://k8s-gubernator.appspot.com/) formats the output of each run
    16  * [PR Dashboard](https://k8s-gubernator.appspot.com/pr) finds PRs that need your attention
    17  * [Prow](https://prow.k8s.io) schedules testing and updates issues
    18    - Prow responds to GitHub events, timers and manual commands
    19    - The [prow dashboard](https://prow.k8s.io/) shows what it is currently testing
    20    - Configure prow to run new tests at [prow/config.yaml](prow/config.yaml)
    21  * [Triage Dashboard](https://go.k8s.io/triage) aggregates failures
    22    - Triage clusters together similar failures
    23    - Search for test failures across jobs
    24    - Filter down failures in a specific regex of tests and/or jobs
    25  * [Test history](https://go.k8s.io/test-history) is a deprecated tool
    26    - Use the triage dashboard instead
    27    - Summarizes the last 24 hours of testing
    28    - See [Kettle](kettle) and the corresponding [bigquery metrics](metrics) that largely supplement this information
    29  
    30  
    31  ## Automated testing
    32  
    33  Test anything with the following pattern:
    34  
    35  ```
    36  git clone https://github.com/kubernetes/test-infra
    37  test-infra/jenkins/bootstrap.py --job=J --repo=R --service-account=S.json --upload=gs://B
    38  ```
    39  
    40  The `--job=J` flag specifies what test job to run.
    41  The `--repo=R` (or `--bare`) flag controls what we check out from git.
    42  
    43  Anyone can reconfigure our CI system with a test-infra PR that updates the
    44  appropriate files. Detailed instructions follow:
    45  
    46  ### E2E Testing
    47  
    48  Our e2e testing uses [kubetest](/kubetest) to build/deploy/test kubernetes
    49  clusters on various providers. Please see those documents for additional details
    50  about this tool as well as e2e testing generally.
    51  
    52  ### Create a new job
    53  
    54  Create a PR in this repo to add/update/remove a job or suite. Specifically
    55  you'll need to do the following:
    56  * Create an entry in [`jobs/config.json`] for the job
    57    - If this is a kubetest job create the corresponding `jobs/env/FOO.env` file
    58    - It will pick a free project from [boskos](/boskos) pool by default, or
    59    - You can also set --gcp-project=foo in [`jobs/config.json`] for a dedicated project, make sure the project has the right [IAM grants](jenkins/check_projects.py)
    60  * Add the job name to the `test_groups` list in [`testgrid/config/config.yaml`](https://github.com/kubernetes/test-infra/blob/master/testgrid/config/config.yaml)
    61    - Also the group to at least one `dashboard_tab`
    62  * Add the job to the appropriate section in [`prow/config.yaml`](https://github.com/kubernetes/test-infra/blob/master/prow/config.yaml)
    63    - Presubmit jobs run on unmerged code in PRs
    64    - Postsubmit jobs run after merging code
    65    - Periodic job run on a timed basis
    66  * (Deprecated!) Some old jobs still run on jenkins
    67    - Please do not add new jobs to jenkins
    68    - Jenkins configuration is defined at `jenkins/job-configs`
    69    - More deprecated details at [jenkins/README.md](jenkins/README.md)
    70  
    71  NOTE: `kubernetes/kubernetes` and `kubernetes-security/kubernetes` must have matching presubmits.
    72  
    73  Please test the job on your local workstation before creating a PR:
    74  ```
    75  mkdir /tmp/whatever && cd /tmp/whatever
    76  $GOPATH/src/k8s.io/test-infra/jenkins/bootstrap.py \
    77    --job=J \  # aka your new job
    78    --repo=R1 --repo=R2 \  # what repos to check out
    79    --service-account ~/S.json  # the service account to use to launch GCE/GKE clusters
    80  # Note: create a service account at the cloud console for the project J uses
    81  ```
    82  
    83  Presubmit will tell you if you forget to do any of this correctly.
    84  
    85  Merge your PR and the bot will deploy your change automatically.
    86  
    87  ### Update an existing job
    88  
    89  Largely similar to creating a new job, except you can just modify the existing
    90  entries rather than adding new ones.
    91  
    92  Update what a job does by editing its definition in [`jobs/config.json`]. For
    93  the kubetest jobs this typically means editing the `jobs/FOO.env` files it uses.
    94  
    95  Update when a job runs by changing its definition in [`prow/config.yaml`].
    96  The [test-infra oncall] must push prow changes (`make -C prow update-config`).
    97  
    98  Update where the job appears on testgrid by changing [`testgrid/config/config.yaml`].
    99  
   100  ### Delete a job
   101  
   102  The reverse of creating a new job: delete the appropriate entries in
   103  [`jobs/config.json`], [`prow/config.yaml`] and [`testgrid/config/config.yaml`].
   104  
   105  The [test-infra oncall] must push prow changes (`make -C prow update-config`).
   106  
   107  ## Building and testing the test-infra
   108  
   109  We use [Bazel](https://www.bazel.io/) to build and test the code in this repo.
   110  The commands `bazel build //...` and `bazel test //...` should be all you need
   111  for most cases. If you modify Go code, run `./verify/update-bazel.sh` to keep
   112  `BUILD` files up-to-date.
   113  
   114  ## Federated Testing
   115  
   116  The Kubernetes project encourages organizations to contribute execution of e2e
   117  test jobs for a variety of platforms (e.g., Azure, rktnetes).  The test-history
   118  scripts gather e2e results from these federated jobs.  For information about
   119  how to contribute test results, see [Federated Testing](docs/federated_testing.md).
   120  
   121  
   122  [`jobs/config.json`]: https://github.com/kubernetes/test-infra/blob/master/jobs/config.json
   123  [`prow/config.yaml`]: https://github.com/kubernetes/test-infra/blob/master/prow/config.yaml
   124  [`testgrid/config/config.yaml`]: https://github.com/kubernetes/test-infra/blob/master/testgrid/config/config.yaml
   125  [test-infra oncall]: https://go.k8s.io/oncall