github.com/jenkins-x/test-infra@v0.0.7/README.md (about)

     1  # Kubernetes Test Infrastructure
     2  
     3  [![Go Report Card](https://goreportcard.com/badge/github.com/kubernetes/test-infra)](https://goreportcard.com/report/github.com/kubernetes/test-infra)  [![GoDoc](https://godoc.org/github.com/kubernetes/test-infra?status.svg)](https://godoc.org/github.com/kubernetes/test-infra)
     4  
     5  The test-infra repository contains a collection of tools for testing Kubernetes
     6  and displaying Kubernetes tests results. See also [CONTRIBUTING.md](CONTRIBUTING.md).
     7  
     8  See the [architecture diagram](docs/architecture.svg) for an overview of how
     9  the different services interact.
    10  
    11  ## Viewing test results
    12  
    13  * The [Kubernetes TestGrid](https://testgrid.k8s.io/) shows historical test results
    14    - Configure your own testgrid dashboard at [testgrid/config.yaml](testgrid/config.yaml)
    15    - [Gubernator](https://gubernator.k8s.io/) formats the output of each run
    16  * [PR Dashboard](https://gubernator.k8s.io/pr) finds PRs that need your attention
    17  * [Prow](https://prow.k8s.io) schedules testing and updates issues
    18    - Prow responds to GitHub events, timers and [manual commands](https://go.k8s.io/bot-commands)
    19      given in GitHub comments.
    20    - The [prow dashboard](https://prow.k8s.io/) shows what it is currently testing
    21    - Configure prow to run new tests at [config/jobs](config/jobs)
    22  * [Triage Dashboard](https://go.k8s.io/triage) aggregates failures
    23    - Triage clusters together similar failures
    24    - Search for test failures across jobs
    25    - Filter down failures in a specific regex of tests and/or jobs
    26  * [Velodrome metrics](http://velodrome.k8s.io/dashboard/db/bigquery-metrics?orgId=1) track job and test health.
    27    - [Kettle](kettle) does collection, [metrics](metrics) does reporting, and [velodrome](velodrome) is the frontend.
    28  
    29  
    30  ## E2E Testing
    31  
    32  Our e2e testing uses [kubetest](/kubetest) to build/deploy/test kubernetes
    33  clusters on various providers. Please see those documents for additional details
    34  about this tool as well as e2e testing generally.
    35  
    36  Anyone can reconfigure our CI system with a test-infra PR that updates the
    37  appropriate files. Detailed instructions follow:
    38  
    39  ## CI Job management
    40  
    41  ### Create a new job
    42  
    43  Bootstrap is deprecated, Please utilize the [podutils](prow/pod-utilities.md#how-to-configure) to create new prowjobs.
    44  
    45  Create a PR in this repo to add/update/remove a job or suite. Specifically
    46  you'll need to do the following:
    47  * Add the job to the appropriate section in [`config/jobs`](config/jobs)
    48    - Directory Structure:
    49      - In general for jobs for github.com/org/repo use config/jobs/org/repo/filename.yaml
    50      - For Kubernetes repos we also allow config/jobs/kubernetes/sig-foo/filename.yaml
    51      - We use basename of the config name as a key in the prow configmap, so the name of your config file need to be unique across the config subdir
    52    - Type of jobs:
    53      - Presubmit jobs run on unmerged code in PRs
    54      - Postsubmit jobs run after merging code
    55      - Periodic job run on a timed basis
    56      - You can find more prowjob definitions at [how-to-add-new-jobs](prow/jobs.md#how-to-configure-new-jobs)
    57    - A simple sample job uses podutil looks like:
    58      ```yaml
    59      - name: foo-repo-presubmit-test
    60        decorate: true
    61        spec:
    62          containers:
    63          - image: gcr.io/k8s-testimages/kubekins-e2e:latest-master
    64            command:
    65            - /path/to/cmd
    66            args:
    67            - positional
    68            - --and
    69            - flags
    70      ```
    71  
    72  * Add the job name to the `test_groups` list in [`testgrid/config.yaml`](testgrid/config.yaml)
    73    - Also the group to at least one `dashboard_tab`
    74  
    75  The configs need to be sorted and kubernetes must be in sync with the security repo, or else presubmit will fail.
    76  You can run the script below to keep them valid:
    77  ```
    78  hack/update-config.sh
    79  ```
    80  
    81  #### Local testing
    82  
    83  `docker run` your image locally, and mount in the repos you depend on.
    84  
    85  <!-- TODO: We are working on have a utility to run the job locally - https://github.com/kubernetes/test-infra/issues/6590 -->
    86  
    87  
    88  ### Release branch jobs & Image validation jobs
    89  
    90  Release branch jobs and image validation jobs are defined in [test_config.yaml](experiment/test_config.yaml).
    91  We test different master/node image versions against multiple k8s branches on different features.
    92  
    93  Those jobs are using channel based versions, current supported testing map is:
    94  - k8s-dev : master
    95  - k8s-beta : release-1.13
    96  - k8s-stable1 : release-1.12
    97  - k8s-stable2 : release-1.11
    98  - k8s-stable3 : release-1.10
    99  
   100  Our build job will generate a ci/(channel-name) file pointer in gcs.
   101  
   102  After you update [test_config.yaml](experiment/test_config.yaml), please run
   103  
   104  ```
   105  bazel run //experiment:generate_tests -- --yaml-config-path=experiment/test_config.yaml
   106  ```
   107  
   108  to regenerate the job configs.
   109  
   110  We are moving towards making more jobs to fit into the generated config.
   111  
   112  
   113  Presubmit will tell you if you forget to do any of this correctly.
   114  
   115  Merge your PR and [@k8s-ci-robot] will deploy your change automatically.
   116  
   117  ### Update an existing job
   118  
   119  Largely similar to creating a new job, except you can just modify the existing
   120  entries rather than adding new ones.
   121  
   122  Update what a job does by editing its definition in [`config/jobs`](config/jobs).
   123  
   124  Update where the job appears on testgrid by changing [`testgrid/config.yaml`].
   125  
   126  ### Delete a job
   127  
   128  The reverse of creating a new job: delete the appropriate entries in
   129  [`config/jobs`] and [`testgrid/config.yaml`].
   130  
   131  Merge your PR and [@k8s-ci-robot] will deploy your change automatically.
   132  
   133  ## Building and testing the test-infra
   134  
   135  We use [Bazel](https://www.bazel.io/) to build and test the code in this repo.
   136  The commands `bazel build //...` and `bazel test //...` should be all you need
   137  for most cases. If you modify Go code, run `./hack/update-bazel.sh` to keep
   138  `BUILD.bazel` files up-to-date.
   139  
   140  ## Contributing Test Results
   141  
   142  The Kubernetes project encourages organizations to contribute execution of e2e
   143  test jobs for a variety of platforms (e.g., Azure, rktnetes). For information about
   144  how to contribute test results, see [Contributing Test Results](docs/contributing-test-results.md).
   145  
   146  ## Other Docs
   147  
   148  * [kubernetes/test-infra dependency management](docs/dep.md)
   149  
   150  
   151  [`config/jobs`]: /config/jobs
   152  [`testgrid/config.yaml`]: /testgrid/config.yaml
   153  [test-infra oncall]: https://go.k8s.io/oncall
   154  [@k8s-ci-robot]: (https://github.com/k8s-ci-robot)