github.com/GoogleCloudPlatform/testgrid@v0.0.174/README.md (about)

     1  # TestGrid
     2  Welcome to TestGrid, a highly-configurable, interactive dashboard for viewing
     3  your test results in a grid!
     4  
     5  Check out Kubernetes' instance at https://testgrid.k8s.io.
     6  
     7  We're in the middle of open-sourcing TestGrid! You can check this repo or
     8  [this issue](https://github.com/kubernetes/test-infra/issues/10409) for more
     9  updates as we continue.
    10  
    11  We have a short [video] from the TestGrid session at the 2018 Kubernetes
    12  contributor summit.
    13  
    14  The video demos power features of TestGrid, including:
    15  * Sorting
    16  * Filtering
    17  * Graphing
    18  * Grouping
    19  * Dashboard groups
    20  * Summaries
    21  * Clustered Failures
    22  
    23  Please have a look!
    24  
    25  ## Contributing
    26  
    27  If you want to modify TestGrid beyond adding new tests or dashboards, see
    28  [Contributing](CONTRIBUTING.md) and [Updating Testgrid](build_test_update.md).
    29  
    30  ## Configuration
    31  
    32  If you need to add a new test that you want TestGrid to display, or otherwise change what is shown
    33  on https://testgrid.k8s.io, see [Testgrid Configuration](config.md).
    34  
    35  Updates to the config are automatically tested and pushed to production.
    36  
    37  ## Using the client
    38  
    39  Here are some quick tips and clarifications for using the TestGrid site!
    40  
    41  ### Tab Statuses
    42  
    43  TestGrid assigns dashboard tabs a status based on recent test runs.
    44  
    45   *  **PASSING**: No failures found in recent (`num_columns_recent`) test runs.
    46   *  **FAILING**: One or more consistent failures in recent test runs.
    47   *  **FLAKY**: The tab is neither PASSING nor FAILING. There is at least one
    48      recent failed result that is not a consistent failure.
    49  
    50  ### Summary Widget
    51  
    52  You can get a small widget showing the status of your dashboard tab, based on
    53  the tab statuses above! For example:
    54  
    55  `sig-testing-misc#ci-bazel`: [![sig-testing-misc/ci-bazel](https://testgrid.k8s.io/q/summary/sig-testing-misc/ci-bazel/tests_status?style=svg)](https://testgrid.k8s.io/sig-testing-misc#ci-bazel)
    56  
    57  Inline it with:
    58  
    59  ```
    60  <!-- Inline with a link to your tab -->
    61  [![<dashboard_name>/<tab_name>](https://testgrid.k8s.io/q/summary/<dashboard_name>/<tab_name>/tests_status?style=svg)](https://testgrid.k8s.io/<dashboard_name>#<tab_name>)
    62  ```
    63  
    64  ### Customizing Test Result Sizes
    65  
    66  Change the size of the test result rectangles.
    67  
    68  The three sizes are Standard, Compact, and Super Compact. You can also specify
    69  `width=X` in the URL (X > 3) to customize the width. For small widths, this may
    70  mean the date and/or changelist, or other custom headers, are no longer
    71  visible.
    72  
    73  ### Filtering Tests
    74  
    75  You can repeatedly add filters to include/exclude test rows. Under **Options**:
    76  
    77  *   **Include/Exclude Filter by RegEx**: Specify a regular expression that
    78      matches test names for rows you'd like to include/exclude.
    79  *   **Exclude non-failed Tests**: Omit rows with no failing results.
    80  
    81  ### Grouping Tests
    82  
    83  Grouped tests are summarized in a single row that is collapsible/expandable by
    84  clicking on the test name (shown as a triangle on the left). Under **Options**:
    85  
    86  *   **Group by RegEx Mask**: Specify a regular expression to mask a portion of
    87      the test name. Any test names that match after applying this mask will be
    88      grouped together.
    89  *   **Group by Target**: Any tests that contain the same target will be
    90      grouped together.
    91  *   **Group by Hierarchy Pattern**: Specify a regular expression that matches
    92      one or more parts of the tests' names and the tests will be grouped
    93      hierarchically. For example, if you have these tests in your dashboard:
    94  
    95      ```text
    96      /test/dir1/target1
    97      /test/dir1/target2
    98      /test/dir2/target3
    99      ```
   100  
   101      By specifying regular expression "\w+", the tests will be organized into:
   102  
   103      ```text
   104      ▼test
   105        ▼dir1
   106          target1
   107        ▼dir2
   108          target2
   109          target3
   110      ```
   111  
   112  ### Sorting Tests
   113  
   114  Under **Options**
   115  
   116  *   **Sort by Failures**: Tests with more recent failures will appear before
   117      other tests.
   118  *   **Sort by Flakiness**: Tests with a higher flakiness score will appear
   119      before tests with a lower flakiness score. The flakiness score, which is not
   120      reported, is based on the number of transitions from passing to failing (and
   121      vice versa) with more weight given to more recent transitions.
   122  *   **Sort by Name**: Sort alphabetically.
   123  
   124  ## Clustered Failures
   125  
   126  You can display identified clustered failures in your test results grid in a
   127  dashboard tab. Select the ***Display Clustered Failures List*** toggle button to
   128  render a list/table of identified failure clusters at the bottom of the browser.
   129  
   130  Clusters can be grouped by:
   131  * test status
   132  * test status and error message
   133  
   134  The clustered failures table shows the test status, error message (if grouped by
   135  error message), and area of the clusters. The clusters are sorted by area in
   136  descending order.
   137  
   138  Selecting a row highlights the cells belonging to that cluster. Multiple row
   139  selection (with multiple cluster highlighting) is supported. To de-select a row,
   140  click on the selected row again.
   141  
   142  ## Creating a TestGrid Instance
   143  
   144  Information on setting up your own TestGrid instance can be found [here](./standalone.md).
   145  
   146  If you're looking to add your configuration to an existing instance
   147  (such as testgrid.k8s.io), see [config.md](./config.md) instead.
   148  
   149  [video]: https://www.youtube.com/watch?v=jm2l2SLq_yE