k8s.io/test-infra@v0.0.0-20240520184403-27c6b4c223d8/testgrid/README.md (about)

     1  # Testgrid
     2  
     3  NOTE: Most TestGrid code is moving to its new home in
     4  https://github.com/GoogleCloudPlatform/testgrid. See
     5  [#10409](https://github.com/kubernetes/test-infra/issues/10409) for more.
     6  
     7  The Kubernetes Testgrid instance is available at https://testgrid.k8s.io
     8  
     9  We have a short [video] from the testgrid session at the 2018 contributor summit.
    10  
    11  The video demos power features of testgrid, including:
    12  * Sorting
    13  * Filtering
    14  * Graphing
    15  * Grouping
    16  * Dashboard groups
    17  * Summaries
    18  
    19  Please have a look!
    20  
    21  ## Using the client
    22  
    23  Here are some quick tips and clarifications for using the TestGrid site!
    24  
    25  ### Tab Statuses
    26  
    27  TestGrid assigns dashboard tabs a status based on recent test runs.
    28  
    29   *  **PASSING**: No failures found in recent (`num_columns_recent`) test runs.
    30   *  **FAILING**: One or more consistent failures in recent test runs.
    31   *  **FLAKY**: The tab is neither PASSING nor FAILING. There is at least one
    32      recent failed result that is not a consistent failure.
    33  
    34  ### Summary Widget
    35  
    36  You can get a small widget showing the status of your dashboard tab, based on
    37  the tab statuses above! For example:
    38  
    39  `sig-testing-misc#ci-bazel`: [![sig-testing-misc/ci-bazel](https://testgrid.k8s.io/q/summary/sig-testing-misc/ci-bazel/tests_status?style=svg)](https://testgrid.k8s.io/sig-testing-misc#ci-bazel)
    40  
    41  Inline it with:
    42  
    43  ```
    44  <!-- Inline with a link to your tab -->
    45  [![<dashboard_name>/<tab_name>](https://testgrid.k8s.io/q/summary/<dashboard_name>/<tab_name>/tests_status?style=svg)](https://testgrid.k8s.io/<dashboard_name>#<tab_name>)
    46  ```
    47  
    48  ### Customizing Test Result Sizes
    49  
    50  Change the size of the test result rectangles.
    51  
    52  The three sizes are Standard, Compact, and Super Compact. You can also specify
    53  `width=X` in the URL (X > 3) to customize the width. For small widths, this may
    54  mean the date and/or changelist, or other custom headers, are no longer
    55  visible.
    56  
    57  ### Filtering Tests
    58  
    59  You can repeatedly add filters to include/exclude test rows. Under **Options**:
    60  
    61  *   **Include/Exclude Filter by RegEx**: Specify a regular expression that
    62      matches test names for rows you'd like to include/exclude.
    63  *   **Exclude non-failed Tests**: Omit rows with no failing results.
    64  
    65  ### Grouping Tests
    66  
    67  Grouped tests are summarized in a single row that is collapsible/expandable by
    68  clicking on the test name (shown as a triangle on the left). Under **Options**:
    69  
    70  *   **Group by RegEx Mask**: Specify a regular expression to mask a portion of
    71      the test name. Any test names that match after applying this mask will be
    72      grouped together.
    73  *   **Group by Target**: Any tests that contain the same target will be
    74      grouped together.
    75  *   **Group by Hierarchy Pattern**: Specify a regular expression that matches
    76      one or more parts of the tests' names and the tests will be grouped
    77      hierarchically. For example, if you have these tests in your dashboard:
    78  
    79      ```text
    80      /test/dir1/target1
    81      /test/dir1/target2
    82      /test/dir2/target3
    83      ```
    84  
    85      By specifying regular expression "\w+", the tests will be organized into:
    86  
    87      ```text
    88      ▼test
    89        ▼dir1
    90          target1
    91        ▼dir2
    92          target2
    93          target3
    94      ```
    95  
    96  ### Sorting Tests
    97  
    98  Under **Options**
    99  
   100  *   **Sort by Failures**: Tests with more recent failures will appear before
   101      other tests.
   102  *   **Sort by Flakiness**: Tests with a higher flakiness score will appear
   103      before tests with a lower flakiness score. The flakiness score, which is not
   104      reported, is based on the number of transitions from passing to failing (and
   105      vice versa) with more weight given to more recent transitions.
   106  *   **Sort by Name**: Sort alphabetically.
   107  
   108  ## Clustered Failures
   109  
   110  You can display identified clustered failures in your test results grid in a
   111  dashboard tab. Select the ***Display Clustered Failures List*** toggle button to
   112  render a list/table of identified failure clusters at the bottom of the browser.
   113  
   114  Clusters can be grouped by:
   115  * test status
   116  * test status and error message
   117  
   118  The clustered failures table shows the test status, error message (if grouped by
   119  error message), and area of the clusters. The clusters are sorted by area in
   120  descending order.
   121  
   122  Selecting a row highlights the cells belonging to that cluster. Multiple row
   123  selection (with multiple cluster highlighting) is supported. To de-select a row,
   124  click on the selected row again.
   125  
   126  ## Configuration
   127  
   128  If you need to add a new test that you want TestGrid to display, or otherwise change what is shown
   129  on https://testgrid.k8s.io, see [Testgrid Configuration](config.md).
   130  
   131  Updates to the config are automatically tested and pushed to production.
   132  
   133  ## Contributing
   134  
   135  If you want to modify TestGrid beyond adding new tests or dashboards, see [Updating Testgrid](build_test_update.md).
   136  
   137  [video]: https://www.youtube.com/watch?v=jm2l2SLq_yE