github.com/GoogleCloudPlatform/testgrid@v0.0.174/standalone.md (about)

     1  # Setting up TestGrid
     2  
     3  This tutorial goes over setting up your own TestGrid instance in [Google Cloud Storage]
     4  (GCS). It assumes you are somewhat familiar with deploying on [Kubernetes], running a job on [Prow],
     5  and using GCS.
     6  
     7  Usually, to get your tests to display in TestGrid, setting up an instance isn't
     8  necessary. Instead, add your tests to an existing [configuration].
     9  
    10  ## Architecture Overview
    11  
    12  TestGrid consists of two major components; its "backend" controllers and its
    13  "frontend" user interface.
    14  
    15  These two components communicate using data stored in a cloud storage provider.
    16  The controllers are responsible for keeping this data up-to-date. The user
    17  interface displays this data.
    18  
    19  - The source for the backend controllers are in the [cmd](./cmd) folder.
    20  - Formats for the data layer are found in the [pb](./pb) folder.
    21  - The frontend user interface isn't open source, but it can still be used as a service.
    22  
    23  ## Configuration
    24  
    25  TestGrid depends on a [configuration](./pb/config) file to
    26  determine what it should display.
    27  
    28  You can get a configuration file in multiple ways:
    29  - Serialize it from a YAML configuration file you write yourself
    30  - Have a Prow instance generate a configuration file with Configurator
    31  
    32  The configuration file should be named "config" (e.g. `gs://your-bucket/config`) in cloud storage,
    33  and will be needed by every controller.
    34  
    35  ## Controllers
    36  
    37  From a configuration file, controller programs fetch and generate the other
    38  necessary data files.
    39  
    40  The [Updater](./cmd/updater) generates and maintains the grid data to be displayed, for each test group.
    41  Each test's current [state](./pb/state) is stored in cloud storage.
    42  
    43  The [Tabulator](./cmd/tabulator) takes grid data from the Updater and creates a viewable grid (also a
    44  [state](./pb/state)) for each dashboard tab. Each of these are stored in cloud storage.
    45  
    46  The [Summarizer](./cmd/summarizer) generates and maintains a summary for each dashboard. These
    47  [summaries](./pb/summary) are stored in cloud storage.
    48  
    49  ## Frontend Usage
    50  
    51  - **Frontend API endpoints are subject to change as development continues.**
    52  
    53  A TestGrid instance, like the one at [testgrid.k8s.io], displays a particular
    54  configuration and state by default.
    55  
    56  You can display other data (generated locally or by another instance) from a GCS bucket.
    57  The TestGrid configuration located at `gs://your-bucket/config` can be rendered at
    58  `testgrid.k8s.io/r/your-bucket/`.
    59  
    60  - The service account `k8s-testgrid@appspot.gserviceaccount.com` must be given
    61  permission to read from these files.
    62  
    63  ## Examples
    64  
    65  ### Standalone Testgrid FE For Knative Prow
    66  
    67  #### Generate testgrid config
    68  
    69  Set up a prow job that generates testgrid config from prow configs with
    70  `configurator`. In knative scenario, the [configurator job example] writes the
    71  config to a GCS bucket `gs://knative-own-testgrid/config`.
    72  
    73  After this step, `testgrid.k8s.io/r/knative-own-testgrid` can already display
    74  dashboards and dashboard tabs, but there are no test results yet.
    75  
    76  #### Load data
    77  
    78  - Create a `testgrid` cluster
    79    - Note: this can be a kubernetes cluster of your choice.
    80  - Create the [deployment] of `Updater`, `Tabulator`, and `Summarizer` in this cluster.
    81    - Make sure to pass `--config=gs://knative-own-testgrid` to each controller.
    82  
    83  After this step, data will slowly roll into `testgrid.k8s.io/r/knative-own-testgrid`. The initial round
    84  could take a couple of hours, then subsequent updates will be much faster.
    85  
    86  
    87  [testgrid.k8s.io]: (http://testgrid.k8s.io)
    88  [configuration]: config.md
    89  [configurator job example]: https://github.com/knative/test-infra/blob/13d8d5df1f5273ff3d751bf8cd92eec873a35d7b/config/prod/prow/jobs/custom/test-infra.yaml#L643
    90  [deployment]:
    91  https://github.com/GoogleCloudPlatform/testgrid/tree/master/cluster/prod/knative
    92  [Prow]: https://docs.prow.k8s.io/docs/
    93  [Google Cloud Storage]: https://cloud.google.com/storage/docs
    94  [Kubernetes]: https://kubernetes.io/docs/home/
    95