github.com/munnerz/test-infra@v0.0.0-20190108210205-ce3d181dc989/prow/spyglass/README.md (about)

     1  [![GoDoc Widget]][GoDoc]
     2  
     3  # Spyglass
     4  A spyglass is an lensed monocular maritime instrument used to see things that may have been
     5  difficult to see otherwise.
     6  
     7  Spyglass is a pluggable artifact viewer framework for [Prow](..) and a crude
     8  metaphor for the real object. It collects artifacts (usually files in a storage
     9  bucket) from various sources and distributes them to registered viewers, which
    10  are responsible for consuming them and rendering a view.
    11  
    12  A typical Spyglass page might look something like this:
    13  ![I'm not a graphic designer I just make the backend](spyglass-example.png)
    14  
    15  A general Spyglass query will proceed as follows:
    16  - User provides a job source in the query (usually a job name and build ID).
    17  - Spyglass finds all artifact names associated with the given job source.
    18  - Spyglass builds a cache of which artifacts match which lenses via
    19    configured regular expressions.
    20  - Lenses with matching artifacts are pre-rendered in order of descending
    21    priority.
    22  - Spyglass then sends render requests to each registered lens with its
    23    matching artifacts.
    24  - Each lens performs any necessary operations on the artifacts and produces
    25    a blob of HTML.
    26  - Views (HTML) are inserted asynchronously as viewers return.
    27  
    28  
    29  ## Lenses
    30  A lens is an set of functions that consume a list of artifacts and produces some
    31  HTML.
    32  
    33  Lens names are unique, and must much the package name for the lens.
    34  
    35  
    36  ### Built-in Viewers
    37  Spyglass comes with some built-in viewers for commonly produced artifacts.
    38  
    39  - Prow Metadata
    40    ```
    41    Name: metadata
    42    Title: Metadata
    43    Match: finished.json|started.json
    44    Priority: 0
    45    ```
    46  - JUnit
    47    ```
    48    Name: junit
    49    Title: JUnit
    50    Matches: artifacts/junit.*\.xml
    51    Priority: 5
    52    ```
    53  - Logs
    54    ```
    55    Name: buildlog
    56    Title: Build Log
    57    Matches: build-log.txt|pod-log
    58    Priority: 10
    59    ```
    60  
    61  ### Building your own viewer
    62  Building a viewer consists of three main steps.
    63  
    64  #### Write Boilerplate
    65  First, create a package `lensnamehere` under `prow/spyglass/lenses` and
    66  import the `lenses` package.
    67  
    68  #### Implement
    69  Next, implement the necessary functions for a viewer. More specifically,
    70  implement the following interface (defined in lenses.go):
    71  ```go
    72  type Lens interface {
    73      // Name returns the name of your lens (which must match the name of the directory it lives in)
    74  	Name() string
    75  	// Title returns a human-readable title for your lens.
    76  	Title() string
    77  	// Priority returns a number that is used to determine the ordering of your lens (lower is more important)
    78  	Priority() int
    79  	// Header is used to inject content into the lens's <head>. It will only ever be called once per load.
    80  	Header(artifacts []Artifact, resourceDir string) string
    81  	// Body is used to generate the contents of the lens's <body>. It will initially be called with empty data, but
    82  	// the lens front-end code may choose to re-render itself with custom data.
    83  	Body(artifacts []Artifact, resourceDir string, data string) string
    84  	// Callback is used for the viewer to exchange arbitrary data with the frontend. It is called with lens-specified
    85  	// data, and returns data to be passed to the lens. JSON encoding is recommended in both directions.
    86  	Callback(artifacts []Artifact, resourceDir string, data string) string
    87  }
    88  ```
    89  
    90  In the `init` method, call `lenses.RegisterLens()` with an instance of your implementation of the interface.
    91  Spyglass should now be aware of your lens.
    92  
    93  Additionally, some front-end TypeScript code can be provided. Configure your BUILD.bazel to build it, then emit a
    94  \<script> tag with a relative reference to it in your `Header()` implementation. See `buildlog/BUILD.bazel` for an
    95  example.
    96  
    97  In your typescript code, a global `spyglass` object will be available, providing the following interface:
    98  
    99  ```ts
   100  export interface Spyglass {
   101    /**
   102     * Replaces the lens display with a new server-rendered page.
   103     * The returned promise will be resolved once the page has been updated.
   104     */
   105    updatePage(data: string): Promise<void>;
   106    /**
   107     * Requests that the server re-render the lens with the provided data, and
   108     * returns a promise that will resolve with that HTML as a string.
   109     *
   110     * This is equivalent to updatePage(), except that the displayed content is
   111     * not automatically changed.
   112     */
   113    requestPage(data: string): Promise<string>;
   114    /**
   115     * Sends a request to the server-side lens backend with the provided data, and
   116     * returns a promise that will resolve with the response as a string.
   117     */
   118    request(data: string): Promise<string>;
   119    /**
   120     * Inform Spyglass that the lens content has updated. This should be called whenever
   121     * the visible content changes, so Spyglass can ensure that all content is visible.
   122     */
   123    contentUpdated(): void;
   124  }
   125  ```
   126  
   127  #### Add to config
   128  Finally, decide which artifacts you want your viewer to consume and create a regex that
   129  matches these artifacts. The JUnit viewer, for example, consumes all
   130  artifacts that match `artifacts/junit.*\.xml`.
   131  
   132  Add a line in `prow` config under the `viewers` section of `spyglass` of the following form:
   133  ```yaml
   134  "myartifactregexp": ["my-view-name"]
   135  ```
   136  
   137  The next time a job is viewed that contains artifacts matched by your regexp,
   138  your view should display.
   139  
   140  See the [GoDoc](https://godoc.org/k8s.io/test-infra/prow/spyglass/lenses) for
   141  more details and examples.
   142  
   143  ## Config
   144  
   145  Spyglass is currently disabled by default. To enable it, add the `--spyglass` arg to your
   146  [deck deployment](https://github.com/kubernetes/test-infra/blob/e9e544733854d54403aa1dfd84ca009fd9b942f0/prow/cluster/starter.yaml#L236).
   147  
   148  To provide any use, Spyglass must by configured. Its config takes the following form:
   149  ```yaml
   150  deck:
   151    spyglass:
   152      size_limit: 500e+6 # 500MB
   153      viewers:
   154        "started.json|finished.json": ["metadata-viewer"]
   155        "build-log.txt": ["build-log-viewer"]
   156        "artifacts/junit.*\\.xml": ["junit-viewer"] # Remember to escape your '\' in yaml strings!
   157  ```
   158  
   159  More formally, it is a single `spyglass` object under the top-level `deck`
   160  object that may contain fields `viewers` and `size_limit`. `viewers` is a map of `string->[]string`
   161  where the key must be a [valid golang regular
   162  expression](https://github.com/google/re2/wiki/Syntax) and the value is a list
   163  of viewer names that consume the artifacts matched by the corresponding regular
   164  expression. `size_limit` is the maximum artifact size `spyglass` will try to
   165  read in entirety before failing.
   166  
   167  
   168  [GoDoc]: https://godoc.org/k8s.io/test-infra/prow/spyglass
   169  [GoDoc Widget]: https://godoc.org/k8s.io/kubernetes?status.svg