go.chromium.org/luci@v0.0.0-20240309015107-7cdc2e660f33/logdog/appengine/cmd/coordinator/README.md (about)

     1  LogDog Coordinator
     2  ==================
     3  
     4  The LogDog Coordinator is an AppEngine application that serves as a central
     5  coordinating and interactive authority for a LogDog instance. The Coordinator
     6  is responsible for:
     7  
     8  * Supplying logs to end users through its logs API.
     9  * Coordinating log stream state throughout its lifecycle.
    10  * Handling Butler Prefix registration.
    11  * Acting as a configuration authority for its deployment.
    12  * Accepting stream registrations from **Collector** instances.
    13  * Dispatching archival tasks to **Archivist** instances.
    14  
    15  ## Services
    16  
    17  A **Coordinator** occupies the AppEngine space of a given cloud project, and
    18  assumes ownership of that project's resources. It is composed of several
    19  cooperative AppEngine services.
    20  
    21  ### Default
    22  
    23  The [default](default/) service handles basic LUCI services. Most other requests are
    24  redirected to other services by [dispatch.yaml](default/dispatch.yaml).
    25  
    26  ### Logs
    27  
    28  The [logs](logs/) service exposes the
    29  [Logs API](../../../api/endpoints/coordinator/logs/v1/) for log stream querying
    30  and consumption.
    31  
    32  It is a AppEngine Flex instance using "custom" runtime, since Flex is the only
    33  supported type of AppEngine instance that can stream responses to the clients,
    34  and we do that when clients read unfinalized logs. It is a custom runtime,
    35  because Flex doesn't support Go runtimes newer than go1.15. To use a newer
    36  version we need to build a Docker image ourselves (which requires specifying
    37  "custom" runtime).
    38  
    39  ### Services
    40  
    41  The [services](services/) service exposes management endpoints to the instance's
    42  microservices, notably the [Collector](../../../server/cmd/logdog_collector) and
    43  [Archivist](../../../server/cmd/logdog_archivist) microservices. These endpoints
    44  are used to coordinate the microservice-managed aspects of the log stream
    45  lifecycle.
    46  
    47  ### Static
    48  
    49  The [static](static/) service hosts static content, including:
    50  * The LogDog Web Application
    51  * The LogDog Lightweight Stream Viewer
    52  * `rpcexplorer`
    53  
    54  ## Deployment
    55  
    56  Prefer to use [the deployment automation]. In particular, there's no other easy
    57  way to deploy `logs` module, since it uses a custom Docker image (because the
    58  latest GAE Flex Go runtime is stuck in the past on go1.15 version and we have to
    59  bring our own image to use a newer version).
    60  
    61  To deploy other modules from the local checkout, you can use `gae.py` tool.
    62  E.g. to update all modules other than `logs`:
    63  
    64  ```shell
    65  cd coordinator
    66  gae.py upload -A luci-logdog-dev default services static
    67  gae.py switch -A luci-logdog-dev
    68  ```
    69  
    70  [the deployment automation]: https://chrome-internal.googlesource.com/infradata/gae