go.chromium.org/luci@v0.0.0-20240309015107-7cdc2e660f33/analysis/README.md (about)

     1  # LUCI Analysis
     2  
     3  LUCI Analysis is a system designed to understand and reduce the impact of test
     4  failures.
     5  
     6  ## Prerequisites
     7  
     8  Commands below assume you are running in the infra environment.
     9  To enter the infra env (via the infra.git checkout), run:
    10  ```
    11  eval infra/go/env.py
    12  ```
    13  
    14  ## Local Development flow
    15  
    16  To run the server locally, first authorize as the correct GCP project (you should only need to do this once):
    17  ```
    18  gcloud config set project luci-analysis-dev
    19  gcloud auth application-default login
    20  ```
    21  
    22  Authenticate in LUCI and in CIPD:
    23  
    24  1. In LUCI Analysis's `frontend` directory run:
    25     ```
    26     luci-auth login -scopes "https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/userinfo.email"
    27     ```
    28  2. In the same directory run:
    29     ```
    30     cipd auth-login
    31     ```
    32  
    33  Once the GCP project is authorized, in one terminal start esbuild to rebuild the UI code after any changes:
    34  
    35  ```
    36  cd frontend/ui
    37  npm run watch
    38  ```
    39  
    40  To run the server, in another terminal use:
    41  ```
    42  cd frontend
    43  go run main.go \
    44   -cloud-project luci-analysis-dev \
    45   -spanner-database projects/luci-analysis-dev/instances/dev/databases/luci-analysis-dev \
    46   -auth-service-host chrome-infra-auth-dev.appspot.com \
    47   -default-request-timeout 10m0s \
    48   -buganizer-mode disable \
    49   -config-local-dir ../configs \
    50   -encrypted-cookies-expose-state-endpoint
    51  ```
    52  
    53  `-default-request-timeout` is needed if exercising cron jobs through the admin
    54  portal as cron jobs run through the /admin/ endpoint attract the default
    55  timeout of 1 minute, instead of the 10 minute timeout of the /internal/ endpoint
    56  (hit by GAE cron jobs when they are actually executing).
    57  
    58  `-buganizer-mode` is needed when running cron jobs that create/update buganizer
    59  issues (e.g. update-analysis-and-bugs). Set the mode to `disable` to prevent the
    60  cron job from filing buganizer bugs.
    61  
    62  Note that `-config-local-dir` is required only if you plan on modifying config
    63  and loading it into Cloud Datastore via the read-config cron job accessible via
    64  http://127.0.0.1:8900/admin/portal/cron for testing. Omitting this, the server
    65  will fetch the current config from Cloud Datastore (as periodically refreshed
    66  from LUCI Config Service).
    67  
    68  You may also be able to use an arbitrary cloud project (e.g. 'dev') if you
    69  setup Cloud Datastore emulator and setup a config for that project under
    70  configs.
    71  
    72  ## Running UI tests
    73  
    74  You can run the UI tests by:
    75  ```
    76  cd frontend/ui
    77  npm run test
    78  ```
    79  
    80  ## Run Spanner integration tests using Cloud Spanner Emulator
    81  
    82  ### Install Cloud Spanner Emulator
    83  
    84  #### Linux
    85  
    86  The Cloud Spanner Emulator is part of the bundled gcloud, to make sure it's installed:
    87  
    88  ```
    89  cd infra
    90  gclient runhooks
    91  eval `./go/env.py`
    92  which gcloud # should show bundled gcloud
    93  gcloud components list # should see cloud-spanner-emulator is installed
    94  ```
    95  
    96  ### Run tests
    97  
    98  From command line, first set environment variables:
    99  
   100  ```
   101  export INTEGRATION_TESTS=1
   102  ```
   103  
   104  Then run go test as usual. For example:
   105  
   106  ```
   107  go test go.chromium.org/luci/analysis/...
   108  ```
   109  
   110  ## Run UI linter
   111  
   112  To run the UI code linter (note: this will also try to auto-fix issues), run:
   113  
   114  ```
   115  cd frontend/ui
   116  npm run lint
   117  ```
   118  
   119  To additionally auto-fix issues, run:
   120  
   121  ```
   122  cd frontend/ui
   123  npm run lint-fix
   124  ```
   125  
   126  ## Regenerating UI proto bindings
   127  
   128  You can regenerate typescript bindings for proto files by running:
   129  ```
   130  cd frontend/ui
   131  npm run gen_proto
   132  ```
   133  
   134  ## Deployment
   135  
   136  ### Developer Testing {#test-deployment}
   137  
   138  LUCI Analysis uses `gae.py` for deployment of the GAE instances for developer
   139  testing (e.g. of local changes).
   140  
   141  First, enter the infra env (via the infra.git checkout):
   142  ```
   143  eval infra/go/env.py
   144  ```
   145  
   146  Then use the following commands to deploy:
   147  ```
   148  cd frontend/ui
   149  npm run build
   150  gae.py upload --target-version ${USER} -A luci-analysis-dev
   151  ```
   152  
   153  ### Dev and Prod Instances
   154  
   155  The dev and prod instances are managed via
   156  [LUCI GAE Automatic Deployment (Googlers-only)](http://go/luci/how_to_deploy.md).
   157  
   158  #### Make a production release
   159  
   160  1. Make sure that you have an `infra_internal` checkout.
   161  2. Navigate to `data/gae` directory under your base checkout directory, should be `~/infra`.
   162  3. Pull the latest with `git rebase-update` or `git checkout main && git pull`.
   163  4. Create a new branch `git new-branch <NAME>` or `git -b <NAME>`.
   164  5. run `./scripts/promote.py luci-analysis --canary --stable --commit`.
   165  6. Upload the CL `git cl upload`.
   166  7. Request approval from space: LUCI Test War Room.
   167