github.com/web-platform-tests/wpt.fyi@v0.0.0-20240530210107-70cf978996f1/api/query/cache/README.md (about) 1 # Searchcache 2 3 This directory contains the implementation of the searchcache for wpt.fyi. The 4 searchcache runs as a separate service from the main dashboard, providing a 5 caching layer between the dashboard and the backing test runs datastore for 6 complex search queries. 7 8 ## Developing locally 9 10 It is possible to run the searchcache locally when developing, though it is 11 difficult to also have it talk to a local build of the dashboard. However 12 running the searchcache and then communicating with it via `curl` serves most 13 development needs. 14 15 ### Prerequisites 16 17 Outside of the standard setup for developing wpt.fyi (see the main 18 CONTRIBUTING.md file), searchcache development requires: 19 20 1. At least 4GB of RAM (8GB+ preferable). The searchcache loves RAM more than 21 Chrome does. 22 1. A GCP [service account](https://cloud.google.com/iam/docs/understanding-service-accounts) 23 with the role 'Cloud Datastore User' for the `wptdashboard-staging` project. 24 The key JSON file for this account must be available locally (called 25 `gcp.json` in this README). 26 27 ### Building 28 29 From the `service/` subdirectory, run: 30 31 ```sh 32 go build 33 ``` 34 35 This should create a new binary, `service`. 36 37 ### Running 38 39 **Do not run the searchcache binary without flags!**. It is configured to grab 40 pretty much all of the RAM on the machine it runs on - great for a VM, bad for 41 your development machine. 42 43 The following are required flags when running `service`: 44 45 * `--project_id=wptdashboard-staging` - set the project to the staging instance 46 of [wpt.fyi](https://wpt.fyi). 47 * `--gcp_credentials_file=gcp.json` - provide the necessary credentials to 48 access the datastore. See [Prerequisites](#prerequisites) above. 49 * `--max_heap_bytes=256000000` - sets a soft limit on memory usage. Emphasis on 50 'soft' - with a 256MB heap set here, the author had their searchcache 51 stabilize at approximately 2 **GB** of memory. 52 53 Putting it together: 54 55 ```sh 56 ./service \ 57 --project_id=wptdashboard-staging \ 58 --gcp_credentials_file=gcp.json \ 59 --max_heap_bytes=256000000 60 ``` 61 62 Other flags can be found by passing `--help` to the `service` binary. 63 64 **Tip**: The `service` binary is quite noisy (as it slurps up test runs from 65 the datastore); it can be useful to pipe its output via `tee` and store the log 66 locally to let you easily grep through it. 67 68 ### Interacting with searchcache 69 70 By default, searchcache is hosted on `localhost:8080`. It can be communicated 71 with using HTTP requests. For example, using `curl`: 72 73 ```sh 74 curl -H "Content-Type: application/json" \ 75 -X POST \ 76 -d '{"run_ids":[267810084, 255750007],"query":{"exists":[{"is":"different"}]}}' \ 77 http://localhost:8080/api/search/cache 78 ```