github.com/m3db/m3@v1.5.0/scripts/comparator/README.md (about)

     1  # Query comparator
     2  
     3  This docker-compose file will setup the following environment:
     4  
     5  1. 1 M3Comparator node that acts as remote gRPC storage. Provides randomized data based on the incoming query's start time.
     6  2. 1 M3Query node that connects to the M3Comparator instance, using it as remote storage. Serves queries and remote reads.
     7  3. 1 Prometheus node that has no scrape settings set, connecting to M3Query instance as a remote_read endpoint.
     8  4. (optionally) 1 Grafana node with pre-configured graphs corresponding to the queries run by the test.
     9  
    10  ## Mechanism
    11  
    12  - Queries are generated from `queries.json`, then run against both the Prometheus and M3Query instances, then results are compared.
    13  
    14  ## Usage
    15  
    16  - Use `make docker-compatibility-test` from the base folder to run the comparator tests.
    17  - Use `CI=FALSE make docker-compatibility-test` from the base folder to run the comparator tests, brings up a Grafana instance and does not perform teardown, allowing manual inspection of query differences.
    18  
    19  ## Grafana
    20  
    21  Use Grafana by navigating to `http://localhost:3000` and using `admin` for both the username and password. The dashboard should already be populated and working, it should be named `Dashboard <git-reference>`.
    22  
    23  ## Comparator tests diagram
    24  
    25  ```
    26                      ┌───────────────────┐                            
    27                      │                   │                            
    28                      │ m3comparator      │                            
    29                      │                   │                            
    30                      │   1. random data  │                            
    31                      │   2. loaded data  │                            
    32                      │                   │ ┌───────────────┐          
    33                      └───────────────────┘ │   raw data    │          
    34                                ▲           │ (GRPC M3Query │          
    35                                │           │Remote Storage)│          
    36                                │           └───────────────┘          
    37                                └──────────────────────────┐           
    38                                                           │           
    39                                                           │           
    40                                                           │           
    41  ┌───────────────────────┐                     ┌─────────────────────┐
    42  │                       │                     │                     │
    43  │                       │                     │                     │
    44  │                       │                     │                     │
    45  │     prometheus        │────────────────────▶│      m3query        │
    46  │                       │  ┌────────────┐     │                     │
    47  │                       │  │  raw data  │     │                     │
    48  │                       │  │(Prom Remote│     │                     │
    49  └───────────────────────┘  │   Read)    │     └─────────────────────┘
    50              ▲              └────────────┘                ▲           
    51              │                                            │           
    52              │                                            │           
    53              │                                            │           
    54              └────────────────────┬───────────────────────┘           
    55      ┌───────────┐                │                ┌───────────┐      
    56      │   query   │                │                │   query   │      
    57      │ (PromQL)  │                │                │ (PromQL)  │      
    58      └───────────┘                │                └───────────┘      
    59                            ┌────────────┐                             
    60                            │            │                             
    61                            │ compare.go │                             
    62                            │            │                             
    63                            └────────────┘                             
    64  ```