github.com/yankunsam/loki/v2@v2.6.3-0.20220817130409-389df5235c27/docs/sources/configuration/query-frontend.md (about)

     1  ---
     2  title: Query Frontend
     3  ---
     4  # Kubernetes Query Frontend Example
     5  
     6  ## Disclaimer
     7  
     8  This aims to be a general purpose example; there are a number of substitutions to make for it to work correctly. These variables take the form of <variable_name>. You should override them with specifics to your environment.
     9  
    10  ## Use case
    11  
    12  It's a common occurrence to start running Grafana Loki as a single binary while trying it out in order to simplify deployments and defer learning the (initially unnecessary) nitty gritty details. As we become more comfortable with its paradigms and begin migrating towards a more production ready deployment there are a number of things to be aware of. A common bottleneck is on the read path: queries that executed effortlessly on small data sets may churn to a halt on larger ones. Sometimes we can solve this with more queriers. However, that doesn't help when our queries are too large for a single querier to execute. Then we need the query frontend.
    13  
    14  ### Parallelization
    15  
    16  One of the most important functions of the query frontend is the ability to split larger queries into smaller ones, execute them in parallel, and stitch the results back together. How often it splits them is determined by the `querier.split-queries-by-interval` flag or the yaml config `query_range.split_queriers_by_interval`. With this set to `1h`, the frontend will dissect a day long query into 24 one hour queries, distribute them to the queriers, and collect the results. This is immensely helpful in production environments as it not only allows us to perform larger queries via aggregation, but also evens the work distribution across queriers so that one or two are not stuck with impossibly large queries while others are left idle.
    17  
    18  ## Kubernetes Deployment
    19  
    20  ### ConfigMap
    21  
    22  Use this ConfigMap to get the benefits of query parallelisation and caching with the query-frontend component.
    23  
    24  ```yaml
    25  apiVersion: v1
    26  kind: ConfigMap
    27  metadata:
    28    name: loki-frontend
    29    namespace: <namespace>
    30  data:
    31    config.yaml: |
    32      # Disable the requirement that every request to Loki has a
    33      # X-Scope-OrgID header. `fake` will be substituted in instead.
    34      auth_enabled: false
    35  
    36      # We don't want the usual /api/prom prefix.
    37      http_prefix:
    38  
    39      server:
    40        http_listen_port: 3100
    41  
    42      query_range:
    43        # make queries more cache-able by aligning them with their step intervals
    44        align_queries_with_step: true
    45        max_retries: 5
    46        cache_results: true
    47  
    48        results_cache:
    49          cache:
    50            # We're going to use the in-process "FIFO" cache
    51            enable_fifocache: true
    52            fifocache:
    53              size: 1024
    54              validity: 24h
    55  
    56      limits_config:
    57        max_cache_freshness_per_query: '10m'
    58        # parallelize queries in 15min intervals
    59        split_queries_by_interval: 15m
    60  
    61      frontend:
    62        log_queries_longer_than: 5s
    63        downstream_url: http://querier.<namespace>.svc.cluster.local:3100
    64        compress_responses: true
    65  ```
    66  
    67  ### Frontend Service
    68  ```yaml
    69  apiVersion: v1
    70  kind: Service
    71  metadata:
    72    annotations:
    73    labels:
    74      name: query-frontend
    75    name: query-frontend
    76    namespace: <namespace>
    77  spec:
    78    ports:
    79    - name: query-frontend-http
    80      port: 3100
    81      protocol: TCP
    82      targetPort: 3100
    83    selector:
    84      name: query-frontend
    85    sessionAffinity: None
    86    type: ClusterIP
    87    ClusterIP: None
    88    publishNotReadyAddresses: true
    89  ```
    90  
    91  ### Frontend Deployment
    92  
    93  ```yaml
    94  apiVersion: extensions/v1beta1
    95  kind: Deployment
    96  metadata:
    97    labels:
    98      name: query-frontend
    99    name: query-frontend
   100    namespace: <namespace>
   101  spec:
   102    minReadySeconds: 10
   103    replicas: 2
   104    selector:
   105      matchLabels:
   106        name: query-frontend
   107    template:
   108      metadata:
   109        labels:
   110          name: query-frontend
   111      spec:
   112        containers:
   113        - args:
   114          - -config.file=/etc/loki/config.yaml
   115          - -log.level=debug
   116          - -target=query-frontend
   117          image: grafana/loki:latest
   118          imagePullPolicy: Always
   119          name: query-frontend
   120          ports:
   121          - containerPort: 3100
   122            name: http
   123            protocol: TCP
   124          resources:
   125            limits:
   126              memory: 1200Mi
   127            requests:
   128              cpu: "2"
   129              memory: 600Mi
   130          volumeMounts:
   131          - mountPath: /etc/loki
   132            name: loki-frontend
   133        restartPolicy: Always
   134        terminationGracePeriodSeconds: 30
   135        volumes:
   136        - configMap:
   137            defaultMode: 420
   138            name: loki-frontend
   139          name: loki-frontend
   140  ```
   141  
   142  ### Grafana
   143  
   144  Once you've deployed these, point your Grafana datasource to the new frontend service. The service is available within the cluster at `http://query-frontend.<namespace>.svc.cluster.local:3100`.
   145  
   146  ### GRPC Mode (Pull model)
   147  
   148  The query frontend operates in one of two ways:
   149  
   150  - Specify `--frontend.downstream-url` or its YAML equivalent, `frontend.downstream_url`. This proxies requests over HTTP to the specified URL.
   151  -  Without `--frontend.downstream-url` or its yaml equivalent `frontend.downstream_url`, the query frontend defaults to a pull service. As a pull service, the frontend instantiates per-tenant queues that downstream queriers pull queries from via GRPC. To act as a pull service, queriers need to specify `-querier.frontend-address` or its YAML equivalent `frontend_worker.frontend_address`.
   152  
   153      Set `ClusterIP=None` for the query frontend pull service.
   154      This causes DNS resolution of each query frontend pod IP address.
   155      It avoids wrongly resolving to the service IP.
   156  
   157      Enable `publishNotReadyAddresses=true` on the query frontend pull service.
   158      Doing so eliminates a race condition in which the query frontend address
   159      is needed before the query frontend becomes ready
   160      when at least one querier connects.