github.com/yankunsam/loki/v2@v2.6.3-0.20220817130409-389df5235c27/docs/sources/configuration/_index.md (about) 1 --- 2 title: Configuration 3 weight: 500 4 --- 5 # Configuring Grafana Loki 6 7 Grafana Loki is configured in a YAML file (usually referred to as `loki.yaml` ) 8 which contains information on the Loki server and its individual components, 9 depending on which mode Loki is launched in. 10 11 Configuration examples can be found in the [Configuration Examples](examples/) document. 12 13 ## Printing Loki Config At Runtime 14 15 If you pass Loki the flag `-print-config-stderr` or `-log-config-reverse-order`, (or `-print-config-stderr=true`) 16 Loki will dump the entire config object it has created from the built in defaults combined first with 17 overrides from config file, and second by overrides from flags. 18 19 The result is the value for every config object in the Loki config struct, which is very large... 20 21 Many values will not be relevant to your install such as storage configs which you are not using and which you did not define, 22 this is expected as every option has a default value if it is being used or not. 23 24 This config is what Loki will use to run, it can be invaluable for debugging issues related to configuration and 25 is especially useful in making sure your config files and flags are being read and loaded properly. 26 27 `-print-config-stderr` is nice when running Loki directly e.g. `./loki ` as you can get a quick output of the entire Loki config. 28 29 `-log-config-reverse-order` is the flag we run Loki with in all our environments, the config entries are reversed so 30 that the order of configs reads correctly top to bottom when viewed in Grafana's Explore. 31 32 ## Configuration File Reference 33 34 To specify which configuration file to load, pass the `-config.file` flag at the 35 command line. The value can be a list of comma separated paths, then the first 36 file that exists will be used. 37 If no `-config.file` argument is specified, Loki will look up the `config.yaml` in the 38 current working directory and the `config/` sub-directory and try to use that. 39 40 The file is written in [YAML 41 format](https://en.wikipedia.org/wiki/YAML), defined by the scheme below. 42 Brackets indicate that a parameter is optional. For non-list parameters the 43 value is set to the specified default. 44 45 ### Use environment variables in the configuration 46 47 > **Note:** This feature is only available in Loki 2.1+. 48 49 You can use environment variable references in the configuration file to set values that need to be configurable during deployment. 50 To do this, pass `-config.expand-env=true` and use: 51 52 ``` 53 ${VAR} 54 ``` 55 56 Where VAR is the name of the environment variable. 57 58 Each variable reference is replaced at startup by the value of the environment variable. 59 The replacement is case-sensitive and occurs before the YAML file is parsed. 60 References to undefined variables are replaced by empty strings unless you specify a default value or custom error text. 61 62 To specify a default value, use: 63 64 ``` 65 ${VAR:-default_value} 66 ``` 67 68 Where default_value is the value to use if the environment variable is undefined. 69 70 Pass the `-config.expand-env` flag at the command line to enable this way of setting configs. 71 72 ### Generic placeholders 73 74 - `<boolean>` : a boolean that can take the values `true` or `false` 75 - `<int>` : any integer matching the regular expression `[1-9]+[0-9]*` 76 - `<duration>` : a duration matching the regular expression `[0-9]+(ns|us|µs|ms|[smh])` 77 - `<labelname>` : a string matching the regular expression `[a-zA-Z_][a-zA-Z0-9_]*` 78 - `<labelvalue>` : a string of unicode characters 79 - `<filename>` : a valid path relative to current working directory or an absolute path. 80 - `<host>` : a valid string consisting of a hostname or IP followed by an optional port number 81 - `<string>` : a string 82 - `<secret>` : a string that represents a secret, such as a password 83 84 ### Supported contents and default values of `loki.yaml` 85 86 ```yaml 87 # A comma-separated list of components to run. 88 # The default value "all" runs Loki in single binary mode. 89 # The value "read" is an alias to run only read-path related components such as 90 # the querier and query-frontend, but all in the same process. 91 # The value "write" is an alias to run only write-path related components such as 92 # the distributor and compactor, but all in the same process. 93 # Supported values: all, compactor, distributor, ingester, querier, query-scheduler, 94 # ingester-querier, query-frontend, index-gateway, ruler, table-manager, read, write. 95 # A full list of available targets can be printed when running Loki with the 96 # `-list-targets` command line flag. 97 [target: <string> | default = "all"] 98 99 # Enables authentication through the X-Scope-OrgID header, which must be present 100 # if true. If false, the OrgID will always be set to "fake". 101 [auth_enabled: <boolean> | default = true] 102 103 # The amount of virtual memory in bytes to reserve as ballast in order to optimize 104 # garbage collection. Larger ballasts result in fewer garbage collection passes, reducing 105 # CPU overhead at the cost of heap size. The ballast will not consume physical memory, 106 # because it is never read from. It will, however, distort metrics, because it is 107 # counted as live memory. 108 [ballast_bytes: <int> | default = 0] 109 110 # Configures the server of the launched module(s). 111 [server: <server>] 112 113 # Configures the distributor. 114 [distributor: <distributor>] 115 116 # Configures the querier. Only appropriate when running all modules or 117 # just the querier. 118 [querier: <querier>] 119 120 # The query_scheduler block configures the Loki query scheduler. 121 # When configured it separates the tenant query queues from the query-frontend 122 [query_scheduler: <query_scheduler>] 123 124 # The frontend block configures the Loki query-frontend. 125 [frontend: <frontend>] 126 127 # The query_range block configures the query splitting and caching in the Loki 128 # query-frontend. 129 [query_range: <query_range>] 130 131 # The ruler block configures the Loki ruler. 132 [ruler: <ruler>] 133 134 # The ingester_client block configures how the distributor will connect 135 # to ingesters. Only appropriate when running all components, the distributor, 136 # or the querier. 137 [ingester_client: <ingester_client>] 138 139 # The ingester block configures the ingester and how the ingester will register itself 140 # to a key value store. 141 [ingester: <ingester>] 142 143 # Configures the index gateway server. 144 [index_gateway: <index_gateway>] 145 146 # Configures where Loki will store data. 147 [storage_config: <storage_config>] 148 149 # Configures how Loki will store data in the specific store. 150 [chunk_store_config: <chunk_store_config>] 151 152 # Configures the chunk index schema and where it is stored. 153 [schema_config: <schema_config>] 154 155 # The compactor block configures the compactor component, which compacts index shards 156 # for performance. 157 [compactor: <compactor>] 158 159 # Configures limits per-tenant or globally. 160 [limits_config: <limits_config>] 161 162 # The frontend_worker configures the worker - running within the Loki 163 # querier - picking up and executing queries enqueued by the query-frontend. 164 [frontend_worker: <frontend_worker>] 165 166 # The table_manager block configures the table manager for retention. 167 [table_manager: <table_manager>] 168 169 # Configuration for "runtime config" module, responsible for reloading runtime 170 # configuration file. 171 [runtime_config: <runtime_config>] 172 173 # Configuration for tracing. 174 [tracing: <tracing>] 175 176 # Common configuration to be shared between multiple modules. 177 # If a more specific configuration is given in other sections, 178 # the related configuration within this section will be ignored. 179 [common: <common>] 180 181 # Configuration for usage report 182 [analytics: <analytics>] 183 ``` 184 185 ## server 186 187 The `server` block 188 configures the HTTP and gRPC server communication of the launched service(s). 189 190 ```yaml 191 # HTTP server listen host 192 # CLI flag: -server.http-listen-address 193 [http_listen_address: <string>] 194 195 # HTTP server listen port 196 # CLI flag: -server.http-listen-port 197 [http_listen_port: <int> | default = 80] 198 199 # gRPC server listen host 200 # CLI flag: -server.grpc-listen-address 201 [grpc_listen_address: <string>] 202 203 # gRPC server listen port 204 # CLI flag: -server.grpc-listen-port 205 [grpc_listen_port: <int> | default = 9095] 206 207 # Register instrumentation handlers (/metrics, etc.) 208 # CLI flag: -server.register-instrumentation 209 [register_instrumentation: <boolean> | default = true] 210 211 # Timeout for graceful shutdowns 212 # CLI flag: -server.graceful-shutdown-timeout 213 [graceful_shutdown_timeout: <duration> | default = 30s] 214 215 # Read timeout for HTTP server 216 # CLI flag: -server.http-read-timeout 217 [http_server_read_timeout: <duration> | default = 30s] 218 219 # Write timeout for HTTP server 220 # CLI flag: -server.http-write-timeout 221 [http_server_write_timeout: <duration> | default = 30s] 222 223 # Idle timeout for HTTP server 224 # CLI flag: -server.http-idle-timeout 225 [http_server_idle_timeout: <duration> | default = 120s] 226 227 # Max gRPC message size that can be received 228 # CLI flag: -server.grpc-max-recv-msg-size-bytes 229 [grpc_server_max_recv_msg_size: <int> | default = 4194304] 230 231 # Max gRPC message size that can be sent 232 # CLI flag: -server.grpc-max-send-msg-size-bytes 233 [grpc_server_max_send_msg_size: <int> | default = 4194304] 234 235 # Limit on the number of concurrent streams for gRPC calls (0 = unlimited) 236 # CLI flag: -server.grpc-max-concurrent-streams 237 [grpc_server_max_concurrent_streams: <int> | default = 100] 238 239 # Log only messages with the given severity or above. Supported values [debug, 240 # info, warn, error] 241 # CLI flag: -log.level 242 [log_level: <string> | default = "info"] 243 244 # Base path to serve all API routes from (e.g., /v1/). 245 # CLI flag: -server.path-prefix 246 [http_path_prefix: <string> | default = ""] 247 ``` 248 249 ## distributor 250 251 The `distributor` block configures the distributor component. 252 253 ```yaml 254 # Configures the distributors ring, used when the "global" ingestion rate 255 # strategy is enabled. 256 ring: 257 kvstore: 258 # The backend storage to use for the ring. Supported values are 259 # consul, etcd, inmemory, memberlist 260 # CLI flag: -distributor.ring.store 261 store: <string> 262 263 # The prefix for the keys in the store. Should end with a /. 264 # CLI flag: -distributor.ring.prefix 265 [prefix: <string> | default = "collectors/"] 266 267 # Configuration for a Consul client. Only applies if store is "consul" 268 # The CLI flags prefix for this block config is: distributor.ring 269 [consul: <consul_config>] 270 271 # Configuration for an ETCD v3 client. Only applies if store is "etcd" 272 # The CLI flags prefix for this block config is: distributor.ring 273 [etcd: <etcd_config>] 274 275 # The heartbeat timeout after which ingesters are skipped for 276 # reading and writing. 277 # CLI flag: -distributor.ring.heartbeat-timeout 278 [heartbeat_timeout: <duration> | default = 1m] 279 ``` 280 281 ## querier 282 283 The `querier` block configures the Loki Querier. 284 285 ```yaml 286 # Timeout when querying ingesters or storage during the execution of a query request. 287 # CLI flag: -querier.query-timeout 288 [query_timeout: <duration> | default = 1m] 289 290 # Maximum duration for which the live tailing requests should be served. 291 # CLI flag: -querier.tail-max-duration 292 [tail_max_duration: <duration> | default = 1h] 293 294 # Time to wait before sending more than the minimum successful query requests. 295 # CLI flag: -querier.extra-query-delay 296 [extra_query_delay: <duration> | default = 0s] 297 298 # Maximum lookback beyond which queries are not sent to ingester. 299 # 0 means all queries are sent to ingester. 300 # CLI flag: -querier.query-ingesters-within 301 [query_ingesters_within: <duration> | default = 3h] 302 303 # The maximum number of concurrent queries allowed. 304 # CLI flag: -querier.max-concurrent 305 [max_concurrent: <int> | default = 10] 306 307 # Only query the store, and not attempt any ingesters. 308 # This is useful for running a standalone querier pool operating only against 309 # stored data. 310 # CLI flag: -querier.query-store-only 311 [query_store_only: <boolean> | default = false] 312 313 # When true, queriers only query the ingesters, and not stored data. 314 # This is useful when the object store is unavailable. 315 # CLI flag: -querier.query-ingester-only 316 [query_ingester_only: <boolean> | default = false] 317 318 # When true, allow queries to span multiple tenants. 319 # CLI flag: -querier.multi-tenant-queries-enabled 320 [multi_tenant_queries_enabled: <boolean> | default = false] 321 322 # Configuration options for the LogQL engine. 323 engine: 324 # Timeout for query execution 325 # CLI flag: -querier.engine.timeout 326 [timeout: <duration> | default = 3m] 327 328 # The maximum amount of time to look back for log lines. Only 329 # applicable for instant log queries. 330 # CLI flag: -querier.engine.max-lookback-period 331 [max_look_back_period: <duration> | default = 30s] 332 ``` 333 334 ## query_scheduler 335 336 The `query_scheduler` block configures the Loki query scheduler. 337 338 ```yaml 339 # Maximum number of outstanding requests per tenant per query-scheduler. 340 # In-flight requests above this limit will fail with HTTP response status code 341 # 429. 342 # CLI flag: -query-scheduler.max-outstanding-requests-per-tenant 343 [max_outstanding_requests_per_tenant: <int> | default = 100] 344 345 # If a querier disconnects without sending notification about graceful shutdown, 346 # the query-scheduler will keep the querier in the tenant's shard until the forget delay has passed. 347 # This feature is useful to reduce the blast radius when shuffle-sharding is enabled. 348 # CLI flag: -query-scheduler.querier-forget-delay 349 [querier_forget_delay: <duration> | default = 0] 350 351 # This configures the gRPC client used to report errors back to the 352 # query-frontend. 353 [grpc_client_config: <grpc_client_config>] 354 355 # Set to true to have the query schedulers create and place themselves in a ring. 356 # If no frontend_address or scheduler_address are present 357 # anywhere else in the configuration, Loki will toggle this value to true. 358 [use_scheduler_ring: <boolean> | default = false] 359 360 # The hash ring configuration. This option is required only if use_scheduler_ring is true 361 # The CLI flags prefix for this block config is scheduler.ring 362 [scheduler_ring: <ring>] 363 ``` 364 365 ## frontend 366 367 The `frontend` block configures the Loki query-frontend. 368 369 ```yaml 370 # Maximum number of outstanding requests per tenant per frontend; requests 371 # beyond this error with HTTP 429. 372 # CLI flag: -querier.max-outstanding-requests-per-tenant 373 [max_outstanding_per_tenant: <int> | default = 2048] 374 375 # In the event a tenant is repeatedly sending queries that lead the querier to crash 376 # or be killed due to an out-of-memory error, the crashed querier will be disconnected 377 # from the query frontend and a new querier will be immediately assigned to the tenant’s shard. 378 # This invalidates the assumption that shuffle sharding can be used to reduce the 379 # impact on tenants. This option mitigates the impact by configuring a delay between when 380 # a querier disconnects because of a crash and when the crashed querier is actually removed 381 # from the tenant's shard. 382 # CLI flag: -query-frontend.querier-forget-delay 383 [querier_forget_delay: <duration> | default = 0s] 384 385 # Compress HTTP responses. 386 # CLI flag: -querier.compress-http-responses 387 [compress_responses: <boolean> | default = false] 388 389 # URL of downstream Loki. 390 # CLI flag: -frontend.downstream-url 391 [downstream_url: <string> | default = ""] 392 393 # Log queries that are slower than the specified duration. Set to 0 to disable. 394 # Set to < 0 to enable on all queries. 395 # CLI flag: -frontend.log-queries-longer-than 396 [log_queries_longer_than: <duration> | default = 0s] 397 398 # URL of querier for tail proxy. 399 # CLI flag: -frontend.tail-proxy-url 400 [tail_proxy_url: <string> | default = ""] 401 402 tail_tls_config: 403 # Path to the client certificate file, which will be used for authenticating 404 # with the server. Also requires the key path to be configured. 405 # CLI flag: -frontend.tail-tls-config.tls-cert-path 406 [tls_cert_path: <string> | default = ""] 407 408 # Path to the key file for the client certificate. Also requires the client 409 # certificate to be configured. 410 # CLI flag: -frontend.tail-tls-config.tls-key-path 411 [tls_key_path: <string> | default = ""] 412 413 # Path to the CA certificates file to validate server certificate against. If 414 # not set, the host's root CA certificates are used. 415 # CLI flag: -frontend.tail-tls-config.tls-ca-path 416 [tls_ca_path: <string> | default = ""] 417 418 # Skip validating server certificate. 419 # CLI flag: -frontend.tail-tls-config.tls-insecure-skip-verify 420 [tls_insecure_skip_verify: <boolean> | default = false] 421 422 423 # DNS hostname used for finding query-schedulers. 424 # CLI flag: -frontend.scheduler-address 425 [scheduler_address: <string> | default = ""] 426 427 # How often to resolve the scheduler-address, in order to look for new 428 # query-scheduler instances. 429 # Also used to determine how often to poll the scheduler-ring for addresses if configured. 430 # CLI flag: -frontend.scheduler-dns-lookup-period 431 [scheduler_dns_lookup_period: <duration> | default = 10s] 432 433 # Number of concurrent workers forwarding queries to single query-scheduler. 434 # CLI flag: -frontend.scheduler-worker-concurrency 435 [scheduler_worker_concurrency: <int> | default = 5] 436 ``` 437 438 ## query_range 439 440 The `query_range` block configures query splitting and caching in the Loki query-frontend. 441 442 ```yaml 443 # Deprecated: Split queries by day and execute in parallel. 444 # Use -querier.split-queries-by-interval instead. 445 # CLI flag: -querier.split-queries-by-day 446 [split_queries_by_day: <boolean> | default = false] 447 448 # Mutate incoming queries to align their start and end with their step. 449 # CLI flag: -querier.align-querier-with-step 450 [align_queries_with_step: <boolean> | default = false] 451 452 results_cache: 453 # The CLI flags prefix for this block config is: frontend 454 cache: <cache_config> 455 456 # Cache query results. 457 # CLI flag: -querier.cache-results 458 [cache_results: <boolean> | default = false] 459 460 # Maximum number of retries for a single request; beyond this, the downstream 461 # error is returned. 462 # CLI flag: -querier.max-retries-per-request 463 [max_retries: <int> | default = 5] 464 465 # Perform query parallelisations based on storage sharding configuration and 466 # query ASTs. This feature is supported only by the chunks storage engine. 467 # CLI flag: -querier.parallelise-shardable-queries 468 [parallelise_shardable_queries: <boolean> | default = true] 469 ``` 470 471 ## ruler 472 473 The `ruler` block configures the Loki ruler. 474 475 ```yaml 476 # URL of alerts return path. 477 # CLI flag: -ruler.external.url 478 [external_url: <url> | default = ] 479 480 # Labels to add to all alerts 481 external_labels: 482 [<labelname>: <labelvalue> ...] 483 484 ruler_client: 485 # Path to the client certificate file, which will be used for authenticating 486 # with the server. Also requires the key path to be configured. 487 # CLI flag: -ruler.client.tls-cert-path 488 [tls_cert_path: <string> | default = ""] 489 490 # Path to the key file for the client certificate. Also requires the client 491 # certificate to be configured. 492 # CLI flag: -ruler.client.tls-key-path 493 [tls_key_path: <string> | default = ""] 494 495 # Path to the CA certificates file to validate server certificate against. If 496 # not set, the host's root CA certificates are used. 497 # CLI flag: -ruler.client.tls-ca-path 498 [tls_ca_path: <string> | default = ""] 499 500 # Skip validating server certificate. 501 # CLI flag: -ruler.client.tls-insecure-skip-verify 502 [tls_insecure_skip_verify: <boolean> | default = false] 503 504 # How frequently to evaluate rules. 505 # CLI flag: -ruler.evaluation-interval 506 [evaluation_interval: <duration> | default = 1m] 507 508 # How frequently to poll for rule changes. 509 # CLI flag: -ruler.poll-interval 510 [poll_interval: <duration> | default = 1m] 511 512 storage: 513 # Method to use for backend rule storage (azure, gcs, s3, swift, local, bos). 514 # CLI flag: -ruler.storage.type 515 [type: <string> ] 516 517 # Configures backend rule storage for Azure. 518 [azure: <azure_storage_config>] 519 520 # Configures backend rule storage for GCS. 521 [gcs: <gcs_storage_config>] 522 523 # Configures backend rule storage for S3. 524 [s3: <s3_storage_config>] 525 526 # Configures backend rule storage for Swift. 527 [swift: <swift_storage_config>] 528 529 # Configures backend rule storage for a local file system directory. 530 [local: <local_storage_config>] 531 532 # Configures backend rule storage for Baidu Object Storage (BOS). 533 [bos: <bos_storage_config>] 534 535 # The `hedging` block configures how to hedge storage requests. 536 [hedging: <hedging>] 537 538 # Remote-write configuration to send rule samples to a Prometheus remote-write endpoint. 539 remote_write: 540 # Enable remote-write functionality. 541 # CLI flag: -ruler.remote-write.enabled 542 [enabled: <boolean> | default = false] 543 # Minimum period to wait between refreshing remote-write reconfigurations. 544 # This should be greater than or equivalent to -limits.per-user-override-period. 545 [config_refresh_period: <duration> | default = 10s] 546 547 client: 548 # The URL of the endpoint to send samples to. 549 url: <string> 550 551 # Timeout for requests to the remote write endpoint. 552 [remote_timeout: <duration> | default = 30s] 553 554 # Custom HTTP headers to be sent along with each remote write request. 555 # Be aware that headers that are set by Loki itself can't be overwritten. 556 headers: 557 [<string>: <string> ...] 558 559 # List of remote write relabel configurations. 560 write_relabel_configs: 561 [- <relabel_config> ...] 562 563 # Name of the remote write config, which if specified must be unique among remote 564 # write configs. 565 # The name will be used in metrics and logging in place of a generated value 566 # to help users distinguish between remote write configs. 567 [name: <string>] 568 569 # Sets the `Authorization` header on every remote write request with the 570 # configured username and password. 571 # password and password_file are mutually exclusive. 572 basic_auth: 573 [username: <string>] 574 [password: <secret>] 575 [password_file: <string>] 576 577 # Optional `Authorization` header configuration. 578 authorization: 579 # Sets the authentication type. 580 [type: <string> | default: Bearer] 581 # Sets the credentials. It is mutually exclusive with 582 # `credentials_file`. 583 [credentials: <secret>] 584 # Sets the credentials to the credentials read from the configured file. 585 # It is mutually exclusive with `credentials`. 586 [credentials_file: <filename>] 587 588 # Optionally configures AWS's Signature Verification 4 signing process to 589 # sign requests. Cannot be set at the same time as basic_auth, authorization, or oauth2. 590 # To use the default credentials from the AWS SDK, use `sigv4: {}`. 591 [sigv4: <sigv4_config>] 592 593 # Configures the remote write request's TLS settings. 594 tls_config: 595 # CA certificate to validate API server certificate with. 596 [ca_file: <filename>] 597 # Certificate and key files for client cert authentication to the server. 598 [cert_file: <filename>] 599 [key_file: <filename>] 600 # ServerName extension to indicate the name of the server. 601 # https://tools.ietf.org/html/rfc4366#section-3.1 602 [server_name: <string>] 603 # Disable validation of the server certificate. 604 [insecure_skip_verify: <boolean>] 605 606 # Optional proxy URL. 607 [proxy_url: <string>] 608 609 # Configure whether HTTP requests follow HTTP 3xx redirects. 610 [follow_redirects: <boolean> | default = true] 611 612 # Configures the queue used to write to remote storage. 613 queue_config: 614 # Number of samples to buffer per shard before we block reading of more 615 # samples from the WAL. It is recommended to have enough capacity in each 616 # shard to buffer several requests to keep throughput up while processing 617 # occasional slow remote requests. 618 [capacity: <int> | default = 2500] 619 # Maximum number of shards, i.e. amount of concurrency. 620 [max_shards: <int> | default = 200] 621 # Minimum number of shards, i.e. amount of concurrency. 622 [min_shards: <int> | default = 1] 623 # Maximum number of samples per send. 624 [max_samples_per_send: <int> | default = 500] 625 # Maximum time a sample will wait in buffer. 626 [batch_send_deadline: <duration> | default = 5s] 627 # Initial retry delay. Gets doubled for every retry. 628 [min_backoff: <duration> | default = 30ms] 629 # Maximum retry delay. 630 [max_backoff: <duration> | default = 100ms] 631 # Retry upon receiving a 429 status code from the remote-write storage. 632 # This is experimental and might change in the future. 633 [retry_on_http_429: <boolean> | default = false] 634 635 wal: 636 # The directory in which to write tenant WAL files. Each tenant will have its own 637 # directory one level below this directory. 638 [dir: <string> | default = "ruler-wal"] 639 # Frequency with which to run the WAL truncation process. 640 [truncate_frequency: <duration> | default = 60m] 641 # Minimum and maximum time series should exist in the WAL for. 642 [min_age: <duration> | default = 5m] 643 [max_age: <duration> | default = 4h] 644 645 wal_cleaner: 646 # The minimum age of a WAL to consider for cleaning. 647 [min_age: <duration> | default = 12h] 648 # How often to run the WAL cleaner. 649 [period: <duration> | default = 0s (disabled)] 650 651 # File path to store temporary rule files. 652 # CLI flag: -ruler.rule-path 653 [rule_path: <filename> | default = "/rules"] 654 655 # Comma-separated list of Alertmanager URLs to send notifications to. 656 # Each Alertmanager URL is treated as a separate group in the configuration. 657 # Multiple Alertmanagers in HA per group can be supported by using DNS 658 # resolution via -ruler.alertmanager-discovery. 659 # CLI flag: -ruler.alertmanager-url 660 [alertmanager_url: <string> | default = ""] 661 662 663 alertmanager_client: 664 # Sets the `Authorization` header on every remote write request with the 665 # configured username and password. 666 # password and password_file are mutually exclusive. 667 basic_auth: 668 [username: <string>] 669 [password: <secret>] 670 671 # Optional `Authorization` header configuration. 672 authorization: 673 # Sets the authentication type. 674 [type: <string> | default: Bearer] 675 # Sets the credentials. It is mutually exclusive with 676 # `credentials_file`. 677 [credentials: <secret>] 678 # Sets the credentials to the credentials read from the configured file. 679 # It is mutually exclusive with `credentials`. 680 [credentials_file: <filename>] 681 682 # Use DNS SRV records to discover Alertmanager hosts. 683 # CLI flag: -ruler.alertmanager-discovery 684 [enable_alertmanager_discovery: <boolean> | default = false] 685 686 # How long to wait between refreshing DNS resolutions of Alertmanager hosts. 687 # CLI flag: -ruler.alertmanager-refresh-interval 688 [alertmanager_refresh_interval: <duration> | default = 1m] 689 690 # If enabled, then requests to Alertmanager use the v2 API. 691 # CLI flag: -ruler.alertmanager-use-v2 692 [enable_alertmanager_v2: <boolean> | default = false] 693 694 # List of alert relabel configs 695 alert_relabel_configs: 696 [- <relabel_config> ...] 697 698 # Capacity of the queue for notifications to be sent to the Alertmanager. 699 # CLI flag: -ruler.notification-queue-capacity 700 [notification_queue_capacity: <int> | default = 10000] 701 702 # HTTP timeout duration when sending notifications to the Alertmanager. 703 # CLI flag: -ruler.notification-timeout 704 [notification_timeout: <duration> | default = 10s] 705 706 # Max time to tolerate outage for restoring "for" state of alert. 707 # CLI flag: -ruler.for-outage-tolerance 708 [for_outage_tolerance: <duration> | default = 1h] 709 710 # Minimum duration between alert and restored "for" state. This is maintained 711 # only for alerts with configured "for" time greater than the grace period. 712 # CLI flag: -ruler.for-grace-period 713 [for_grace_period: <duration> | default = 10m] 714 715 # Minimum amount of time to wait before resending an alert to Alertmanager. 716 # CLI flag: -ruler.resend-delay 717 [resend_delay: <duration> | default = 1m] 718 719 # Distribute rule evaluation using ring backend. 720 # CLI flag: -ruler.enable-sharding 721 [enable_sharding: <boolean> | default = false] 722 723 # Time to spend searching for a pending ruler when shutting down. 724 # CLI flag: -ruler.search-pending-for 725 [search_pending_for: <duration> | default = 5m] 726 727 # Ring used by Loki ruler. 728 # The CLI flags prefix for this block config is ruler.ring 729 ring: 730 kvstore: 731 # Backend storage to use for the ring. Supported values are: consul, etcd, 732 # inmemory, memberlist, multi. 733 # CLI flag: -<prefix>.store 734 [store: <string> | default = "memberlist"] 735 736 # The prefix for the keys in the store. Should end with a /. 737 # CLI flag: -<prefix>.prefix 738 [prefix: <string> | default = "collectors/"] 739 740 # The consul_config configures the consul client. 741 [consul: <consul_config>] 742 743 # The etcd_config configures the etcd client. 744 [etcd: <etcd_config>] 745 746 multi: 747 # Primary backend storage used by multi-client. 748 # CLI flag: -<prefix>.multi.primary 749 [primary: <string> | default = ""] 750 751 # Secondary backend storage used by multi-client. 752 # CLI flag: -<prefix>.multi.secondary 753 [secondary: <string> | default = ""] 754 755 # Mirror writes to secondary store. 756 # CLI flag: -<prefix>.multi.mirror-enabled 757 [mirror_enabled: <boolean> | default = false] 758 759 # Timeout for storing value to secondary store. 760 # CLI flag: -<prefix>.multi.mirror-timeout 761 [mirror_timeout: <duration> | default = 2s] 762 763 # Interval between heartbeats sent to the ring. 0 = disabled. 764 # CLI flag: -<prefix>.heartbeat-period 765 [heartbeat_period: <duration> | default = 15s] 766 767 # The heartbeat timeout after which ruler ring members are considered unhealthy 768 # within the ring. 0 = never (timeout disabled). 769 # CLI flag: -<prefix>.heartbeat-timeout 770 [heartbeat_timeout: <duration> | default = 1m] 771 772 # Name of network interface to read addresses from. 773 # CLI flag: -<prefix>.instance-interface-names 774 [instance_interface_names: <list of string> | default = [<private network interfaces>]] 775 776 # The number of tokens the lifecycler will generate and put into the ring if 777 # it joined without transferring tokens from another lifecycler. 778 # CLI flag: -<prefix>.num-tokens 779 [num_tokens: <int> | default = 128] 780 ``` 781 782 ## bos_storage_config 783 784 The `bos_storage_config` block configures Baidu Object Storage (BOS) as general storage for data generated by Loki. 785 786 ```yaml 787 # Name of BOS bucket. 788 # CLI flag: <prefix>.baidubce.bucket-name 789 [ bucket_name: <string> | default = "" ] 790 # BOS endpoint to connect to. 791 # CLI flag: <prefix>.baidubce.endpoint 792 [ endpoint: <string> | default = "bj.bcebos.com" ] 793 # Baidu Cloud Engine (BCE) Access Key ID 794 # CLI flag: <prefix>.baidubce.access-key-id 795 [ access_key_id: <string> | default = "" ] 796 # BCE Secret Access Key 797 # CLI flag: <prefix>.baidubce.secret-access-key 798 [ secret_access_key: <string> | default = "" ] 799 ``` 800 801 ## azure_storage_config 802 803 The `azure_storage_config` configures Azure as a general storage for different data generated by Loki. 804 805 ```yaml 806 # Azure Cloud environment. Supported values are: AzureGlobal, 807 # AzureChinaCloud, AzureGermanCloud, AzureUSGovernment. 808 # CLI flag: -<prefix>.azure.environment 809 [environment: <string> | default = "AzureGlobal"] 810 811 # Azure storage account name. 812 # CLI flag: -<prefix>.azure.account-name 813 [account_name: <string> | default = ""] 814 815 # Azure storage account key. 816 # CLI flag: -<prefix>.azure.account-key 817 [account_key: <string> | default = ""] 818 819 # Name of the storage account blob container used to store chunks. 820 # This container must be created before running Loki. 821 # CLI flag: -<prefix>.azure.container-name 822 [container_name: <string> | default = "loki"] 823 824 # Azure storage endpoint suffix without schema. The storage account name will 825 # be prefixed to this value to create the FQDN. 826 # CLI flag: -<prefix>.azure.endpoint-suffix 827 [endpoint_suffix: <string> | default = ""] 828 829 # Use Managed Identity to authenticate to the Azure storage account. 830 # CLI flag: -<prefix>.azure.use-managed-identity 831 [use_managed_identity: <boolean> | default = false] 832 833 # User assigned identity ID to authenticate to the Azure storage account. 834 # CLI flag: -<prefix>.azure.user-assigned-id 835 [user_assigned_id: <string> | default = ""] 836 837 # Chunk delimiter to build the blobID 838 # CLI flag: -<prefix>.azure.chunk-delimiter 839 [chunk_delimiter: <string> | default = "-"] 840 841 # Preallocated buffer size for downloads. 842 # CLI flag: -<prefix>.azure.download-buffer-size 843 [download_buffer_size: <int> | default = 512000] 844 845 # Preallocated buffer size for uploads. 846 # CLI flag: -<prefix>.azure.upload-buffer-size 847 [upload_buffer_size: <int> | default = 256000] 848 849 # Number of buffers used to used to upload a chunk. 850 # CLI flag: -<prefix>.azure.download-buffer-count 851 [upload_buffer_count: <int> | default = 1] 852 853 # Timeout for requests made against azure blob storage. 854 # CLI flag: -<prefix>.azure.request-timeout 855 [request_timeout: <duration> | default = 30s] 856 857 # Number of retries for a request which times out. 858 # CLI flag: -<prefix>.azure.max-retries 859 [max_retries: <int> | default = 5] 860 861 # Minimum time to wait before retrying a request. 862 # CLI flag: -<prefix>.azure.min-retry-delay 863 [min_retry_delay: <duration> | default = 10ms] 864 865 # Maximum time to wait before retrying a request. 866 # CLI flag: -<prefix>.azure.max-retry-delay 867 [max_retry_delay: <duration> | default = 500ms] 868 ``` 869 870 ## gcs_storage_config 871 872 The `gcs_storage_config` configures GCS as a general storage for different data generated by Loki. 873 874 ```yaml 875 # Name of GCS bucket to put chunks in. 876 # CLI flag: -<prefix>.gcs.bucketname 877 [bucket_name: <string> | default = ""] 878 879 # Service account key content in JSON format. 880 # CLI flag: -<prefix>.gcs.service-account 881 [service_account: <string> | default = ""] 882 883 # The size of the buffer that GCS client for each PUT request. 0 to disable 884 # buffering. 885 # CLI flag: -<prefix>.gcs.chunk-buffer-size 886 [chunk_buffer_size: <int> | default = 0] 887 888 # The duration after which the requests to GCS should be timed out. 889 # CLI flag: -<prefix>.gcs.request-timeout 890 [request_timeout: <duration> | default = 0s] 891 892 # Enable HTTP/2 when connecting to GCS. This configuration only applies to GET operations. 893 # CLI flag: -<prefix>.gcs.enable-http2 894 [enable_http2: <boolean> | default = true] 895 ``` 896 897 ## s3_storage_config 898 899 The `s3_storage_config` configures S3 as a general storage for different data generated by Loki. 900 901 ```yaml 902 # S3 endpoint URL with escaped Key and Secret encoded. If only region is 903 # specified as a host, proper endpoint will be deduced. Use 904 # inmemory:///<bucket-name> to use a mock in-memory implementation. 905 # CLI flag: -<prefix>.s3.url 906 [s3: <url> | default = ] 907 908 # Set this to `true` to force the request to use path-style addressing. 909 # CLI flag: -<prefix>.s3.force-path-style 910 [s3forcepathstyle: <boolean> | default = false] 911 912 # Comma separated list of bucket names to evenly distribute chunks over. 913 # Overrides any buckets specified in s3.url flag 914 # CLI flag: -<prefix>.s3.buckets 915 [bucketnames: <string> | default = ""] 916 917 # S3 Endpoint to connect to. 918 # CLI flag: -<prefix>.s3.endpoint 919 [endpoint: <string> | default = ""] 920 921 # AWS region to use. 922 # CLI flag: -<prefix>.s3.region 923 [region: <string> | default = ""] 924 925 # AWS Access Key ID 926 # CLI flag: -<prefix>.s3.access-key-id 927 [access_key_id: <string> | default = ""] 928 929 # AWS Secret Access Key 930 # CLI flag: -<prefix>.s3.secret-access-key 931 [secret_access_key: <string> | default = ""] 932 933 # Disable https on S3 connection. 934 # CLI flag: -<prefix>.s3.insecure 935 [insecure: <boolean> | default = false] 936 937 # Enable AES256 AWS server-side encryption 938 # CLI flag: -<prefix>.s3.sse-encryption 939 [sse_encryption: <boolean> | default = false] 940 941 http_config: 942 # The maximum amount of time an idle connection will be held open. 943 # CLI flag: -<prefix>.s3.http.idle-conn-timeout 944 [idle_conn_timeout: <duration> | default = 1m30s] 945 946 # If non-zero, specifies the amount of time to wait for a server's 947 # response headers after fully writing the request. 948 # CLI flag: -<prefix>.s3.http.response-header-timeout 949 [response_header_timeout: <duration> | default = 0s] 950 951 # Set to true to skip verifying the certificate chain and hostname. 952 # CLI flag: -<prefix>.s3.http.insecure-skip-verify 953 [insecure_skip_verify: <boolean> | default = false] 954 955 # Path to the trusted CA file that signed the SSL certificate of the S3 956 # endpoint. 957 # CLI flag: -<prefix>.s3.http.ca-file 958 [ca_file: <string> | default = ""] 959 ``` 960 961 ## swift_storage_config 962 963 The `swift_storage_config` configures Swift as a general storage for different data generated by Loki. 964 965 ```yaml 966 # Openstack authentication URL. 967 # CLI flag: -<prefix>.swift.auth-url 968 [auth_url: <string> | default = ""] 969 970 # Openstack username for the api. 971 # CLI flag: -<prefix>.swift.username 972 [username: <string> | default = ""] 973 974 # Openstack user's domain name. 975 # CLI flag: -<prefix>.swift.user-domain-name 976 [user_domain_name: <string> | default = ""] 977 978 # Openstack user's domain ID. 979 # CLI flag: -<prefix>.swift.user-domain-id 980 [user_domain_id: <string> | default = ""] 981 982 # Openstack user ID for the API. 983 # CLI flag: -<prefix>.swift.user-id 984 [user_id: <string> | default = ""] 985 986 # Openstack API key. 987 # CLI flag: -<prefix>.swift.password 988 [password: <string> | default = ""] 989 990 # Openstack user's domain ID. 991 # CLI flag: -<prefix>.swift.domain-id 992 [domain_id: <string> | default = ""] 993 994 # Openstack user's domain name. 995 # CLI flag: -<prefix>.swift.domain-name 996 [domain_name: <string> | default = ""] 997 998 # Openstack project ID (v2,v3 auth only). 999 # CLI flag: -<prefix>.swift.project-id 1000 [project_id: <string> | default = ""] 1001 1002 # Openstack project name (v2,v3 auth only). 1003 # CLI flag: -<prefix>.swift.project-name 1004 [project_name: <string> | default = ""] 1005 1006 # ID of the project's domain (v3 auth only), only needed if it differs the 1007 # from user domain. 1008 # CLI flag: -<prefix>.swift.project-domain-id 1009 [project_domain_id: <string> | default = ""] 1010 1011 # Name of the project's domain (v3 auth only), only needed if it differs 1012 # from the user domain. 1013 # CLI flag: -<prefix>.swift.project-domain-name 1014 [project_domain_name: <string> | default = ""] 1015 1016 # Openstack Region to use eg LON, ORD - default is use first region (v2,v3 1017 # auth only) 1018 # CLI flag: -<prefix>.swift.region-name 1019 [region_name: <string> | default = ""] 1020 1021 # Name of the Swift container to put chunks in. 1022 # CLI flag: -<prefix>.swift.container-name 1023 [container_name: <string> | default = ""] 1024 ``` 1025 1026 ## hedging 1027 1028 The `hedging` block configures how to hedge storage requests. 1029 1030 The hedging implementation sends a second storage request once a first request has 1031 been outstanding for more than a configured expected latency for this class of requests. 1032 Calculate your latency to be the 99th percentile of object storage response times. 1033 1034 ```yaml 1035 # An optional duration that sets the quantity of time after a first storage request 1036 # is sent and before a second request is sent, when no response is received for the first 1037 # storage request. The recommended duration is the measured 99th percentile of object 1038 # storage response times, to reduce long tail latency. This option is most impactful 1039 # when used with queriers, and has minimal to no impact on other components. 1040 # The default value of 0 disables the hedging of storage requests. 1041 # Example: "at: 500ms" 1042 [at: <duration> | default = 0] 1043 1044 # An optional maximum quantity of hedged requests to be issued for a given request. 1045 [up_to: <int> | default = 2] 1046 1047 # Caps the rate of hedged requests by optionally defining the maximum quantity of 1048 # hedged requests issued per second. 1049 [max_per_second: <int> | default = 5] 1050 ``` 1051 1052 ## local_storage_config 1053 1054 The `local_storage_config` configures a (local) file system as a general storage for different data generated by Loki. 1055 1056 ```yaml 1057 # Filesystem directory to be used as storage. 1058 # CLI flag: -<prefix>.local.directory 1059 [directory: <filename> | default = ""] 1060 ``` 1061 1062 ## frontend_worker 1063 1064 The `frontend_worker` configures the worker - running within the Loki querier - picking up and executing queries enqueued by the query-frontend. 1065 1066 ```yaml 1067 # Address of query frontend service, in host:port format. 1068 # CLI flag: -querier.frontend-address 1069 [frontend_address: <string> | default = ""] 1070 1071 # Number of simultaneous queries to process. 1072 # CLI flag: -querier.worker-parallelism 1073 [parallelism: <int> | default = 10] 1074 1075 # Force worker concurrency to match the -querier.max-concurrent option. Overrides querier.worker-parallelism. 1076 # CLI flag: -querier.worker-match-max-concurrent 1077 [match_max_concurrent: <boolean> | default = true] 1078 1079 # How often to query the frontend_address DNS to resolve frontend addresses. 1080 # Also used to determine how often to poll the scheduler-ring for addresses if configured. 1081 # CLI flag: -querier.dns-lookup-period 1082 [dns_lookup_duration: <duration> | default = 3s] 1083 1084 # The CLI flags prefix for this block config is: querier.frontend-client 1085 [grpc_client_config: <grpc_client_config>] 1086 1087 # DNS hostname used for finding query-schedulers. 1088 # CLI flag: -querier.scheduler-address 1089 [scheduler_address: <string> | default = ""] 1090 ``` 1091 1092 ## ingester_client 1093 1094 The `ingester_client` block configures how connections to ingesters 1095 operate. 1096 1097 ```yaml 1098 # Configures how connections are pooled 1099 pool_config: 1100 # Whether or not to do health checks. 1101 # CLI flag: -distributor.health-check-ingesters 1102 [health_check_ingesters: <boolean> | default = false] 1103 1104 # How frequently to clean up clients for servers that have gone away after 1105 # a health check. 1106 # CLI flag: -distributor.client-cleanup-period 1107 [client_cleanup_period: <duration> | default = 15s] 1108 1109 # How quickly a dead client will be removed after it has been detected 1110 # to disappear. Set this to a value to allow time for a secondary 1111 # health check to recover the missing client. 1112 # CLI flag: -ingester.client.healthcheck-timeout 1113 [remote_timeout: <duration> | default = 1s] 1114 1115 # The remote request timeout on the client side. 1116 # CLI flag: -ingester.client.timeout 1117 [remote_timeout: <duration> | default = 5s] 1118 1119 # Configures how the gRPC connection to ingesters work as a client 1120 # The CLI flags prefix for this block config is: ingester.client 1121 [grpc_client_config: <grpc_client_config>] 1122 ``` 1123 1124 ## ingester 1125 1126 The `ingester` block configures the Loki Ingesters. 1127 1128 ```yaml 1129 # Configures how the lifecycle of the ingester will operate 1130 # and where it will register for discovery. 1131 lifecycler: 1132 ring: 1133 kvstore: 1134 # Backend storage to use for the ring. Supported values are: consul, etcd, 1135 # inmemory, memberlist 1136 # CLI flag: -ring.store 1137 [store: <string> | default = "consul"] 1138 1139 # The prefix for the keys in the store. Should end with a /. 1140 # CLI flag: -ring.prefix 1141 [prefix: <string> | default = "collectors/"] 1142 1143 # The consul_config configures the consul client. 1144 # CLI flag: <no prefix> 1145 [consul: <consul_config>] 1146 1147 # The etcd_config configures the etcd client. 1148 # CLI flag: <no prefix> 1149 [etcd: <etcd_config>] 1150 1151 # The heartbeat timeout after which ingesters are skipped for reads/writes. 1152 # CLI flag: -ring.heartbeat-timeout 1153 [heartbeat_timeout: <duration> | default = 1m] 1154 1155 # The number of ingesters to write to and read from. 1156 # CLI flag: -distributor.replication-factor 1157 [replication_factor: <int> | default = 3] 1158 1159 # The number of tokens the lifecycler will generate and put into the ring if 1160 # it joined without transferring tokens from another lifecycler. 1161 # CLI flag: -ingester.num-tokens 1162 [num_tokens: <int> | default = 128] 1163 1164 # Period at which to heartbeat to the underlying ring. 1165 # CLI flag: -ingester.heartbeat-period 1166 [heartbeat_period: <duration> | default = 5s] 1167 1168 # How long to wait to claim tokens and chunks from another member when 1169 # that member is leaving. Will join automatically after the duration expires. 1170 # CLI flag: -ingester.join-after 1171 [join_after: <duration> | default = 0s] 1172 1173 # Observe tokens after generating to resolve collisions. Useful when using a gossip ring. 1174 # CLI flag: -ingester.observe-period 1175 [observe_period: <duration> | default = 0s] 1176 1177 # Minimum duration to wait before becoming ready. This is to work around race 1178 # conditions with ingesters exiting and updating the ring. 1179 # CLI flag: -ingester.min-ready-duration 1180 [min_ready_duration: <duration> | default = 15s] 1181 1182 # Name of network interfaces to read addresses from. 1183 # CLI flag: -ingester.lifecycler.interface 1184 interface_names: 1185 1186 - [<string> ... | default = [<private network interfaces>]] 1187 1188 # Duration to sleep before exiting to ensure metrics are scraped. 1189 # CLI flag: -ingester.final-sleep 1190 [final_sleep: <duration> | default = 30s] 1191 1192 # Number of times to try and transfer chunks when leaving before 1193 # falling back to flushing to the store. Zero = no transfers are done. 1194 # CLI flag: -ingester.max-transfer-retries 1195 [max_transfer_retries: <int> | default = 0] 1196 1197 # How many flushes can happen concurrently from each stream. 1198 # CLI flag: -ingester.concurrent-flushes 1199 [concurrent_flushes: <int> | default = 32] 1200 1201 # How often should the ingester see if there are any blocks to flush 1202 # CLI flag: -ingester.flush-check-period 1203 [flush_check_period: <duration> | default = 30s] 1204 1205 # The timeout before a flush is cancelled 1206 # CLI flag: -ingester.flush-op-timeout 1207 [flush_op_timeout: <duration> | default = 10m] 1208 1209 # How long chunks should be retained in-memory after they've been flushed. 1210 # CLI flag: -ingester.chunks-retain-period 1211 [chunk_retain_period: <duration> | default = 0s] 1212 1213 # How long chunks should sit in-memory with no updates before 1214 # being flushed if they don't hit the max block size. This means 1215 # that half-empty chunks will still be flushed after a certain 1216 # period as long as they receive no further activity. 1217 # CLI flag: -ingester.chunks-idle-period 1218 [chunk_idle_period: <duration> | default = 30m] 1219 1220 # The targeted _uncompressed_ size in bytes of a chunk block 1221 # When this threshold is exceeded the head block will be cut and compressed inside the chunk. 1222 # CLI flag: -ingester.chunks-block-size 1223 [chunk_block_size: <int> | default = 262144] 1224 1225 # A target _compressed_ size in bytes for chunks. 1226 # This is a desired size not an exact size, chunks may be slightly bigger 1227 # or significantly smaller if they get flushed for other reasons (e.g. chunk_idle_period) 1228 # A value of 0 creates chunks with a fixed 10 blocks, 1229 # A non zero value will create chunks with a variable number of blocks to meet the target size. 1230 # CLI flag: -ingester.chunk-target-size 1231 [chunk_target_size: <int> | default = 1572864] 1232 1233 # The compression algorithm to use for chunks. (supported: gzip, lz4, snappy) 1234 # You should choose your algorithm depending on your need: 1235 # - `gzip` highest compression ratio but also slowest decompression speed. (144 kB per chunk) 1236 # - `lz4` fastest compression speed (188 kB per chunk) 1237 # - `snappy` fast and popular compression algorithm (272 kB per chunk) 1238 # CLI flag: -ingester.chunk-encoding 1239 [chunk_encoding: <string> | default = gzip] 1240 1241 # Parameters used to synchronize ingesters to cut chunks at the same moment. 1242 # Sync period is used to roll over incoming entry to a new chunk. If chunk's utilization 1243 # isn't high enough (eg. less than 50% when sync_min_utilization is set to 0.5), then 1244 # this chunk rollover doesn't happen. 1245 # CLI flag: -ingester.sync-period 1246 [sync_period: <duration> | default = 0] 1247 1248 # CLI flag: -ingester.sync-min-utilization 1249 [sync_min_utilization: <float> | Default = 0] 1250 1251 # The maximum number of errors a stream will report to the user 1252 # when a push fails. 0 to make unlimited. 1253 # CLI flag: -ingester.max-ignored-stream-errors 1254 [max_returned_stream_errors: <int> | default = 10] 1255 1256 # The maximum duration of a timeseries chunk in memory. If a timeseries runs for longer than this, 1257 # the current chunk will be flushed to the store and a new chunk created. 1258 # CLI flag: -ingester.max-chunk-age 1259 [max_chunk_age: <duration> | default = 2h] 1260 1261 # How far in the past an ingester is allowed to query the store for data. 1262 # This is only useful for running multiple Loki binaries with a shared ring 1263 # with a `filesystem` store, which is NOT shared between the binaries. 1264 # When using any "shared" object store like S3 or GCS, this value must always be left as 0. 1265 # It is an error to configure this to a non-zero value when using any object store other 1266 # than `filesystem`. 1267 # Use a value of -1 to allow the ingester to query the store infinitely far back in time. 1268 # CLI flag: -ingester.query-store-max-look-back-period 1269 [query_store_max_look_back_period: <duration> | default = 0] 1270 1271 # Forget about ingesters having heartbeat timestamps older than `ring.kvstore.heartbeat_timeout`. 1272 # This is equivalent to clicking on the `/ring` `forget` button in the UI: 1273 # the ingester is removed from the ring. 1274 # This is a useful setting when you are sure that an unhealthy node won't return. 1275 # An example is when not using stateful sets or the equivalent. 1276 # Use `memberlist.rejoin_interval` > 0 to handle network partition cases when using a memberlist. 1277 # CLI flag: -ingester.autoforget-unhealthy 1278 [autoforget_unhealthy: <boolean> | default = false] 1279 1280 # The ingester WAL (Write Ahead Log) records incoming logs and stores them on 1281 # the local file systems in order to guarantee persistence of acknowledged data 1282 # in the event of a process crash. 1283 wal: 1284 # Enables writing to WAL. 1285 # CLI flag: -ingester.wal-enabled 1286 [enabled: <boolean> | default = true] 1287 1288 # Directory where the WAL data should be stored and/or recovered from. 1289 # CLI flag: -ingester.wal-dir 1290 [dir: <filename> | default = "wal"] 1291 1292 # When WAL is enabled, should chunks be flushed to long-term storage on shutdown. 1293 # CLI flag: -ingester.flush-on-shutdown 1294 [flush_on_shutdown: <boolean> | default = false] 1295 1296 # Interval at which checkpoints should be created. 1297 # CLI flag: ingester.checkpoint-duration 1298 [checkpoint_duration: <duration> | default = 5m] 1299 1300 # Maximum memory size the WAL may use during replay. After hitting this, 1301 # it will flush data to storage before continuing. 1302 # A unit suffix (KB, MB, GB) may be applied. 1303 [replay_memory_ceiling: <string> | default = 4GB] 1304 1305 # Shard factor used in the ingesters for the in process reverse index. 1306 # This MUST be evenly divisible by ALL schema shard factors or Loki will not start. 1307 [index_shards: <int> | default = 32] 1308 ``` 1309 1310 ## consul_config 1311 1312 The `consul_config` configures the consul client. The supported CLI flags <prefix> used to reference this config block are: 1313 1314 ```yaml 1315 # The hostname and port of Consul. 1316 # CLI flag: -<prefix>.consul.hostname 1317 [host: <string> | default = "localhost:8500"] 1318 1319 # The ACL Token used to interact with Consul. 1320 # CLI flag: -<prefix>.consul.acl-token 1321 [acl_token: <string>] 1322 1323 # The HTTP timeout when communicating with Consul 1324 # CLI flag: -<prefix>.consul.client-timeout 1325 [http_client_timeout: <duration> | default = 20s] 1326 1327 # Whether or not consistent reads to Consul are enabled. 1328 # CLI flag: -<prefix>.consul.consistent-reads 1329 [consistent_reads: <boolean> | default = true] 1330 ``` 1331 1332 ## etcd_config 1333 1334 The `etcd_config` configures the etcd client. The supported CLI flags <prefix> used to reference this config block are: 1335 1336 ```yaml 1337 # The etcd endpoints to connect to. 1338 # CLI flag: -<prefix>.etcd.endpoints 1339 [endpoints: <list of string> | default = []] 1340 1341 # The dial timeout for the etcd connection. 1342 # CLI flag: -<prefix>.etcd.dial-timeout 1343 [dial_timeout: <duration> | default = 10s] 1344 1345 # The maximum number of retries to do for failed ops. 1346 # CLI flag: -<prefix>.etcd.max-retries 1347 [max_retries: <int> | default = 10] 1348 ``` 1349 1350 ## memberlist_config 1351 1352 The `memberlist_config` block configures the gossip ring to discover and connect 1353 between distributors, ingesters and queriers. The configuration is unique for all 1354 three components to ensure a single shared ring. 1355 1356 When a `memberlist_config` with least 1 `join_members` is defined, a `kvstore` of type `memberlist` is 1357 automatically configured for the `distributor`, `ingester`, and `ruler` rings unless otherwise specified in 1358 those components specific configuration sections. 1359 1360 ```yaml 1361 # Name of the node in memberlist cluster. Defaults to hostname. 1362 # CLI flag: -memberlist.nodename 1363 [node_name: <string> | default = ""] 1364 1365 # Add random suffix to the node name. 1366 # CLI flag: -memberlist.randomize-node-name 1367 [randomize_node_name: <boolean> | default = true] 1368 1369 # The timeout for establishing a connection with a remote node, and for 1370 # read/write operations. Uses memberlist LAN defaults if 0. 1371 # CLI flag: -memberlist.stream-timeout 1372 [stream_timeout: <duration> | default = 0s] 1373 1374 # Multiplication factor used when sending out messages (factor * log(N+1)). 1375 # CLI flag: -memberlist.retransmit-factor 1376 [retransmit_factor: <int> | default = 0] 1377 1378 # How often to use pull/push sync. Uses memberlist LAN defaults if 0. 1379 # CLI flag: -memberlist.pullpush-interval 1380 [pull_push_interval: <duration> | default = 0s] 1381 1382 # How often to gossip. Uses memberlist LAN defaults if 0. 1383 # CLI flag: -memberlist.gossip-interval 1384 [gossip_interval: <duration> | default = 0s] 1385 1386 # How many nodes to gossip to. Uses memberlist LAN defaults if 0. 1387 # CLI flag: -memberlist.gossip-nodes 1388 [gossip_nodes: <int> | default = 0] 1389 1390 # How long to keep gossiping to dead nodes, to give them chance to refute their 1391 # death. Uses memberlist LAN defaults if 0. 1392 # CLI flag: -memberlist.gossip-to-dead-nodes-time 1393 [gossip_to_dead_nodes_time: <duration> | default = 0s] 1394 1395 # How soon can dead node's name be reclaimed with new address. Defaults to 0, 1396 # which is disabled. 1397 # CLI flag: -memberlist.dead-node-reclaim-time 1398 [dead_node_reclaim_time: <duration> | default = 0s] 1399 1400 # Other cluster members to join. Can be specified multiple times. It can be an 1401 # IP, hostname or an entry specified in the DNS Service Discovery format (see 1402 # https://grafana.com/docs/mimir/latest/operators-guide/configuring/about-dns-service-discovery/ 1403 # for more details). 1404 # CLI flag: -memberlist.join 1405 [join_members: <list of string> | default = ] 1406 1407 # Min backoff duration to join other cluster members. 1408 # CLI flag: -memberlist.min-join-backoff 1409 [min_join_backoff: <duration> | default = 1s] 1410 1411 # Max backoff duration to join other cluster members. 1412 # CLI flag: -memberlist.max-join-backoff 1413 [max_join_backoff: <duration> | default = 1m] 1414 1415 # Max number of retries to join other cluster members. 1416 # CLI flag: -memberlist.max-join-retries 1417 [max_join_retries: <int> | default = 10] 1418 1419 # If this node fails to join memberlist cluster, abort. 1420 # CLI flag: -memberlist.abort-if-join-fails 1421 [abort_if_cluster_join_fails: <boolean> | default = true] 1422 1423 # If not 0, how often to rejoin the cluster. Occasional rejoin can help to fix 1424 # the cluster split issue, and is harmless otherwise. For example when using 1425 # only few components as a seed nodes (via -memberlist.join), then it's 1426 # recommended to use rejoin. If -memberlist.join points to dynamic service that 1427 # resolves to all gossiping nodes (eg. Kubernetes headless service), then rejoin 1428 # is not needed. 1429 # CLI flag: -memberlist.rejoin-interval 1430 [rejoin_interval: <duration> | default = 0s] 1431 1432 # How long to keep LEFT ingesters in the ring. 1433 # CLI flag: -memberlist.left-ingesters-timeout 1434 [left_ingesters_timeout: <duration> | default = 5m] 1435 1436 # Timeout for leaving memberlist cluster. 1437 # CLI flag: -memberlist.leave-timeout 1438 [leave_timeout: <duration> | default = 5s] 1439 1440 # IP address to listen on for gossip messages. Multiple addresses may be 1441 # specified. Defaults to 0.0.0.0 1442 # CLI flag: -memberlist.bind-addr 1443 [bind_addr: <list of string> | default = ] 1444 1445 # Port to listen on for gossip messages. 1446 # CLI flag: -memberlist.bind-port 1447 [bind_port: <int> | default = 7946] 1448 1449 # Timeout used when connecting to other nodes to send packet. 1450 # CLI flag: -memberlist.packet-dial-timeout 1451 [packet_dial_timeout: <duration> | default = 5s] 1452 1453 # Timeout for writing 'packet' data. 1454 # CLI flag: -memberlist.packet-write-timeout 1455 [packet_write_timeout: <duration> | default = 5s] 1456 ``` 1457 1458 ## storage_config 1459 1460 The `storage_config` block configures one of many possible stores for both the 1461 index and chunks. Which configuration to be picked should be defined in schema_config 1462 block. 1463 1464 ```yaml 1465 # Configures storing chunks in AWS. Required options only required when aws is 1466 # present. 1467 aws: 1468 # S3 or S3-compatible endpoint URL with escaped Key and Secret encoded. 1469 # If only region is specified as a host, the proper endpoint will be deduced. 1470 # Use inmemory:///<bucket-name> to use a mock in-memory implementation. 1471 # CLI flag: -s3.url 1472 [s3: <string>] 1473 1474 # Set to true to force the request to use path-style addressing 1475 # CLI flag: -s3.force-path-style 1476 [s3forcepathstyle: <boolean> | default = false] 1477 1478 # Comma separated list of bucket names to evenly distribute chunks over. 1479 # Overrides any buckets specified in s3.url flag 1480 # CLI flag: -s3.buckets 1481 [bucketnames: <string> | default = ""] 1482 1483 # S3 Endpoint to connect to. 1484 # CLI flag: -s3.endpoint 1485 [endpoint: <string> | default = ""] 1486 1487 # AWS region to use. 1488 # CLI flag: -s3.region 1489 [region: <string> | default = ""] 1490 1491 # AWS Access Key ID. 1492 # CLI flag: -s3.access-key-id 1493 [access_key_id: <string> | default = ""] 1494 1495 # AWS Secret Access Key. 1496 # CLI flag: -s3.secret-access-key 1497 [secret_access_key: <string> | default = ""] 1498 1499 # Disable https on S3 connection. 1500 # CLI flag: -s3.insecure 1501 [insecure: <boolean> | default = false] 1502 1503 # Enable AES256 AWS Server Side Encryption. 1504 # CLI flag: -s3.sse-encryption 1505 [sse_encryption: <boolean> | default = false] 1506 1507 http_config: 1508 # The maximum amount of time an idle connection will be held open. 1509 # CLI flag: -s3.http.idle-conn-timeout 1510 [idle_conn_timeout: <duration> | default = 1m30s] 1511 1512 # If non-zero, specifies the amount of time to wait for a server's response 1513 # headers after fully writing the request. 1514 # CLI flag: -s3.http.response-header-timeout 1515 [response_header_timeout: <duration> | default = 0s] 1516 1517 # Set to true to skip verifying the certificate chain and hostname. 1518 # CLI flag: -s3.http.insecure-skip-verify 1519 [insecure_skip_verify: <boolean> | default = false] 1520 1521 # Path to the trusted CA file that signed the SSL certificate of the S3 1522 # endpoint. 1523 # CLI flag: -s3.http.ca-file 1524 [ca_file: <string> | default = ""] 1525 1526 # Configures back off when s3 get Object. 1527 backoff_config: 1528 # Minimum duration to back off. 1529 # CLI flag: -s3.backoff-min-period 1530 [min_period: <duration> | default = 100ms] 1531 1532 # The duration to back off. 1533 # CLI flag: -s3.backoff-max-period 1534 [max_period: <duration> | default = 3s] 1535 1536 # Number of times to back off and retry before failing. 1537 # CLI flag: -s3.backoff-retries 1538 [max_retries: <int> | default = 5] 1539 1540 # Configure the DynamoDB connection 1541 dynamodb: 1542 # URL for DynamoDB with escaped Key and Secret encoded. If only region is specified as a 1543 # host, the proper endpoint will be deduced. Use inmemory:///<bucket-name> to 1544 # use a mock in-memory implementation. 1545 # CLI flag: -dynamodb.url 1546 dynamodb_url: <string> 1547 1548 # DynamoDB table management requests per-second limit. 1549 # CLI flag: -dynamodb.api-limit 1550 [api_limit: <float> | default = 2.0] 1551 1552 # DynamoDB rate cap to back off when throttled. 1553 # CLI flag: -dynamodb.throttle-limit 1554 [throttle_limit: <float> | default = 10.0] 1555 1556 # Metrics-based autoscaling configuration. 1557 metrics: 1558 # Use metrics-based autoscaling via this Prometheus query URL. 1559 # CLI flag: -metrics.url 1560 [url: <string>] 1561 1562 # Queue length above which we will scale up capacity. 1563 # CLI flag: -metrics.target-queue-length 1564 [target_queue_length: <int> | default = 100000] 1565 1566 # Scale up capacity by this multiple 1567 # CLI flag: -metrics.scale-up-factor 1568 [scale_up_factor: <float64> | default = 1.3] 1569 1570 # Ignore throttling below this level (rate per second) 1571 # CLI flag: -metrics.ignore-throttle-below 1572 [ignore_throttle_below: <float64> | default = 1] 1573 1574 # Number of chunks to group together to parallelise fetches (0 to disable) 1575 # CLI flag: -dynamodb.chunk-gang-size 1576 [chunk_gang_size: <int> | default = 10] 1577 1578 # Max number of chunk get operations to start in parallel. 1579 # CLI flag: -dynamodb.chunk.get-max-parallelism 1580 [chunk_get_max_parallelism: <int> | default = 32] 1581 1582 # Configures storing indexes in Bigtable. Required fields only required 1583 # when bigtable is defined in config. 1584 bigtable: 1585 # BigTable project ID 1586 # CLI flag: -bigtable.project 1587 project: <string> 1588 1589 # BigTable instance ID 1590 # CLI flag: -bigtable.instance 1591 instance: <string> 1592 1593 # Configures the gRPC client used to connect to Bigtable. 1594 # The CLI flags prefix for this block config is: bigtable 1595 [grpc_client_config: <grpc_client_config>] 1596 1597 # Configures storing chunks in GCS. Required fields only required 1598 # when gcs is defined in config. 1599 gcs: 1600 # Name of GCS bucket to put chunks in. 1601 # CLI flag: -gcs.bucketname 1602 bucket_name: <string> 1603 1604 # The size of the buffer that the GCS client uses for each PUT request. 0 1605 # to disable buffering. 1606 # CLI flag: -gcs.chunk-buffer-size 1607 [chunk_buffer_size: <int> | default = 0] 1608 1609 # The duration after which the requests to GCS should be timed out. 1610 # CLI flag: -gcs.request-timeout 1611 [request_timeout: <duration> | default = 0s] 1612 1613 # Configures storing chunks and/or the index in Cassandra 1614 cassandra: 1615 # Comma-separated hostnames or IPs of Cassandra instances 1616 # CLI flag: -cassandra.addresses 1617 addresses: <string> 1618 1619 # Port that cassandra is running on 1620 # CLI flag: -cassandra.port 1621 [port: <int> | default = 9042] 1622 1623 # Keyspace to use in Cassandra 1624 # CLI flag: -cassandra.keyspace 1625 keyspace: <string> 1626 1627 # Consistency level for Cassandra 1628 # CLI flag: -cassandra.consistency 1629 [consistency: <string> | default = "QUORUM"] 1630 1631 # Replication factor to use in Cassandra. 1632 # CLI flag: -cassandra.replication-factor 1633 [replication_factor: <int> | default = 1] 1634 1635 # Instruct the Cassandra driver to not attempt to get host 1636 # info from the system.peers table. 1637 # CLI flag: -cassandra.disable-initial-host-lookup 1638 [disable_initial_host_lookup: <boolean> | default = false] 1639 1640 # Use SSL when connecting to Cassandra instances. 1641 # CLI flag: -cassandra.ssl 1642 [SSL: <boolean> | default = false] 1643 1644 # Require SSL certificate validation when SSL is enabled. 1645 # CLI flag: -cassandra.host-verification 1646 [host_verification: <boolean> | default = true] 1647 1648 # Path to CA certificate file to verify the peer when SSL is enabled. 1649 # CLI flag: -cassandra.ca-path 1650 [CA_path: <string>] 1651 1652 # Path to client certificate file when SSL is enabled. 1653 # CLI flag: -cassandra.tls-cert-path 1654 [tls_cert_path: <string>] 1655 1656 # Path to key certificate file when SSL is enabled. 1657 # CLI flag: -cassandra.tls-key-path 1658 [tls_key_path: <string>] 1659 1660 # Enable password authentication when connecting to Cassandra. 1661 # CLI flag: -cassandra.auth 1662 [auth: <boolean> | default = false] 1663 1664 # Username for password authentication when auth is true. 1665 # CLI flag: -cassandra.username 1666 [username: <string>] 1667 1668 # Password for password authentication when auth is true. 1669 # CLI flag: -cassandra.password 1670 [password: <string>] 1671 1672 # Timeout when connecting to Cassandra. 1673 # CLI flag: -cassandra.timeout 1674 [timeout: <duration> | default = 600ms] 1675 1676 # Initial connection timeout during initial dial to server. 1677 # CLI flag: -cassandra.connect-timeout 1678 [connect_timeout: <duration> | default = 600ms] 1679 1680 swift: 1681 # Openstack authentication URL. 1682 # CLI flag: -ruler.storage.swift.auth-url 1683 [auth_url: <string> | default = ""] 1684 1685 # Openstack username for the api. 1686 # CLI flag: -ruler.storage.swift.username 1687 [username: <string> | default = ""] 1688 1689 # Openstack user's domain name. 1690 # CLI flag: -ruler.storage.swift.user-domain-name 1691 [user_domain_name: <string> | default = ""] 1692 1693 # Openstack user's domain id. 1694 # CLI flag: -ruler.storage.swift.user-domain-id 1695 [user_domain_id: <string> | default = ""] 1696 1697 # Openstack userid for the api. 1698 # CLI flag: -ruler.storage.swift.user-id 1699 [user_id: <string> | default = ""] 1700 1701 # Openstack api key. 1702 # CLI flag: -ruler.storage.swift.password 1703 [password: <string> | default = ""] 1704 1705 # Openstack user's domain id. 1706 # CLI flag: -ruler.storage.swift.domain-id 1707 [domain_id: <string> | default = ""] 1708 1709 # Openstack user's domain name. 1710 # CLI flag: -ruler.storage.swift.domain-name 1711 [domain_name: <string> | default = ""] 1712 1713 # Openstack project id (v2,v3 auth only). 1714 # CLI flag: -ruler.storage.swift.project-id 1715 [project_id: <string> | default = ""] 1716 1717 # Openstack project name (v2,v3 auth only). 1718 # CLI flag: -ruler.storage.swift.project-name 1719 [project_name: <string> | default = ""] 1720 1721 # Id of the project's domain (v3 auth only), only needed if it differs the 1722 # from user domain. 1723 # CLI flag: -ruler.storage.swift.project-domain-id 1724 [project_domain_id: <string> | default = ""] 1725 1726 # Name of the project's domain (v3 auth only), only needed if it differs 1727 # from the user domain. 1728 # CLI flag: -ruler.storage.swift.project-domain-name 1729 [project_domain_name: <string> | default = ""] 1730 1731 # Openstack Region to use eg LON, ORD - default is use first region (v2,v3 1732 # auth only) 1733 # CLI flag: -ruler.storage.swift.region-name 1734 [region_name: <string> | default = ""] 1735 1736 # Name of the Swift container to put chunks in. 1737 # CLI flag: -ruler.storage.swift.container-name 1738 [container_name: <string> | default = "cortex"] 1739 1740 # Configures storing index in BoltDB. Required fields only 1741 # required when boltdb is present in the configuration. 1742 boltdb: 1743 # Location of BoltDB index files. 1744 # CLI flag: -boltdb.dir 1745 directory: <string> 1746 1747 # Configures storing the chunks on the local file system. Required 1748 # fields only required when filesystem is present in the configuration. 1749 filesystem: 1750 # Directory to store chunks in. 1751 # CLI flag: -local.chunk-directory 1752 directory: <string> 1753 1754 # Configures storing index in an Object Store(GCS/S3/Azure/Swift/Filesystem) in the form of 1755 # boltdb files. 1756 # Required fields only required when boltdb-shipper is defined in config. 1757 boltdb_shipper: 1758 # Directory where ingesters would write boltdb files which would then be 1759 # uploaded by shipper to configured storage 1760 # CLI flag: -boltdb.shipper.active-index-directory 1761 [active_index_directory: <string> | default = ""] 1762 1763 # Shared store for keeping boltdb files. Supported types: gcs, s3, azure, 1764 # filesystem 1765 # CLI flag: -boltdb.shipper.shared-store 1766 [shared_store: <string> | default = ""] 1767 1768 # Cache location for restoring boltDB files for queries 1769 # CLI flag: -boltdb.shipper.cache-location 1770 [cache_location: <string> | default = ""] 1771 1772 # TTL for boltDB files restored in cache for queries 1773 # CLI flag: -boltdb.shipper.cache-ttl 1774 [cache_ttl: <duration> | default = 24h] 1775 1776 # Resync downloaded files with the storage 1777 # CLI flag: -boltdb.shipper.resync-interval 1778 [resync_interval: <duration> | default = 5m] 1779 1780 # Number of days of index to be kept downloaded for queries. Works only with 1781 # tables created with 24h period. 1782 # CLI flag: -boltdb.shipper.query-ready-num-days 1783 [query_ready_num_days: <int> | default = 0] 1784 1785 index_gateway_client: 1786 # "Hostname or IP of the Index Gateway gRPC server. 1787 # CLI flag: -boltdb.shipper.index-gateway-client.server-address 1788 [server_address: <string> | default = ""] 1789 1790 # Configures the gRPC client used to connect to the Index Gateway gRPC server. 1791 # The CLI flags prefix for this block config is: boltdb.shipper.index-gateway-client 1792 [grpc_client_config: <grpc_client_config>] 1793 1794 # Configures if gateway requests should be logged or not. 1795 # CLI flag: -boltdb.shipper.index-gateway-client.log-gateway-requests 1796 [log_gateway_requests: <bool> | default = false] 1797 1798 # Cache validity for active index entries. Should be no higher than 1799 # the chunk_idle_period in the ingester settings. 1800 # CLI flag: -store.index-cache-validity 1801 [index_cache_validity: <duration> | default = 5m] 1802 1803 # Disable broad index queries, which results in reduced cache usage and faster query 1804 # performance at the expense of a somewhat higher QPS on the index store. 1805 # CLI flag: -store.disable-broad-index-queries 1806 [disable_broad_index_queries: <bool> | default = false] 1807 1808 # The maximum number of chunks to fetch per batch. 1809 # CLI flag: -store.max-chunk-batch-size 1810 [max_chunk_batch_size: <int> | default = 50] 1811 1812 # Config for how the cache for index queries should be built. 1813 # The CLI flags prefix for this block config is: store.index-cache-read 1814 index_queries_cache_config: <cache_config> 1815 ``` 1816 1817 ## chunk_store_config 1818 1819 The `chunk_store_config` block configures how chunks will be cached and how long 1820 to wait before saving them to the backing store. 1821 1822 ```yaml 1823 # The cache configuration for storing chunks 1824 # The CLI flags prefix for this block config is: store.chunks-cache 1825 [chunk_cache_config: <cache_config>] 1826 1827 # The cache configuration for deduplicating writes 1828 # The CLI flags prefix for this block config is: store.index-cache-write 1829 [write_dedupe_cache_config: <cache_config>] 1830 1831 # Cache index entries older than this period. Default is disabled. 1832 # CLI flag: -store.cache-lookups-older-than 1833 [cache_lookups_older_than: <duration>] 1834 1835 # Limit how long back data can be queried. Default is disabled. 1836 # This should always be set to a value less than or equal to 1837 # what is set in `table_manager.retention_period` . 1838 # CLI flag: -store.max-look-back-period 1839 [max_look_back_period: <duration>] 1840 ``` 1841 1842 ## cache_config 1843 1844 The `cache_config` block configures how Loki will cache requests, chunks, and 1845 the index to a backing cache store. 1846 1847 <span style="background-color:#f3f973;">The memcached configuration variable addresses is experimental.</span> 1848 1849 ```yaml 1850 # Enable in-memory cache. 1851 # CLI flag: -<prefix>.cache.enable-fifocache 1852 [enable_fifocache: <boolean>] 1853 1854 # The default validity of entries for caches unless overridden. 1855 # NOTE In Loki versions older than 1.4.0 this was "defaul_validity". 1856 # CLI flag: -<prefix>.default-validity 1857 [default_validity: <duration>] 1858 1859 # Configures the background cache when memcached is used. 1860 background: 1861 # How many goroutines to use to write back to memcached. 1862 # CLI flag: -<prefix>.background.write-back-concurrency 1863 [writeback_goroutines: <int> | default = 10] 1864 1865 # How many chunks to buffer for background write back to memcached. 1866 # CLI flagL -<prefix>.background.write-back-buffer 1867 [writeback_buffer: <int> = 10000] 1868 1869 # Configures memcached settings. 1870 memcached: 1871 # Configures how long keys stay in memcached. 1872 # CLI flag: -<prefix>.memcached.expiration 1873 expiration: <duration> 1874 1875 # Configures how many keys to fetch in each batch request. 1876 # CLI flag: -<prefix>.memcached.batchsize 1877 batch_size: <int> | default = 1024 1878 1879 # Maximum active requests to memcached. 1880 # CLI flag: -<prefix>.memcached.parallelism 1881 [parallelism: <int> | default = 100] 1882 1883 # Configures how to connect to one or more memcached servers. 1884 memcached_client: 1885 # The hostname to use for memcached services when caching chunks. If 1886 # empty, no memcached will be used. A SRV lookup will be used. 1887 # CLI flag: -<prefix>.memcached.hostname 1888 [host: <string>] 1889 1890 # SRV service used to discover memcached servers. 1891 # CLI flag: -<prefix>.memcached.service 1892 [service: <string> | default = "memcached"] 1893 1894 # (Experimental) Comma-separated addresses list in DNS Service Discovery format: 1895 # https://grafana.com/docs/mimir/latest/operators-guide/configuring/about-dns-service-discovery/ 1896 # CLI flag: -<prefix>.memcached.addresses 1897 [addresses: <string> | default = ""] 1898 1899 # Maximum time to wait before giving up on memcached requests. 1900 # CLI flag: -<prefix>.memcached.timeout 1901 [timeout: <duration> | default = 100ms] 1902 1903 # The maximum number of idle connections in the memcached client pool. 1904 # CLI flag: -<prefix>.memcached.max-idle-conns 1905 [max_idle_conns: <int> | default = 16] 1906 1907 # The period with which to poll the DNS for memcached servers. 1908 # CLI flag: -<prefix>.memcached.update-interval 1909 [update_interval: <duration> | default = 1m] 1910 1911 # Whether or not to use a consistent hash to discover multiple memcached servers. 1912 # CLI flag: -<prefix>.memcached.consistent-hash 1913 [consistent_hash: <boolean> | default = true] 1914 1915 # Trip the circuit breaker after this number of consecutive dial failures. 1916 # A value of 0 disables the circuit breaker. 1917 # CLI flag: -<prefix>.memcached.circuit-breaker-consecutive-failures 1918 [circuit_breaker_consecutive_failures: <int> | default = 10] 1919 1920 # Duration the circuit breaker remains open after tripping. 1921 # If set to 0, the duration is 60 seconds. 1922 # CLI flag: -<prefix>.memcached.circuit-breaker-timeout 1923 [circuit_breaker_timeout: <duration> | default = 10s] 1924 1925 # Reset the circuit breaker counts after this duration. 1926 # A value of 0 never resets the circuit breaker. 1927 # CLI flag: -<prefix>.memcached.circuit-breaker-interval 1928 [circuit_breaker_interval: <duration> | default = 10s] 1929 1930 # The maximum size of an item stored in memcached. 1931 # Bigger items are not stored. If set to 0, no maximum size is enforced. 1932 # CLI flag: -<prefix>.memcached.max-item-size 1933 [max_item_size: <int> | default = 0] 1934 1935 redis: 1936 # Redis Server or Cluster configuration endpoint to use for caching. A comma-separated list of endpoints 1937 # for Redis Cluster or Redis Sentinel. If empty, no redis will be used. 1938 # CLI flag: -<prefix>.redis.endpoint 1939 [endpoint: <string>] 1940 1941 # Redis Sentinel master name. An empty string for Redis Server or Redis Cluster. 1942 # CLI flag: -<prefix>.redis.master-name 1943 [master_name: <string>] 1944 1945 # Maximum time to wait before giving up on redis requests. 1946 # CLI flag: -<prefix>.redis.timeout 1947 [timeout: <duration> | default = 500ms] 1948 1949 # How long keys stay in the redis. 1950 # CLI flag: -<prefix>.redis.expiration 1951 [expiration: <duration> | default = 0s] 1952 1953 # Database index. 1954 # CLI flag: -<prefix>.redis.db 1955 [db: <int> | default = 0] 1956 1957 # Maximum number of connections in the pool. 1958 # CLI flag: -<prefix>.redis.pool-size 1959 [pool_size: <int> | default = 0] 1960 1961 # Password to use when connecting to redis. 1962 # CLI flag: -<prefix>.redis.password 1963 [password: <string>] 1964 1965 # Enables connecting to redis with TLS. 1966 # CLI flag: -<prefix>.redis.tls-enabled 1967 [tls_enabled: <boolean> | default = false] 1968 1969 # Skip validating server certificate. 1970 # CLI flag: -<prefix>.redis.tls-insecure-skip-verify 1971 [tls_insecure_skip_verify: <boolean> | default = false] 1972 1973 # Close connections after remaining idle for this duration. 1974 # If the value is zero, then idle connections are not closed. 1975 # CLI flag: -<prefix>.redis.idle-timeout 1976 [idle_timeout: <duration> | default = 0s] 1977 1978 # Close connections older than this duration. If the value is zero, then 1979 # the pool does not close connections based on age. 1980 # CLI flag: -<prefix>.redis.max-connection-age 1981 [max_connection_age: <duration> | default = 0s] 1982 1983 fifocache: 1984 # Maximum memory size of the cache in bytes. A unit suffix (KB, MB, GB) may be 1985 # applied. 1986 # CLI flag: -<prefix>.fifocache.max-size-bytes 1987 [max_size_bytes: <string> | default = "1GB"] 1988 1989 # Maximum number of entries in the cache. 1990 # CLI flag: -<prefix>.fifocache.max-size-items 1991 [max_size_items: <int> | default = 0] 1992 1993 # Deprecated: The expiry duration for the cache. Use `-<prefix>.fifocache.ttl`. 1994 # The default value of 0 disables expiration. 1995 # CLI flag: -<prefix>.fifocache.duration 1996 [validity: <duration>] 1997 1998 # The time for items to live in the cache before those items are purged. 1999 # The value of 0 disables auto-expiration. 2000 # CLI flag: -<prefix>.fifocache.ttl 2001 [ttl: <duration> | default = 1h] 2002 ``` 2003 2004 ## schema_config 2005 2006 The `schema_config` block configures schemas from given dates. 2007 2008 ```yaml 2009 # The configuration for chunk index schemas. 2010 configs: 2011 - [<period_config>] 2012 ``` 2013 2014 ### period_config 2015 2016 The `period_config` block configures what index schemas should be used 2017 for from specific time periods. 2018 2019 ```yaml 2020 # The date of the first day that index buckets should be created. Use 2021 # a date in the past if this is your only period_config, otherwise 2022 # use a date when you want the schema to switch over. 2023 # In YYYY-MM-DD format, for example: 2018-04-15. 2024 [from: <daytime>] 2025 2026 # store and object_store below affect which <storage_config> key is 2027 # used. 2028 2029 # Which store to use for the index. Either aws, aws-dynamo, gcp, bigtable, bigtable-hashed, 2030 # cassandra, boltdb or boltdb-shipper. 2031 store: <string> 2032 2033 # Which store to use for the chunks. Either aws, azure, gcp, 2034 # bigtable, gcs, cassandra, swift or filesystem. If omitted, defaults to the same 2035 # value as store. 2036 [object_store: <string>] 2037 2038 # The schema version to use, current recommended schema is v11. 2039 schema: <string> 2040 2041 # Configures how the index is updated and stored. 2042 index: 2043 # Table prefix for all period tables. 2044 prefix: <string> 2045 # Table period. 2046 [period: <duration> | default = 168h] 2047 # A map to be added to all managed tables. 2048 tags: 2049 [<string>: <string> ...] 2050 2051 # Configured how the chunks are updated and stored. 2052 chunks: 2053 # Table prefix for all period tables. 2054 prefix: <string> 2055 # Table period. 2056 [period: <duration> | default = 168h] 2057 # A map to be added to all managed tables. 2058 tags: 2059 [<string>: <string> ...] 2060 2061 # How many shards will be created. Only used if schema is v10 or greater. 2062 [row_shards: <int> | default = 16] 2063 ``` 2064 2065 ## compactor 2066 2067 The `compactor` block configures the compactor component. This component periodically 2068 compacts index shards to more performant forms. 2069 2070 ```yaml 2071 # Directory where files can be downloaded for compaction. 2072 # CLI flag: -boltdb.shipper.compactor.working-directory 2073 [working_directory: <string>] 2074 2075 # The shared store used for storing boltdb files. 2076 # Supported types: gcs, s3, azure, swift, filesystem, bos. 2077 # CLI flag: -boltdb.shipper.compactor.shared-store 2078 [shared_store: <string>] 2079 2080 # Prefix to add to object keys in shared store. 2081 # Path separator(if any) should always be a '/'. 2082 # Prefix should never start with a separator but should always end with it. 2083 # CLI flag: -boltdb.shipper.compactor.shared-store.key-prefix 2084 [shared_store_key_prefix: <string> | default = "index/"] 2085 2086 # Interval at which to re-run the compaction operation (or retention if enabled). 2087 # CLI flag: -boltdb.shipper.compactor.compaction-interval 2088 [compaction_interval: <duration> | default = 10m] 2089 2090 # (Experimental) Activate custom (per-stream,per-tenant) retention. 2091 # CLI flag: -boltdb.shipper.compactor.retention-enabled 2092 [retention_enabled: <boolean> | default = false] 2093 2094 # Delay after which chunks will be fully deleted during retention. 2095 # CLI flag: -boltdb.shipper.compactor.retention-delete-delay 2096 [retention_delete_delay: <duration> | default = 2h] 2097 2098 # The total amount of worker to use to delete chunks. 2099 # CLI flag: -boltdb.shipper.compactor.retention-delete-worker-count 2100 [retention_delete_worker_count: <int> | default = 150] 2101 2102 # Allow cancellation of delete request until duration after they are created. 2103 # Data would be deleted only after delete requests have been older than this duration. 2104 # Ideally this should be set to at least 24h. 2105 # CLI flag: -boltdb.shipper.compactor.delete-request-cancel-period 2106 [delete_request_cancel_period: <duration> | default = 24h] 2107 2108 # The max number of delete requests to run per compaction cycle. 2109 # CLI flag: -boltdb.shipper.compactor.delete-batch-size 2110 [delete_batch_size: <duration> | default = 70] 2111 2112 # The maximum amount of time to spend running retention and deletion 2113 # on any given table in the index. 0 is no timeout 2114 # 2115 # NOTE: This timeout prioritizes runtime over completeness of retention/deletion. 2116 # It may take several compaction runs to fully perform retention and process 2117 # all outstanding delete requests 2118 # CLI flag: -boltdb.shipper.compactor.retention-table-timeout 2119 [retention_table_timeout: <duration> | default = 0] 2120 2121 # Maximum number of tables to compact in parallel. 2122 # While increasing this value, please make sure compactor has enough disk space 2123 # allocated to be able to store and compact as many tables. 2124 # CLI flag: -boltdb.shipper.compactor.max-compaction-parallelism 2125 [max_compaction_parallelism: <int> | default = 1] 2126 2127 # Deletion mode. 2128 # Can be one of "disabled", "filter-only", or "filter-and-delete". 2129 # When set to "filter-only" or "filter-and-delete", and if 2130 # retention_enabled is true, then the log entry deletion API endpoints are available. 2131 # CLI flag: -boltdb.shipper.compactor.deletion-mode 2132 [deletion_mode: <string> | default = "disabled"] 2133 2134 # The hash ring configuration used by compactors to elect a single instance for running compactions 2135 # The CLI flags prefix for this block config is: boltdb.shipper.compactor.ring 2136 [compactor_ring: <ring>] 2137 ``` 2138 2139 ## limits_config 2140 2141 The `limits_config` block configures global and per-tenant limits in Loki. 2142 2143 ```yaml 2144 # Whether the ingestion rate limit should be applied individually to each 2145 # distributor instance (local), or evenly shared across the cluster (global). 2146 # The ingestion rate strategy cannot be overridden on a per-tenant basis. 2147 # 2148 # - local: enforces the limit on a per distributor basis. The actual effective 2149 # rate limit will be N times higher, where N is the number of distributor 2150 # replicas. 2151 # - global: enforces the limit globally, configuring a per-distributor local 2152 # rate limiter as "ingestion_rate / N", where N is the number of distributor 2153 # replicas (it's automatically adjusted if the number of replicas change). 2154 # The global strategy requires the distributors to form their own ring, which 2155 # is used to keep track of the current number of healthy distributor replicas. 2156 # CLI flag: -distributor.ingestion-rate-limit-strategy 2157 [ingestion_rate_strategy: <string> | default = "global"] 2158 2159 # Per-user ingestion rate limit in sample size per second. Units in MB. 2160 # CLI flag: -distributor.ingestion-rate-limit-mb 2161 [ingestion_rate_mb: <float> | default = 4] 2162 2163 # Per-user allowed ingestion burst size (in sample size). Units in MB. 2164 # The burst size refers to the per-distributor local rate limiter even in the 2165 # case of the "global" strategy, and should be set at least to the maximum logs 2166 # size expected in a single push request. 2167 # CLI flag: -distributor.ingestion-burst-size-mb 2168 [ingestion_burst_size_mb: <int> | default = 6] 2169 2170 # Maximum length of a label name. 2171 # CLI flag: -validation.max-length-label-name 2172 [max_label_name_length: <int> | default = 1024] 2173 2174 # Maximum length of a label value. 2175 # CLI flag: -validation.max-length-label-value 2176 [max_label_value_length: <int> | default = 2048] 2177 2178 # Maximum number of label names per series. 2179 # CLI flag: -validation.max-label-names-per-series 2180 [max_label_names_per_series: <int> | default = 30] 2181 2182 # Whether or not old samples will be rejected. 2183 # CLI flag: -validation.reject-old-samples 2184 [reject_old_samples: <boolean> | default = true] 2185 2186 # Maximum accepted sample age before rejecting. 2187 # CLI flag: -validation.reject-old-samples.max-age 2188 [reject_old_samples_max_age: <duration> | default = 168h] 2189 2190 # Duration for a table to be created/deleted before/after it's 2191 # needed. Samples won't be accepted before this time. 2192 # CLI flag: -validation.create-grace-period 2193 [creation_grace_period: <duration> | default = 10m] 2194 2195 # Enforce every sample has a metric name. 2196 # CLI flag: -validation.enforce-metric-name 2197 [enforce_metric_name: <boolean> | default = true] 2198 2199 # Maximum line size on ingestion path. Example: 256kb. 2200 # There is no limit when unset or set to 0. 2201 # CLI flag: -distributor.max-line-size 2202 [max_line_size: <string> | default = 0 ] 2203 2204 # Truncate log lines when they exceed max_line_size. 2205 # CLI flag: -distributor.max-line-size-truncate 2206 [max_line_size_truncate: <boolean> | default = false ] 2207 2208 # Alter the log line timestamp during ingestion when the timestamp is the same as the 2209 # previous entry for the same stream. When enabled, if a log line in a push request has 2210 # the same timestamp as the previous line for the same stream, one nanosecond is added 2211 # to the log line. This will preserve the received order of log lines with the exact 2212 # same timestamp when they are queried, by slightly altering their stored timestamp. 2213 # NOTE: This is imperfect, because Loki accepts out of order writes, and another push 2214 # request for the same stream could contain duplicate timestamps to existing 2215 # entries and they will not be incremented. 2216 # CLI flag: -validation.increment-duplicate-timestamps 2217 [increment_duplicate_timestamp: <boolean> | default = false ] 2218 2219 # Maximum number of log entries that will be returned for a query. 2220 # CLI flag: -validation.max-entries-limit 2221 [max_entries_limit_per_query: <int> | default = 5000 ] 2222 2223 # Maximum number of active streams per user, per ingester. 0 to make it unlimited. 2224 # CLI flag: -ingester.max-streams-per-user 2225 [max_streams_per_user: <int> | default 0] 2226 2227 # Maximum number of active streams per user, across the cluster. 0 to disable. 2228 # When the global limit is enabled, each ingester is configured with a dynamic 2229 # local limit based on the replication factor and the current number of healthy 2230 # ingesters, and is kept updated whenever the number of ingesters change. 2231 # CLI flag: -ingester.max-global-streams-per-user 2232 [max_global_streams_per_user: <int> | default = 5000] 2233 2234 # When true, out-of-order writes are accepted. 2235 # CLI flag: -ingester.unordered-writes 2236 [unordered_writes: <boolean> | default = true] 2237 2238 # Maximum number of chunks that can be fetched by a single query. 2239 # CLI flag: -store.query-chunk-limit 2240 [max_chunks_per_query: <int> | default = 2000000] 2241 2242 # The limit to length of chunk store queries. 0 to disable. 2243 # CLI flag: -store.max-query-length 2244 [max_query_length: <duration> | default = 721h] 2245 2246 # Maximum number of queries that will be scheduled in parallel by the frontend. 2247 # CLI flag: -querier.max-query-parallelism 2248 [max_query_parallelism: <int> | default = 32] 2249 2250 # Limit the maximum of unique series that is returned by a metric query. 2251 # When the limit is reached an error is returned. 2252 # CLI flag: -querier.max-query-series 2253 [max_query_series: <int> | default = 500] 2254 2255 # Cardinality limit for index queries. 2256 # CLI flag: -store.cardinality-limit 2257 [cardinality_limit: <int> | default = 100000] 2258 2259 # Maximum number of stream matchers per query. 2260 # CLI flag: -querier.max-streams-matcher-per-query 2261 [max_streams_matchers_per_query: <int> | default = 1000] 2262 2263 # Maximum number of concurrent tail requests. 2264 # CLI flag: -querier.max-concurrent-tail-requests 2265 [max_concurrent_tail_requests: <int> | default = 10] 2266 2267 # Duration to delay the evaluation of rules to ensure. 2268 # CLI flag: -ruler.evaluation-delay-duration 2269 [ruler_evaluation_delay_duration: <duration> | default = 0s] 2270 2271 # Maximum number of rules per rule group per-tenant. 0 to disable. 2272 # CLI flag: -ruler.max-rules-per-rule-group 2273 [ruler_max_rules_per_rule_group: <int> | default = 0] 2274 2275 # Maximum number of rule groups per-tenant. 0 to disable. 2276 # CLI flag: -ruler.max-rule-groups-per-tenant 2277 [ruler_max_rule_groups_per_tenant: <int> | default = 0] 2278 2279 # Retention to apply for the store, if the retention is enable on the compactor side. 2280 # CLI flag: -store.retention 2281 [retention_period: <duration> | default = 744h] 2282 2283 # Per-stream retention to apply, if the retention is enable on the compactor side. 2284 # Example: 2285 # retention_stream: 2286 # - selector: '{namespace="dev"}' 2287 # priority: 1 2288 # period: 24h 2289 # - selector: '{container="nginx"}' 2290 # priority: 1 2291 # period: 744h 2292 # Selector is a Prometheus labels matchers that will apply the `period` retention only if 2293 # the stream is matching. In case multiple stream are matching, the highest 2294 # priority will be picked. If no rule is matched the `retention_period` is used. 2295 [retention_stream: <array> | default = none] 2296 2297 # Feature renamed to 'runtime configuration', flag deprecated in favor of -runtime-config.file 2298 # (runtime_config.file in YAML). 2299 # CLI flag: -limits.per-user-override-config 2300 [per_tenant_override_config: <string>] 2301 2302 # Feature renamed to 'runtime configuration'; flag deprecated in favor of 2303 # -runtime-config.reload-period (runtime_config.period in YAML). 2304 # CLI flag: -limits.per-user-override-period 2305 [per_tenant_override_period: <duration> | default = 10s] 2306 2307 # Most recent allowed cacheable result per-tenant, to prevent caching very recent results that 2308 # might still be in flux. 2309 # CLI flag: -frontend.max-cache-freshness 2310 [max_cache_freshness_per_query: <duration> | default = 1m] 2311 2312 # Maximum number of queriers that can handle requests for a single tenant. If 2313 # set to 0 or value higher than number of available queriers, *all* queriers 2314 # will handle requests for the tenant. Each frontend (or query-scheduler, if 2315 # used) will select the same set of queriers for the same tenant (given that all 2316 # queriers are connected to all frontends / query-schedulers). This option only 2317 # works with queriers connecting to the query-frontend / query-scheduler, not 2318 # when using downstream URL. 2319 # CLI flag: -frontend.max-queriers-per-tenant 2320 [max_queriers_per_tenant: <int> | default = 0] 2321 2322 # Maximum byte rate per second per stream, 2323 # also expressible in human readable forms (1MB, 256KB, etc). 2324 # CLI flag: -ingester.per-stream-rate-limit 2325 [per_stream_rate_limit: <string|int> | default = "3MB"] 2326 2327 # Maximum burst bytes per stream, 2328 # also expressible in human readable forms (1MB, 256KB, etc). 2329 # This is how far above the rate limit a stream can "burst" before the stream is limited. 2330 # CLI flag: -ingester.per-stream-rate-limit-burst 2331 [per_stream_rate_limit_burst: <string|int> | default = "15MB"] 2332 2333 # Limit how far back in time series data and metadata can be queried, 2334 # up until lookback duration ago. 2335 # This limit is enforced in the query frontend, the querier and the ruler. 2336 # If the requested time range is outside the allowed range, the request will not fail, 2337 # but will be modified to only query data within the allowed time range. 2338 # The default value of 0 does not set a limit. 2339 # CLI flag: -querier.max-query-lookback 2340 [max_query_lookback: <duration> | default = 0] 2341 2342 # Disable recording rules remote-write. 2343 [ruler_remote_write_disabled: <boolean> | default = false] 2344 2345 # The URL of the endpoint to send samples to. 2346 [ruler_remote_write_url: <string>] 2347 2348 # Timeout for requests to the remote write endpoint. 2349 [ruler_remote_write_timeout: <duration>] 2350 2351 # Custom HTTP headers to be sent along with each remote write request. 2352 # Be aware that headers that are set by Loki itself can't be overwritten. 2353 [ruler_remote_write_headers: <headers>] 2354 2355 # List of remote write relabel configurations. 2356 [ruler_remote_write_relabel_configs: <relabel_config>] 2357 2358 # Number of samples to buffer per shard before we block reading of more 2359 # samples from the WAL. It is recommended to have enough capacity in each 2360 # shard to buffer several requests to keep throughput up while processing 2361 # occasional slow remote requests. 2362 [ruler_remote_write_queue_capacity: <int>] 2363 2364 # Minimum number of shards, i.e. amount of concurrency. 2365 [ruler_remote_write_queue_min_shards: <int>] 2366 2367 # Maximum number of shards, i.e. amount of concurrency. 2368 [ruler_remote_write_queue_max_shards: <int>] 2369 2370 # Maximum number of samples per send. 2371 [ruler_remote_write_queue_max_samples_per_send: <int>] 2372 2373 # Maximum time a sample will wait in buffer. 2374 [ruler_remote_write_queue_batch_send_deadline: <duration>] 2375 2376 # Initial retry delay. Gets doubled for every retry. 2377 [ruler_remote_write_queue_min_backoff: <duration>] 2378 2379 # Maximum retry delay. 2380 [ruler_remote_write_queue_max_backoff: <duration>] 2381 # Retry upon receiving a 429 status code from the remote-write storage. 2382 # This is experimental and might change in the future. 2383 [ruler_remote_write_queue_retry_on_ratelimit: <boolean>] 2384 2385 # Configures AWS's Signature Verification 4 signing process to 2386 # sign every remote write request. 2387 [ruler_remote_write_sigv4_config: <sigv4_config>] 2388 2389 # Limit queries that can be sharded. 2390 # Queries within the time range of now and now minus this sharding lookback 2391 # are not sharded. The default value of 0s disables the lookback, causing 2392 # sharding of all queries at all times. 2393 # CLI flag: -frontend.min-sharding-lookback 2394 [min_sharding_lookback: <duration> | default = 0s] 2395 2396 # Split queries by an interval and execute in parallel, any value less than zero disables it. 2397 # This also determines how cache keys are chosen when result caching is enabled 2398 # CLI flag: -querier.split-queries-by-interval 2399 [split_queries_by_interval: <duration> | default = 30m] 2400 ``` 2401 2402 ## sigv4_config 2403 2404 The `sigv4_config` block configures AWS's Signature Verification 4 signing process to 2405 sign every remote write request. 2406 2407 ```yaml 2408 # The AWS region. If blank, the region from the default credentials chain 2409 # is used. 2410 [region: <string>] 2411 2412 # The AWS API keys. If blank, the environment variables `AWS_ACCESS_KEY_ID` 2413 # and `AWS_SECRET_ACCESS_KEY` are used. 2414 [access_key: <string>] 2415 [secret_key: <secret>] 2416 2417 # Named AWS profile used to authenticate. 2418 [profile: <string>] 2419 2420 # AWS Role ARN, an alternative to using AWS API keys. 2421 [role_arn: <string>] 2422 ``` 2423 2424 ### grpc_client_config 2425 2426 The `grpc_client_config` block configures a client connection to a gRPC service. 2427 2428 ```yaml 2429 # The maximum size in bytes the client can receive. 2430 # CLI flag: -<prefix>.grpc-max-recv-msg-size 2431 [max_recv_msg_size: <int> | default = 104857600] 2432 2433 # The maximum size in bytes the client can send. 2434 # CLI flag: -<prefix>.grpc-max-send-msg-size 2435 [max_send_msg_size: <int> | default = 16777216] 2436 2437 # Use compression when sending messages. Supported values are: 'gzip', 'snappy', 2438 # and '' (disable compression). 2439 # CLI flag: -<prefix>.grpc-compression 2440 [grpc_compression: <string> | default = ''] 2441 2442 # Rate limit for gRPC client. 0 is disabled. 2443 # CLI flag: -<prefix>.grpc-client-rate-limit 2444 [rate_limit: <float> | default = 0] 2445 2446 # Rate limit burst for gRPC client. 2447 # CLI flag: -<prefix>.grpc-client-rate-limit-burst 2448 [rate_limit_burst: <int> | default = 0] 2449 2450 # Enable backoff and retry when a rate limit is hit. 2451 # CLI flag: -<prefix>.backoff-on-ratelimits 2452 [backoff_on_ratelimits: <boolean> | default = false] 2453 2454 # Configures backoff when enabled. 2455 backoff_config: 2456 # Minimum delay when backing off. 2457 # CLI flag: -<prefix>.backoff-min-period 2458 [min_period: <duration> | default = 100ms] 2459 2460 # The maximum delay when backing off. 2461 # CLI flag: -<prefix>.backoff-max-period 2462 [max_period: <duration> | default = 10s] 2463 2464 # Number of times to backoff and retry before failing. 2465 # CLI flag: -<prefix>.backoff-retries 2466 [max_retries: <int> | default = 10] 2467 ``` 2468 2469 ## index_gateway 2470 2471 The `index_gateway` block configures the Loki index gateway server, responsible for serving index queries 2472 without the need to constantly interact with the object store. 2473 2474 ```yaml 2475 # Defines in which mode the index gateway server will operate (default to 'simple'). 2476 # It supports two modes: 2477 # 'simple': an index gateway server instance is responsible for handling, 2478 # storing and returning requests for all indices for all tenants. 2479 # 'ring': an index gateway server instance is responsible for a subset of tenants instead 2480 # of all tenants. 2481 [mode: <string> | default = simple] 2482 2483 # Defines the ring to be used by the index gateway servers and clients in case the servers 2484 # are configured to run in 'ring' mode. In case this isn't configured, this block supports 2485 # inheriting configuration from the common ring section. 2486 [ring: <ring>] 2487 ``` 2488 2489 ## table_manager 2490 2491 The `table_manager` block configures the Loki table-manager. 2492 2493 ```yaml 2494 # Master 'off-switch' for table capacity updates, e.g. when troubleshooting. 2495 # CLI flag: -table-manager.throughput-updates-disabled 2496 [throughput_updates_disabled: <boolean> | default = false] 2497 2498 # Master 'on-switch' for table retention deletions. 2499 # CLI flag: -table-manager.retention-deletes-enabled 2500 [retention_deletes_enabled: <boolean> | default = false] 2501 2502 # How far back tables will be kept before they are deleted. 0s disables 2503 # deletion. The retention period must be a multiple of the index / chunks 2504 # table "period" (see period_config). 2505 # CLI flag: -table-manager.retention-period 2506 [retention_period: <duration> | default = 0s] 2507 2508 # Period with which the table manager will poll for tables. 2509 # CLI flag: -table-manager.poll-interval 2510 [poll_interval: <duration> | default = 2m] 2511 2512 # Duration a table will be created before it is needed. 2513 # CLI flag: -table-manager.periodic-table.grace-period 2514 [creation_grace_period: <duration> | default = 10m] 2515 2516 # Configures management of the index tables for DynamoDB. 2517 # The CLI flags prefix for this block config is: table-manager.index-table 2518 index_tables_provisioning: <provision_config> 2519 2520 # Configures management of the chunk tables for DynamoDB. 2521 # The CLI flags prefix for this block config is: table-manager.chunk-table 2522 chunk_tables_provisioning: <provision_config> 2523 ``` 2524 2525 ### provision_config 2526 2527 The `provision_config` block configures provisioning capacity for DynamoDB. 2528 2529 ```yaml 2530 # Enables on-demand throughput provisioning for the storage 2531 # provider, if supported. Applies only to tables which are not autoscaled. 2532 # CLI flag: -<prefix>.enable-ondemand-throughput-mode 2533 [enable_ondemand_throughput_mode: <boolean> | default = false] 2534 2535 # DynamoDB table default write throughput. 2536 # CLI flag: -<prefix>.write-throughput 2537 [provisioned_write_throughput: <int> | default = 3000] 2538 2539 # DynamoDB table default read throughput. 2540 # CLI flag: -<prefix>.read-throughput 2541 [provisioned_read_throughput: <int> | default = 300] 2542 2543 # Enables on-demand throughput provisioning for the storage provide, 2544 # if supported. Applies only to tables which are not autoscaled. 2545 # CLI flag: -<prefix>.inactive-enable-ondemand-throughput-mode 2546 [enable_inactive_throughput_on_demand_mode: <boolean> | default = false] 2547 2548 # DynamoDB table write throughput for inactive tables. 2549 # CLI flag: -<prefix>.inactive-write-throughput 2550 [inactive_write_throughput: <int> | default = 1] 2551 2552 # DynamoDB table read throughput for inactive tables. 2553 # CLI flag: -<prefix>.inactive-read-throughput 2554 [inactive_read_throughput: <int> | default = 300] 2555 2556 # Active table write autoscale config. 2557 # The CLI flags prefix for this block config is: -<prefix>.write-throughput 2558 [write_scale: <auto_scaling_config>] 2559 2560 # Inactive table write autoscale config. 2561 # The CLI flags prefix for this block config is: -<prefix>.inactive-write-throughput 2562 [inactive_write_scale: <auto_scaling_config>] 2563 2564 # Number of last inactive tables to enable write autoscale. 2565 # CLI flag: -<prefix>.enable-ondemand-throughput-mode 2566 [inactive_write_scale_lastn: <int>] 2567 2568 # Active table read autoscale config. 2569 # The CLI flags prefix for this block config is: -<prefix>.read-throughput 2570 [read_scale: <auto_scaling_config>] 2571 2572 # Inactive table read autoscale config. 2573 # The CLI flags prefix for this block config is: -<prefix>.inactive-read-throughput 2574 [inactive_read_scale: <auto_scaling_config>] 2575 2576 # Number of last inactive tables to enable read autoscale. 2577 # CLI flag: -<prefix>.enable-ondemand-throughput-mode 2578 [inactive_read_scale_lastn: <int>] 2579 ``` 2580 2581 #### auto_scaling_config 2582 2583 The `auto_scaling_config` block configures autoscaling for DynamoDB. 2584 2585 ```yaml 2586 # Whether or not autoscaling should be enabled. 2587 # CLI flag: -<prefix>.scale.enabled 2588 [enabled: <boolean>: default = false] 2589 2590 # AWS AutoScaling role ARN. 2591 # CLI flag: -<prefix>.scale.role-arn 2592 [role_arn: <string>] 2593 2594 # DynamoDB minimum provision capacity. 2595 # CLI flag: -<prefix>.scale.min-capacity 2596 [min_capacity: <int> | default = 3000] 2597 2598 # DynamoDB maximum provision capacity. 2599 # CLI flag: -<prefix>.scale.max-capacity 2600 [max_capacity: <int> | default = 6000] 2601 2602 # DynamoDB minimum seconds between each autoscale up. 2603 # CLI flag: -<prefix>.scale.out-cooldown 2604 [out_cooldown: <int> | default = 1800] 2605 2606 # DynamoDB minimum seconds between each autoscale down. 2607 # CLI flag: -<prefix>.scale.in-cooldown 2608 [in_cooldown: <int> | default = 1800] 2609 2610 # DynamoDB target ratio of consumed capacity to provisioned capacity. 2611 # CLI flag: -<prefix>.scale.target-value 2612 [target: <float> | default = 80] 2613 ``` 2614 2615 ## tracing 2616 2617 The `tracing` block configures tracing for Jaeger. Currently limited to disable auto-configuration per [environment variables](https://www.jaegertracing.io/docs/1.16/client-features/) only. 2618 2619 ```yaml 2620 # Whether or not tracing should be enabled. 2621 # CLI flag: -tracing.enabled 2622 [enabled: <boolean>: default = true] 2623 ``` 2624 2625 ## common 2626 2627 The `common` block sets common definitions to be shared by different components. 2628 This way, one doesn't have to replicate configuration in multiple places. 2629 2630 ```yaml 2631 # A common storage configuration to be used by the different Loki components. 2632 [storage: <storage>] 2633 2634 # When defined, the given prefix will be present in front of the endpoint paths. 2635 [path_prefix: <string>] 2636 2637 # How many times incoming data should be replicated to the ingester component. 2638 [replication_factor: <int> | default = 3] 2639 2640 # When true, the ingester, compactor, and query_scheduler ring tokens will be saved 2641 # to files in the path_prefix directory. Loki will error if you set this to true 2642 # and path_prefix is empty. 2643 [persist_tokens: <boolean>: default = false] 2644 2645 # A common list of net interfaces used internally to look for addresses. 2646 # If a more specific "instance_interface_names" is set, this is ignored. 2647 # If "instance_interface_names" under the common ring section is configured, 2648 # this common "instance_interface_names" is only applied to the frontend, but not for 2649 # ring related components (ex: distributor, ruler, etc). 2650 [instance_interface_names: <list of string> | default = [<private network interfaces>]] 2651 2652 # A common address used by Loki components to advertise their address. 2653 # If a more specific "instance_addr" is set, this is ignored. 2654 # If "instance_addr" under the common ring section is configured, this common "instance_addr" 2655 # is only applied to the frontend, but not for ring related components (ex: distributor, ruler, etc). 2656 [instance_addr: <string>] 2657 2658 # A common ring configuration to be used by all Loki rings. 2659 # If a common ring is given, its values are used to define any undefined ring values. 2660 # For instance, you can expect the `heartbeat_period` defined in the common section 2661 # to be used by the distributor's ring, but only if the distributor's ring itself 2662 # doesn't have a `heartbeat_period` set. 2663 [ring: <ring>] 2664 2665 # Address and port number where the compactor API is served. 2666 # CLI flag: -common.compactor-address 2667 [compactor_address: <string> | default = ""] 2668 2669 # Groupcache is an in-process, distributed cache that behaves similarly to memcached but is built-in to Loki 2670 groupcache: 2671 # Enable groupcache 2672 # CLI flag: -common.groupcache.enabled 2673 [enabled: <boolean>: default = false] 2674 # Set the maximum available memory to use for each groupcache group 2675 # NOTE: there are 3 caches (result, chunk, and index query), so the maximum used memory will be *triple* the value specified here. 2676 # CLI flag: -common.groupcache.capacity-per-cache-mb 2677 [capacity_per_cache_mb: <int>: default = 100] 2678 ``` 2679 2680 ## analytics 2681 2682 The `analytics` block configures the reporting of Loki analytics to grafana.com. 2683 2684 ```yaml 2685 # By default, Loki will send anonymous, but uniquely-identifiable usage and configuration 2686 # analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/ 2687 # 2688 # Statistics help us better understand how Loki is used, and they show us performance 2689 # levels for most users. This helps us prioritize features and documentation. 2690 # For more information on what's sent, look at 2691 # https://github.com/grafana/loki/blob/main/pkg/usagestats/stats.go 2692 # Refer to the buildReport method to see what goes into a report. 2693 # 2694 # When true, enables usage reporting. 2695 # CLI flag: -reporting.enabled 2696 [reporting_enabled: <boolean>: default = true] 2697 ``` 2698 2699 ### storage 2700 2701 The common `storage` block defines a common storage to be reused by different 2702 components as a way to facilitate storage configuration. 2703 If any specific configuration for an object storage client have been provided elsewhere in the configuration file, the specific configuration will supersede the common storage configuration. 2704 2705 ```yaml 2706 # Configures Azure as the common storage. 2707 [azure: <azure_storage_config>] 2708 2709 # Configures GCS as the common storage. 2710 [gcs: <gcs_storage_config>] 2711 2712 # Configures S3 as the common storage. 2713 [s3: <s3_storage_config>] 2714 2715 # Configures Swift as the common storage. 2716 [swift: <swift_storage_config>] 2717 2718 # Configures a (local) file system as the common storage. 2719 [filesystem: <filesystem>] 2720 2721 # Configures Baidu Object Storage (BOS) as the common storage. 2722 [bos: <bos_storage_config>] 2723 2724 # The `hedging_config` configures how to hedge requests for the storage. 2725 [hedging: <hedging_config>] 2726 2727 ``` 2728 2729 ### filesystem 2730 2731 The common `filesystem` block configures a local file system as a general 2732 storage for various types of data generated by Loki. 2733 2734 ```yaml 2735 # File system directory to be used for chunks storage. 2736 [chunks_directory: <filename> | default = ""] 2737 2738 # File system directory to be used for rules storage. 2739 [rules_directory: <filename> | default = ""] 2740 ``` 2741 2742 ### ring 2743 2744 The common `ring` block defines a ring configuration used by a Loki component. 2745 2746 ```yaml 2747 # The key-value store used to share the hash ring across multiple instances. 2748 kvstore: 2749 # Backend storage to use for the ring. Supported values are: consul, etcd, 2750 # inmemory, memberlist, multi. 2751 # CLI flag: -<prefix>.store 2752 [store: <string> | default = "memberlist"] 2753 2754 # The prefix for the keys in the store. Should end with a /. 2755 # CLI flag: -<prefix>.prefix 2756 [prefix: <string> | default = "collectors/"] 2757 2758 # The consul_config configures the consul client. 2759 [consul: <consul_config>] 2760 2761 # The etcd_config configures the etcd client. 2762 [etcd: <etcd_config>] 2763 2764 multi: 2765 # Primary backend storage used by multi-client. 2766 # CLI flag: -<prefix>.multi.primary 2767 [primary: <string> | default = ""] 2768 2769 # Secondary backend storage used by multi-client. 2770 # CLI flag: -<prefix>.multi.secondary 2771 [secondary: <string> | default = ""] 2772 2773 # Mirror writes to secondary store. 2774 # CLI flag: -<prefix>.multi.mirror-enabled 2775 [mirror_enabled: <boolean> | default = false] 2776 2777 # Timeout for storing value to secondary store. 2778 # CLI flag: -<prefix>.multi.mirror-timeout 2779 [mirror_timeout: <duration> | default = 2s] 2780 2781 # Interval between heartbeats sent to the ring. 0 = disabled. 2782 # CLI flag: -<prefix>.heartbeat-period 2783 [heartbeat_period: <duration> | default = 15s] 2784 2785 # The heartbeat timeout after which store gateways are considered unhealthy 2786 # within the ring. 0 = never (timeout disabled). This option needs be set both 2787 # on the store-gateway and querier when running in microservices mode. 2788 # CLI flag: -<prefix>.heartbeat-timeout 2789 [heartbeat_timeout: <duration> | default = 1m] 2790 2791 # File path where tokens are stored. If empty, tokens are neither stored at 2792 # shutdown nor restored at startup. 2793 # CLI flag: -<prefix>.tokens-file-path 2794 [tokens_file_path: <string> | default = ""] 2795 2796 # True to enable zone-awareness and replicate blocks across different 2797 # availability zones. 2798 # CLI flag: -<prefix>.zone-awareness-enabled 2799 [zone_awareness_enabled: <boolean> | default = false] 2800 2801 # Name of network interface to read addresses from. 2802 # CLI flag: -<prefix>.instance-interface-names 2803 [instance_interface_names: <list of string> | default = [<private network interfaces>]] 2804 2805 # IP address to advertise in the ring. 2806 # CLI flag: -<prefix>.instance-addr 2807 [instance_addr: <list of string> | default = first from instance_interface_names] 2808 2809 # Port to advertise in the ring 2810 # CLI flag: -<prefix>.instance-port 2811 [instance_port: <list of string> | default = server.grpc-listen-port] 2812 2813 # Instance ID to register in the ring. 2814 # CLI flag: -<prefix>.instance-id 2815 [instance_id: <list of string> | default = os.Hostname()] 2816 2817 # The availability zone where this instance is running. Required if 2818 # zone-awareness is enabled. 2819 # CLI flag: -<prefix>.instance-availability-zone 2820 [instance_availability_zone: <string> | default = ""] 2821 ``` 2822 2823 ## Runtime Configuration file 2824 2825 Loki has a concept of "runtime config" file, which is simply a file that is reloaded while Loki is running. It is used by some Loki components to allow operator to change some aspects of Loki configuration without restarting it. File is specified by using `-runtime-config.file=<filename>` flag and reload period (which defaults to 10 seconds) can be changed by `-runtime-config.reload-period=<duration>` flag. Previously this mechanism was only used by limits overrides, and flags were called `-limits.per-user-override-config=<filename>` and `-limits.per-user-override-period=10s` respectively. These are still used, if `-runtime-config.file=<filename>` is not specified. 2826 2827 At the moment, two components use runtime configuration: limits and multi KV store. 2828 2829 Options for runtime configuration reload can also be configured via YAML: 2830 2831 ```yaml 2832 # Configuration file to periodically check and reload. 2833 [file: <string>: default = empty] 2834 2835 # How often to check the file. 2836 [period: <duration>: default 10s] 2837 ``` 2838 2839 Example runtime configuration file: 2840 2841 ```yaml 2842 overrides: 2843 tenant1: 2844 ingestion_rate_mb: 10 2845 max_streams_per_user: 100000 2846 max_chunks_per_query: 100000 2847 tenant2: 2848 max_streams_per_user: 1000000 2849 max_chunks_per_query: 1000000 2850 2851 multi_kv_config: 2852 mirror-enabled: false 2853 primary: consul 2854 ``` 2855 2856 ## Accept out-of-order writes 2857 2858 Since the beginning of Loki, log entries had to be written to Loki in order 2859 by time. 2860 This limitation has been lifted. 2861 Out-of-order writes are enabled globally by default, but can be disabled/enabled 2862 on a cluster or per-tenant basis. 2863 2864 - To disable out-of-order writes for all tenants, 2865 place in the `limits_config` section: 2866 2867 ``` 2868 limits_config: 2869 unordered_writes: false 2870 ``` 2871 2872 - To disable out-of-order writes for specific tenants, 2873 configure a runtime configuration file: 2874 2875 ``` 2876 runtime_config: overrides.yaml 2877 ``` 2878 2879 In the `overrides.yaml` file, add `unordered_writes` for each tenant 2880 permitted to have out-of-order writes: 2881 2882 ``` 2883 overrides: 2884 "tenantA": 2885 unordered_writes: false 2886 ``` 2887 2888 How far into the past accepted out-of-order log entries may be 2889 is configurable with `max_chunk_age`. 2890 `max_chunk_age` defaults to 1 hour. 2891 Loki calculates the earliest time that out-of-order entries may have 2892 and be accepted with 2893 2894 ``` 2895 time_of_most_recent_line - (max_chunk_age/2) 2896 ``` 2897 2898 Log entries with timestamps that are after this earliest time are accepted. 2899 Log entries further back in time return an out-of-order error. 2900 2901 For example, if `max_chunk_age` is 2 hours 2902 and the stream `{foo="bar"}` has one entry at `8:00`, 2903 Loki will accept data for that stream as far back in time as `7:00`. 2904 If another log line is written at `10:00`, 2905 Loki will accept data for that stream as far back in time as `9:00`.