github.com/yankunsam/loki/v2@v2.6.3-0.20220817130409-389df5235c27/docs/sources/clients/fluentbit/_index.md (about) 1 --- 2 title: Fluent Bit 3 weight: 50 4 --- 5 # Fluent Bit Loki Output 6 7 [Fluent Bit](https://fluentbit.io/) is a fast and lightweight logs and metrics processor and forwarder that can be configured with the [Grafana Loki output plugin](https://docs.fluentbit.io/manual/pipeline/outputs/loki) to ship logs to Loki. You can define which log files you want to collect using the [`Tail`](https://docs.fluentbit.io/manual/pipeline/inputs/tail) or [`Stdin`](https://docs.fluentbit.io/manual/pipeline/inputs/standard-input) data pipeline input. Additionally, Fluent Bit supports multiple `Filter` and `Parser` plugins (`Kubernetes`, `JSON`, etc.) to structure and alter log lines. 8 9 ## Usage 10 11 ### Docker 12 13 You can run a Fluent Bit container with Loki output plugin pre-installed using our [Docker Hub](https://hub.docker.com/r/grafana/fluent-bit-plugin-loki) image: 14 15 ```bash 16 docker run -v /var/log:/var/log \ 17 -e LOG_PATH="/var/log/*.log" -e LOKI_URL="http://localhost:3100/loki/api/v1/push" \ 18 grafana/fluent-bit-plugin-loki:latest 19 ``` 20 21 ### Kubernetes 22 23 You can run Fluent Bit as a [Daemonset](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) to collect all your Kubernetes workload logs. 24 25 To do so you can use our [Fluent Bit helm chart](https://github.com/grafana/helm-charts/tree/main/charts/fluent-bit): 26 27 ```bash 28 helm repo add grafana https://grafana.github.io/helm-charts 29 helm repo update 30 helm upgrade --install fluent-bit grafana/fluent-bit \ 31 --set loki.serviceName=loki.svc.cluster.local 32 ``` 33 34 By default it will collect all containers logs and extract labels from Kubernetes API (`container_name`, `namespace`, etc..). 35 36 Alternatively you can install the Loki and Fluent Bit all together using: 37 38 ```bash 39 helm upgrade --install loki-stack grafana/loki-stack \ 40 --set fluent-bit.enabled=true,promtail.enabled=false 41 ``` 42 43 ### AWS Elastic Container Service (ECS) 44 45 You can use fluent-bit Loki Docker image as a Firelens log router in AWS ECS. 46 For more information about this see our [AWS documentation](../aws/ecs) 47 48 ### Local 49 50 First, you need to follow the [instructions](https://github.com/grafana/loki/blob/main/clients/cmd/fluent-bit/README.md) in order to build the plugin dynamic library. 51 52 The assuming you have Fluent Bit installed in your `$PATH` you can run the plugin using: 53 54 ```bash 55 fluent-bit -e /path/to/built/out_grafana_loki.so -c fluent-bit.conf 56 ``` 57 58 You can also adapt your plugins.conf, removing the need to change the command line options: 59 60 ```conf 61 [PLUGINS] 62 Path /path/to/built/out_grafana_loki.so 63 ``` 64 65 ## Configuration Options 66 67 | Key | Description | Default | 68 |----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------| 69 | Url | Url of loki server API endpoint. | http://localhost:3100/loki/api/v1/push | 70 | TenantID | The tenant ID used by default to push logs to Loki. If omitted or empty it assumes Loki is running in single-tenant mode and no `X-Scope-OrgID` header is sent. | "" | 71 | BatchWait | Time to wait before send a log batch to Loki, full or not. | 1s | 72 | BatchSize | Log batch size to send a log batch to Loki (unit: Bytes). | 10 KiB (10 * 1024 Bytes) | 73 | Timeout | Maximum time to wait for loki server to respond to a request. | 10s | 74 | MinBackoff | Initial backoff time between retries. | 500ms | 75 | MaxBackoff | Maximum backoff time between retries. | 5m | 76 | MaxRetries | Maximum number of retries when sending batches. Setting it to `0` will retry indefinitely. | 10 | 77 | Labels | labels for API requests. | {job="fluent-bit"} | 78 | LogLevel | LogLevel for plugin logger. | "info" | 79 | RemoveKeys | Specify removing keys. | none | 80 | AutoKubernetesLabels | If set to true, it will add all Kubernetes labels to Loki labels | false | 81 | LabelKeys | Comma separated list of keys to use as stream labels. All other keys will be placed into the log line. LabelKeys is deactivated when using `LabelMapPath` label mapping configuration. | none | 82 | LineFormat | Format to use when flattening the record to a log line. Valid values are "json" or "key_value". If set to "json" the log line sent to Loki will be the fluentd record (excluding any keys extracted out as labels) dumped as json. If set to "key_value", the log line will be each item in the record concatenated together (separated by a single space) in the format <key>=<value>. | json | 83 | DropSingleKey | If set to true and after extracting label_keys a record only has a single key remaining, the log line sent to Loki will just be the value of the record key. | true | 84 | LabelMapPath | Path to a json file defining how to transform nested records. | none | 85 | Buffer | Enable buffering mechanism | false | 86 | BufferType | Specify the buffering mechanism to use (currently only dque is implemented). | dque | 87 | DqueDir | Path to the directory for queued logs | /tmp/flb-storage/loki | 88 | DqueSegmentSize | Segment size in terms of number of records per segment | 500 | 89 | DqueSync | Whether to fsync each queue change. Specify no fsync with "normal", and fsync with "full". | "normal" | 90 | DqueName | Queue name, must be uniq per output | dque | 91 92 ### Labels 93 94 Labels are used to [query logs](../../logql) `{container_name="nginx", cluster="us-west1"}`, they are usually metadata about the workload producing the log stream (`instance`, `container_name`, `region`, `cluster`, `level`). In Loki labels are indexed consequently you should be cautious when choosing them (high cardinality label values can have performance drastic impact). 95 96 You can use `Labels`, `RemoveKeys` , `LabelKeys` and `LabelMapPath` to how the output plugin will perform labels extraction. 97 98 ### AutoKubernetesLabels 99 100 If set to true, it will add all Kubernetes labels to Loki labels automatically and ignore parameters `LabelKeys`, LabelMapPath. 101 102 ### LabelMapPath 103 104 When using the `Parser` and `Filter` plugins Fluent Bit can extract and add data to the current record/log data. While Loki labels are key value pair, record data can be nested structures. 105 You can pass a JSON file that defines how to extract labels from each record. Each json key from the file will be matched with the log record to find label values. Values from the configuration are used as label names. 106 107 Considering the record below : 108 109 ```json 110 { 111 "kubernetes": { 112 "container_name": "promtail", 113 "pod_name": "promtail-xxx", 114 "namespace_name": "prod", 115 "labels" : { 116 "team": "x-men" 117 } 118 }, 119 "HOSTNAME": "docker-desktop", 120 "log" : "a log line", 121 "time": "20190926T152206Z" 122 } 123 ``` 124 125 and a LabelMap file as follow : 126 127 ```json 128 { 129 "kubernetes": { 130 "container_name": "container", 131 "pod_name": "pod", 132 "namespace_name": "namespace", 133 "labels" : { 134 "team": "team" 135 } 136 } 137 } 138 ``` 139 140 The labels extracted will be `{team="x-men", container="promtail", pod="promtail-xxx", namespace="prod"}`. 141 142 If you don't want the `kubernetes` and `HOSTNAME` fields to appear in the log line you can use the `RemoveKeys` configuration field. (e.g. `RemoveKeys kubernetes,HOSTNAME`). 143 144 ### Buffering 145 146 Buffering refers to the ability to store the records somewhere, and while they are processed and delivered, still be able to store more. The Loki output plugin can be blocked by the Loki client because of its design: 147 148 - If the BatchSize is over the limit, the output plugin pauses receiving new records until the pending batch is successfully sent to the server 149 - If the Loki server is unreachable (retry 429s, 500s and connection-level errors), the output plugin blocks new records until the Loki server is available again, and the pending batch is successfully sent to the server or as long as the maximum number of attempts has been reached within configured back-off mechanism 150 151 The blocking state with some of the input plugins is not acceptable, because it can have an undesirable side effect on the part that generates the logs. Fluent Bit implements a buffering mechanism that is based on parallel processing. Therefore, it cannot send logs in order. There are two ways of handling the out-of-order logs: 152 153 - Configure Loki to [accept out-of-order writes](../../configuration/#accept-out-of-order-writes). 154 155 - Configure the Loki output plugin to use the buffering mechanism based on [`dque`](https://github.com/joncrlsn/dque), which is compatible with the Loki server strict time ordering: 156 157 ```properties 158 [Output] 159 Name grafana-loki 160 Match * 161 Url http://localhost:3100/loki/api/v1/push 162 Buffer true 163 DqueSegmentSize 8096 164 DqueDir /tmp/flb-storage/buffer 165 DqueName loki.0 166 ``` 167 168 ### Configuration examples 169 170 To configure the Loki output plugin add this section to fluent-bit.conf 171 172 ```properties 173 [Output] 174 Name grafana-loki 175 Match * 176 Url http://localhost:3100/loki/api/v1/push 177 BatchWait 1s 178 BatchSize 30720 179 # (30KiB) 180 Labels {test="fluent-bit-go", lang="Golang"} 181 RemoveKeys key1,key2 182 LabelKeys key3,key4 183 LineFormat key_value 184 ``` 185 186 ```properties 187 [Output] 188 Name grafana-loki 189 Match * 190 Url http://localhost:3100/loki/api/v1/push 191 BatchWait 1s 192 BatchSize 30720 # (30KiB) 193 AutoKubernetesLabels true 194 RemoveKeys key1,key2 195 ``` 196 197 A full [example configuration file](https://github.com/grafana/loki/blob/master/clients/cmd/fluent-bit/fluent-bit.conf) is also available in the Loki repository. 198 199 ### Running multiple plugin instances 200 201 You can run multiple plugin instances in the same fluent-bit process, for example if you want to push to different Loki servers or route logs into different Loki tenant IDs. To do so, add additional `[Output]` sections.