github.com/yankunsam/loki/v2@v2.6.3-0.20220817130409-389df5235c27/docs/sources/tools/logcli.md (about) 1 --- 2 title: LogCLI 3 weight: 20 4 aliases: 5 - /docs/loki/latest/getting-started/logcli/ 6 --- 7 # LogCLI, Grafana Loki's command-line interface 8 9 LogCLI is the command-line interface to Grafana Loki. 10 It facilitates running [LogQL]({{< relref "../logql/_index.md" >}}) 11 queries against a Loki instance. 12 13 ## Installation 14 15 ### Binary (Recommended) 16 17 Download the `logcli` binary from the 18 [Loki releases page](https://github.com/grafana/loki/releases). 19 20 ### Build LogCLI from source 21 22 Clone the Loki repository and build `logcli` from source: 23 24 ```bash 25 git clone https://github.com/grafana/loki.git 26 cd loki 27 make logcli 28 ``` 29 30 Optionally, move the binary into a directory that is part of your `$PATH`. 31 32 ```bash 33 cp cmd/logcli/logcli /usr/local/bin/logcli 34 ``` 35 36 ## LogCLI usage 37 38 ### Grafana Cloud example 39 40 If you are running on Grafana Cloud, use: 41 42 ```bash 43 export LOKI_ADDR=https://logs-us-west1.grafana.net 44 export LOKI_USERNAME=<username> 45 export LOKI_PASSWORD=<password> 46 ``` 47 48 Otherwise you can point LogCLI to a local instance directly 49 without needing a username and password: 50 51 ```bash 52 export LOKI_ADDR=http://localhost:3100 53 ``` 54 55 > Note: If you are running Loki behind a proxy server and you have 56 > authentication configured, you will also have to pass in LOKI_USERNAME 57 > and LOKI_PASSWORD, LOKI_BEARER_TOKEN or LOKI_BEARER_TOKEN_FILE accordingly. 58 59 ```bash 60 $ logcli labels job 61 https://logs-dev-ops-tools1.grafana.net/api/prom/label/job/values 62 loki-ops/consul 63 loki-ops/loki-gw 64 ... 65 66 $ logcli query '{job="loki-ops/consul"}' 67 https://logs-dev-ops-tools1.grafana.net/api/prom/query?query=%7Bjob%3D%22loki-ops%2Fconsul%22%7D&limit=30&start=1529928228&end=1529931828&direction=backward®exp= 68 Common labels: {job="loki-ops/consul", namespace="loki-ops"} 69 2018-06-25T12:52:09Z {instance="consul-8576459955-pl75w"} 2018/06/25 12:52:09 [INFO] raft: Snapshot to 475409 complete 70 2018-06-25T12:52:09Z {instance="consul-8576459955-pl75w"} 2018/06/25 12:52:09 [INFO] raft: Compacting logs from 456973 to 465169 71 ... 72 73 $ logcli series -q --match='{namespace="loki",container_name="loki"}' 74 {app="loki", container_name="loki", controller_revision_hash="loki-57c9df47f4", filename="/var/log/pods/loki_loki-0_8ed03ded-bacb-4b13-a6fe-53a445a15887/loki/0.log", instance="loki-0", job="loki/loki", name="loki", namespace="loki", release="loki", statefulset_kubernetes_io_pod_name="loki-0", stream="stderr"} 75 ``` 76 77 ### Batched queries 78 79 LogCLI sends queries to Loki such that query results arrive in batches. 80 81 The `--limit` option for a `logcli query` command caps the quantity of 82 log lines for a single query. 83 When not set, `--limit` defaults to 30. 84 The limit protects the user from overwhelming the system 85 for cases in which the specified query would have returned a large quantity 86 of log lines. 87 The limit also protects the user from unexpectedly large responses. 88 89 The quantity of log line results that arrive in each batch 90 is set by the `--batch` option in a `logcli query` command. 91 When not set, `--batch` defaults to 1000. 92 93 Setting a `--limit` value larger than the `--batch` value causes the 94 requests from LogCLI to Loki to be batched. 95 Loki has a server-side limit that defaults to 5000 for the maximum quantity 96 of lines returned for a single query. 97 The batching of requests allows you to query for a results set that 98 is larger than the server-side limit, 99 as long as the `--batch` value is less than the server limit. 100 101 Query metadata is output to `stderr` for each batch. 102 Set the `--quiet` option on the `logcli query` command line to suppress 103 the output of the query metadata. 104 105 ### Configuration 106 107 Configuration values are considered in the following order (lowest to highest): 108 109 - Environment variables 110 - Command-line options 111 112 ### LogCLI command reference 113 114 The output of `logcli help`: 115 116 ```nohighlight 117 usage: logcli [<flags>] <command> [<args> ...] 118 119 A command-line for loki. 120 121 Flags: 122 --help Show context-sensitive help (also try --help-long and 123 --help-man). 124 --version Show application version. 125 -q, --quiet Suppress query metadata 126 --stats Show query statistics 127 -o, --output=default Specify output mode [default, raw, jsonl]. raw 128 suppresses log labels and timestamp. 129 -z, --timezone=Local Specify the timezone to use when formatting output 130 timestamps [Local, UTC] 131 --cpuprofile="" Specify the location for writing a CPU profile. 132 --memprofile="" Specify the location for writing a memory profile. 133 --stdin Take input logs from stdin 134 --addr="http://localhost:3100" 135 Server address. Can also be set using LOKI_ADDR env 136 var. 137 --username="" Username for HTTP basic auth. Can also be set using 138 LOKI_USERNAME env var. 139 --password="" Password for HTTP basic auth. Can also be set using 140 LOKI_PASSWORD env var. 141 --ca-cert="" Path to the server Certificate Authority. Can also be 142 set using LOKI_CA_CERT_PATH env var. 143 --tls-skip-verify Server certificate TLS skip verify. 144 --cert="" Path to the client certificate. Can also be set using 145 LOKI_CLIENT_CERT_PATH env var. 146 --key="" Path to the client certificate key. Can also be set 147 using LOKI_CLIENT_KEY_PATH env var. 148 --org-id="" adds X-Scope-OrgID to API requests for representing 149 tenant ID. Useful for requesting tenant data when 150 bypassing an auth gateway. 151 152 Commands: 153 help [<command>...] 154 Show help. 155 156 query [<flags>] <query> 157 Run a LogQL query. 158 159 The "query" command is useful for querying for logs. Logs can be returned in 160 a few output modes: 161 162 raw: log line 163 default: log timestamp + log labels + log line 164 jsonl: JSON response from Loki API of log line 165 166 The output of the log can be specified with the "-o" flag, for example, "-o 167 raw" for the raw output format. 168 169 The "query" command will output extra information about the query and its 170 results, such as the API URL, set of common labels, and set of excluded 171 labels. This extra information can be suppressed with the --quiet flag. 172 173 By default we look over the last hour of data; use --since to modify or 174 provide specific start and end times with --from and --to respectively. 175 176 Notice that when using --from and --to then ensure to use RFC3339Nano time 177 format, but without timezone at the end. The local timezone will be added 178 automatically or if using --timezone flag. 179 180 Example: 181 182 logcli query 183 --timezone=UTC 184 --from="2021-01-19T10:00:00Z" 185 --to="2021-01-19T20:00:00Z" 186 --output=jsonl 187 'my-query' 188 189 The output is limited to 30 entries by default; use --limit to increase. 190 191 While "query" does support metrics queries, its output contains multiple 192 data points between the start and end query time. This output is used to 193 build graphs, similar to what is seen in the Grafana Explore graph view. If 194 you are querying metrics and just want the most recent data point (like what 195 is seen in the Grafana Explore table view), then you should use the 196 "instant-query" command instead. 197 198 instant-query [<flags>] <query> 199 Run an instant LogQL query. 200 201 The "instant-query" command is useful for evaluating a metric query for a 202 single point in time. This is equivalent to the Grafana Explore table view; 203 if you want a metrics query that is used to build a Grafana graph, you 204 should use the "query" command instead. 205 206 This command does not produce useful output when querying for log lines; you 207 should always use the "query" command when you are running log queries. 208 209 For more information about log queries and metric queries, refer to the 210 LogQL documentation: 211 212 https://grafana.com/docs/loki/latest/logql/ 213 214 labels [<flags>] [<label>] 215 Find values for a given label. 216 217 series [<flags>] <matcher> 218 Run series query. 219 220 The "series" command will take the provided label matcher and return all the 221 log streams found in the time window. 222 223 It is possible to send an empty label matcher '{}' to return all streams. 224 225 Use the --analyze-labels flag to get a summary of the labels found in all 226 streams. This is helpful to find high cardinality labels. 227 ``` 228 229 ### LogCLI query command reference 230 231 The output of `logcli help query`: 232 233 ``` 234 usage: logcli query [<flags>] <query> 235 236 Run a LogQL query. 237 238 The "query" command is useful for querying for logs. Logs can be returned in a 239 few output modes: 240 241 raw: log line 242 default: log timestamp + log labels + log line 243 jsonl: JSON response from Loki API of log line 244 245 The output of the log can be specified with the "-o" flag, for example, "-o raw" 246 for the raw output format. 247 248 The "query" command will output extra information about the query and its 249 results, such as the API URL, set of common labels, and set of excluded labels. 250 This extra information can be suppressed with the --quiet flag. 251 252 By default we look over the last hour of data; use --since to modify or provide 253 specific start and end times with --from and --to respectively. 254 255 Notice that when using --from and --to then ensure to use RFC3339Nano time 256 format, but without timezone at the end. The local timezone will be added 257 automatically or if using --timezone flag. 258 259 Example: 260 261 logcli query 262 --timezone=UTC 263 --from="2021-01-19T10:00:00Z" 264 --to="2021-01-19T20:00:00Z" 265 --output=jsonl 266 'my-query' 267 268 The output is limited to 30 entries by default; use --limit to increase. 269 270 While "query" does support metrics queries, its output contains multiple data 271 points between the start and end query time. This output is used to build 272 graphs, similar to what is seen in the Grafana Explore graph view. If you are 273 querying metrics and just want the most recent data point (like what is seen in 274 the Grafana Explore table view), then you should use the "instant-query" command 275 instead. 276 277 Flags: 278 --help Show context-sensitive help (also try --help-long and 279 --help-man). 280 --version Show application version. 281 -q, --quiet Suppress query metadata 282 --stats Show query statistics 283 -o, --output=default Specify output mode [default, raw, jsonl]. raw 284 suppresses log labels and timestamp. 285 -z, --timezone=Local Specify the timezone to use when formatting output 286 timestamps [Local, UTC] 287 --cpuprofile="" Specify the location for writing a CPU profile. 288 --memprofile="" Specify the location for writing a memory profile. 289 --stdin Take input logs from stdin 290 --addr="http://localhost:3100" 291 Server address. Can also be set using LOKI_ADDR env 292 var. 293 --username="" Username for HTTP basic auth. Can also be set using 294 LOKI_USERNAME env var. 295 --password="" Password for HTTP basic auth. Can also be set using 296 LOKI_PASSWORD env var. 297 --ca-cert="" Path to the server Certificate Authority. Can also be 298 set using LOKI_CA_CERT_PATH env var. 299 --tls-skip-verify Server certificate TLS skip verify. 300 --cert="" Path to the client certificate. Can also be set using 301 LOKI_CLIENT_CERT_PATH env var. 302 --key="" Path to the client certificate key. Can also be set 303 using LOKI_CLIENT_KEY_PATH env var. 304 --org-id="" adds X-Scope-OrgID to API requests for representing 305 tenant ID. Useful for requesting tenant data when 306 bypassing an auth gateway. 307 --limit=30 Limit on number of entries to print. 308 --since=1h Lookback window. 309 --from=FROM Start looking for logs at this absolute time 310 (inclusive) 311 --to=TO Stop looking for logs at this absolute time 312 (exclusive) 313 --step=STEP Query resolution step width, for metric queries. 314 Evaluate the query at the specified step over the 315 time range. 316 --interval=INTERVAL Query interval, for log queries. Return entries at 317 the specified interval, ignoring those between. 318 **This parameter is experimental, please see Issue 319 1779** 320 --batch=1000 Query batch size to use until 'limit' is reached 321 --forward Scan forwards through logs. 322 --no-labels Do not print any labels 323 --exclude-label=EXCLUDE-LABEL ... 324 Exclude labels given the provided key during output. 325 --include-label=INCLUDE-LABEL ... 326 Include labels given the provided key during output. 327 --labels-length=0 Set a fixed padding to labels 328 --store-config="" Execute the current query using a configured storage 329 from a given Loki configuration file. 330 --colored-output Show output with colored labels 331 -t, --tail Tail the logs 332 -f, --follow Alias for --tail 333 --delay-for=0 Delay in tailing by number of seconds to accumulate 334 logs for re-ordering 335 336 Args: 337 <query> eg '{foo="bar",baz=~".*blip"} |~ ".*error.*"' 338 ``` 339 340 ### LogCLI labels command reference 341 342 The output of `logcli help labels`: 343 344 ``` 345 usage: logcli labels [<flags>] [<label>] 346 347 Find values for a given label. 348 349 Flags: 350 --help Show context-sensitive help (also try --help-long and 351 --help-man). 352 --version Show application version. 353 -q, --quiet Suppress query metadata 354 --stats Show query statistics 355 -o, --output=default Specify output mode [default, raw, jsonl]. raw 356 suppresses log labels and timestamp. 357 -z, --timezone=Local Specify the timezone to use when formatting output 358 timestamps [Local, UTC] 359 --cpuprofile="" Specify the location for writing a CPU profile. 360 --memprofile="" Specify the location for writing a memory profile. 361 --stdin Take input logs from stdin 362 --addr="http://localhost:3100" 363 Server address. Can also be set using LOKI_ADDR env 364 var. 365 --username="" Username for HTTP basic auth. Can also be set using 366 LOKI_USERNAME env var. 367 --password="" Password for HTTP basic auth. Can also be set using 368 LOKI_PASSWORD env var. 369 --ca-cert="" Path to the server Certificate Authority. Can also be 370 set using LOKI_CA_CERT_PATH env var. 371 --tls-skip-verify Server certificate TLS skip verify. 372 --cert="" Path to the client certificate. Can also be set using 373 LOKI_CLIENT_CERT_PATH env var. 374 --key="" Path to the client certificate key. Can also be set 375 using LOKI_CLIENT_KEY_PATH env var. 376 --org-id="" adds X-Scope-OrgID to API requests for representing 377 tenant ID. Useful for requesting tenant data when 378 bypassing an auth gateway. 379 --since=1h Lookback window. 380 --from=FROM Start looking for labels at this absolute time 381 (inclusive) 382 --to=TO Stop looking for labels at this absolute time 383 (exclusive) 384 385 Args: 386 [<label>] The name of the label. 387 ``` 388 389 ### LogCLI series command reference 390 391 The output of `logcli help series`: 392 393 ``` 394 usage: logcli series [<flags>] <matcher> 395 396 Run series query. 397 398 The "series" command will take the provided label matcher and return all the log 399 streams found in the time window. 400 401 It is possible to send an empty label matcher '{}' to return all streams. 402 403 Use the --analyze-labels flag to get a summary of the labels found in all 404 streams. This is helpful to find high cardinality labels. 405 406 Flags: 407 --help Show context-sensitive help (also try --help-long and 408 --help-man). 409 --version Show application version. 410 -q, --quiet Suppress query metadata 411 --stats Show query statistics 412 -o, --output=default Specify output mode [default, raw, jsonl]. raw 413 suppresses log labels and timestamp. 414 -z, --timezone=Local Specify the timezone to use when formatting output 415 timestamps [Local, UTC] 416 --cpuprofile="" Specify the location for writing a CPU profile. 417 --memprofile="" Specify the location for writing a memory profile. 418 --stdin Take input logs from stdin 419 --addr="http://localhost:3100" 420 Server address. Can also be set using LOKI_ADDR env 421 var. 422 --username="" Username for HTTP basic auth. Can also be set using 423 LOKI_USERNAME env var. 424 --password="" Password for HTTP basic auth. Can also be set using 425 LOKI_PASSWORD env var. 426 --ca-cert="" Path to the server Certificate Authority. Can also be 427 set using LOKI_CA_CERT_PATH env var. 428 --tls-skip-verify Server certificate TLS skip verify. 429 --cert="" Path to the client certificate. Can also be set using 430 LOKI_CLIENT_CERT_PATH env var. 431 --key="" Path to the client certificate key. Can also be set 432 using LOKI_CLIENT_KEY_PATH env var. 433 --org-id="" adds X-Scope-OrgID to API requests for representing 434 tenant ID. Useful for requesting tenant data when 435 bypassing an auth gateway. 436 --since=1h Lookback window. 437 --from=FROM Start looking for logs at this absolute time 438 (inclusive) 439 --to=TO Stop looking for logs at this absolute time (exclusive) 440 --analyze-labels Printout a summary of labels including count of label 441 value combinations, useful for debugging high 442 cardinality series 443 444 Args: 445 <matcher> eg '{foo="bar",baz=~".*blip"}' 446 ``` 447 448 ### LogCLI `--stdin` usage 449 450 You can consume log lines from your `stdin` instead of Loki servers. 451 452 Say you have log files in your local, and just want to do run some LogQL queries for that, `--stdin` flag can help. 453 454 **NOTE: Currently it doesn't support any type of metric queries** 455 456 You may have to use `stdin` flag for several reasons 457 1. Quick way to check and validate a LogQL expressions. 458 2. Learn basics of LogQL with just Log files and `LogCLI`tool ( without needing set up Loki servers, Grafana etc.) 459 3. Easy discussion on public forums. Like Q&A, Share the LogQL expressions. 460 461 **NOTES on Usage** 462 1. `--limits` flag doesn't have any meaning when using `--stdin` (use pager like `less` for that) 463 1. Be aware there are no **labels** when using `--stdin` 464 - So stream selector in the query is optional e.g just `|="timeout"|logfmt|level="error"` is same as `{foo="bar"}|="timeout|logfmt|level="error"` 465 466 **Examples** 467 1. Line filter - `cat mylog.log | logcli --stdin query '|="too many open connections"'` 468 2. Label matcher - `echo 'msg="timeout happened" level="warning"' | logcli --stdin query '|logfmt|level="warning"'` 469 3. Different parsers (logfmt, json, pattern, regexp) - `cat mylog.log | logcli --stdin query '|pattern <ip> - - <_> "<method> <uri> <_>" <status> <size> <_> "<agent>" <_>'` 470 4. Line formatters - `cat mylog.log | logcli --stdin query '|logfmt|line_format "{{.query}} {{.duration}}"'`