github.com/yankunsam/loki/v2@v2.6.3-0.20220817130409-389df5235c27/docs/sources/tools/logcli.md (about)

     1  ---
     2  title: LogCLI
     3  weight: 20
     4  aliases:
     5      - /docs/loki/latest/getting-started/logcli/
     6  ---
     7  # LogCLI, Grafana Loki's command-line interface
     8  
     9  LogCLI is the command-line interface to Grafana Loki.
    10  It facilitates running [LogQL]({{< relref "../logql/_index.md" >}})
    11  queries against a Loki instance.
    12  
    13  ## Installation
    14  
    15  ### Binary (Recommended)
    16  
    17  Download the `logcli` binary from the
    18  [Loki releases page](https://github.com/grafana/loki/releases).
    19  
    20  ### Build LogCLI from source
    21  
    22  Clone the Loki repository and build `logcli` from source:
    23  
    24  ```bash
    25  git clone https://github.com/grafana/loki.git
    26  cd loki
    27  make logcli
    28  ```
    29  
    30  Optionally, move the binary into a directory that is part of your `$PATH`.
    31  
    32  ```bash
    33  cp cmd/logcli/logcli /usr/local/bin/logcli
    34  ```
    35  
    36  ## LogCLI usage
    37  
    38  ### Grafana Cloud example
    39  
    40  If you are running on Grafana Cloud, use:
    41  
    42  ```bash
    43  export LOKI_ADDR=https://logs-us-west1.grafana.net
    44  export LOKI_USERNAME=<username>
    45  export LOKI_PASSWORD=<password>
    46  ```
    47  
    48  Otherwise you can point LogCLI to a local instance directly
    49  without needing a username and password:
    50  
    51  ```bash
    52  export LOKI_ADDR=http://localhost:3100
    53  ```
    54  
    55  > Note: If you are running Loki behind a proxy server and you have
    56  > authentication configured, you will also have to pass in LOKI_USERNAME
    57  > and LOKI_PASSWORD, LOKI_BEARER_TOKEN or LOKI_BEARER_TOKEN_FILE accordingly.
    58  
    59  ```bash
    60  $ logcli labels job
    61  https://logs-dev-ops-tools1.grafana.net/api/prom/label/job/values
    62  loki-ops/consul
    63  loki-ops/loki-gw
    64  ...
    65  
    66  $ logcli query '{job="loki-ops/consul"}'
    67  https://logs-dev-ops-tools1.grafana.net/api/prom/query?query=%7Bjob%3D%22loki-ops%2Fconsul%22%7D&limit=30&start=1529928228&end=1529931828&direction=backward&regexp=
    68  Common labels: {job="loki-ops/consul", namespace="loki-ops"}
    69  2018-06-25T12:52:09Z {instance="consul-8576459955-pl75w"} 2018/06/25 12:52:09 [INFO] raft: Snapshot to 475409 complete
    70  2018-06-25T12:52:09Z {instance="consul-8576459955-pl75w"} 2018/06/25 12:52:09 [INFO] raft: Compacting logs from 456973 to 465169
    71  ...
    72  
    73  $ logcli series -q --match='{namespace="loki",container_name="loki"}'
    74  {app="loki", container_name="loki", controller_revision_hash="loki-57c9df47f4", filename="/var/log/pods/loki_loki-0_8ed03ded-bacb-4b13-a6fe-53a445a15887/loki/0.log", instance="loki-0", job="loki/loki", name="loki", namespace="loki", release="loki", statefulset_kubernetes_io_pod_name="loki-0", stream="stderr"}
    75  ```
    76  
    77  ### Batched queries
    78  
    79  LogCLI sends queries to Loki such that query results arrive in batches.
    80  
    81  The `--limit` option for a `logcli query` command caps the quantity of
    82  log lines for a single query.
    83  When not set, `--limit` defaults to 30.
    84  The limit protects the user from overwhelming the system
    85  for cases in which the specified query would have returned a large quantity
    86  of log lines.
    87  The limit also protects the user from unexpectedly large responses.
    88  
    89  The quantity of log line results that arrive in each batch
    90  is set by the `--batch` option in a `logcli query` command.
    91  When not set, `--batch` defaults to 1000.
    92  
    93  Setting a `--limit` value larger than the `--batch` value causes the
    94  requests from LogCLI to Loki to be batched.
    95  Loki has a server-side limit that defaults to 5000 for the maximum quantity
    96  of lines returned for a single query.
    97  The batching of requests allows you to query for a results set that
    98  is larger than the server-side limit,
    99  as long as the `--batch` value is less than the server limit.
   100  
   101  Query metadata is output to `stderr` for each batch.
   102  Set the `--quiet` option on the `logcli query` command line to suppress
   103  the output of the query metadata.
   104  
   105  ### Configuration
   106  
   107  Configuration values are considered in the following order (lowest to highest):
   108  
   109  - Environment variables
   110  - Command-line options
   111  
   112  ### LogCLI command reference
   113  
   114  The output of `logcli help`:
   115  
   116  ```nohighlight
   117  usage: logcli [<flags>] <command> [<args> ...]
   118  
   119  A command-line for loki.
   120  
   121  Flags:
   122        --help             Show context-sensitive help (also try --help-long and
   123                           --help-man).
   124        --version          Show application version.
   125    -q, --quiet            Suppress query metadata
   126        --stats            Show query statistics
   127    -o, --output=default   Specify output mode [default, raw, jsonl]. raw
   128                           suppresses log labels and timestamp.
   129    -z, --timezone=Local   Specify the timezone to use when formatting output
   130                           timestamps [Local, UTC]
   131        --cpuprofile=""    Specify the location for writing a CPU profile.
   132        --memprofile=""    Specify the location for writing a memory profile.
   133        --stdin            Take input logs from stdin
   134        --addr="http://localhost:3100"
   135                           Server address. Can also be set using LOKI_ADDR env
   136                           var.
   137        --username=""      Username for HTTP basic auth. Can also be set using
   138                           LOKI_USERNAME env var.
   139        --password=""      Password for HTTP basic auth. Can also be set using
   140                           LOKI_PASSWORD env var.
   141        --ca-cert=""       Path to the server Certificate Authority. Can also be
   142                           set using LOKI_CA_CERT_PATH env var.
   143        --tls-skip-verify  Server certificate TLS skip verify.
   144        --cert=""          Path to the client certificate. Can also be set using
   145                           LOKI_CLIENT_CERT_PATH env var.
   146        --key=""           Path to the client certificate key. Can also be set
   147                           using LOKI_CLIENT_KEY_PATH env var.
   148        --org-id=""        adds X-Scope-OrgID to API requests for representing
   149                           tenant ID. Useful for requesting tenant data when
   150                           bypassing an auth gateway.
   151  
   152  Commands:
   153    help [<command>...]
   154      Show help.
   155  
   156    query [<flags>] <query>
   157      Run a LogQL query.
   158  
   159      The "query" command is useful for querying for logs. Logs can be returned in
   160      a few output modes:
   161  
   162        raw: log line
   163        default: log timestamp + log labels + log line
   164        jsonl: JSON response from Loki API of log line
   165  
   166      The output of the log can be specified with the "-o" flag, for example, "-o
   167      raw" for the raw output format.
   168  
   169      The "query" command will output extra information about the query and its
   170      results, such as the API URL, set of common labels, and set of excluded
   171      labels. This extra information can be suppressed with the --quiet flag.
   172  
   173      By default we look over the last hour of data; use --since to modify or
   174      provide specific start and end times with --from and --to respectively.
   175  
   176      Notice that when using --from and --to then ensure to use RFC3339Nano time
   177      format, but without timezone at the end. The local timezone will be added
   178      automatically or if using --timezone flag.
   179  
   180      Example:
   181  
   182        logcli query
   183           --timezone=UTC
   184           --from="2021-01-19T10:00:00Z"
   185           --to="2021-01-19T20:00:00Z"
   186           --output=jsonl
   187           'my-query'
   188  
   189      The output is limited to 30 entries by default; use --limit to increase.
   190  
   191      While "query" does support metrics queries, its output contains multiple
   192      data points between the start and end query time. This output is used to
   193      build graphs, similar to what is seen in the Grafana Explore graph view. If
   194      you are querying metrics and just want the most recent data point (like what
   195      is seen in the Grafana Explore table view), then you should use the
   196      "instant-query" command instead.
   197  
   198    instant-query [<flags>] <query>
   199      Run an instant LogQL query.
   200  
   201      The "instant-query" command is useful for evaluating a metric query for a
   202      single point in time. This is equivalent to the Grafana Explore table view;
   203      if you want a metrics query that is used to build a Grafana graph, you
   204      should use the "query" command instead.
   205  
   206      This command does not produce useful output when querying for log lines; you
   207      should always use the "query" command when you are running log queries.
   208  
   209      For more information about log queries and metric queries, refer to the
   210      LogQL documentation:
   211  
   212      https://grafana.com/docs/loki/latest/logql/
   213  
   214    labels [<flags>] [<label>]
   215      Find values for a given label.
   216  
   217    series [<flags>] <matcher>
   218      Run series query.
   219  
   220      The "series" command will take the provided label matcher and return all the
   221      log streams found in the time window.
   222  
   223      It is possible to send an empty label matcher '{}' to return all streams.
   224  
   225      Use the --analyze-labels flag to get a summary of the labels found in all
   226      streams. This is helpful to find high cardinality labels.
   227  ```
   228  
   229  ### LogCLI query command reference
   230  
   231  The output of `logcli help query`:
   232  
   233  ```
   234  usage: logcli query [<flags>] <query>
   235  
   236  Run a LogQL query.
   237  
   238  The "query" command is useful for querying for logs. Logs can be returned in a
   239  few output modes:
   240  
   241    raw: log line
   242    default: log timestamp + log labels + log line
   243    jsonl: JSON response from Loki API of log line
   244  
   245  The output of the log can be specified with the "-o" flag, for example, "-o raw"
   246  for the raw output format.
   247  
   248  The "query" command will output extra information about the query and its
   249  results, such as the API URL, set of common labels, and set of excluded labels.
   250  This extra information can be suppressed with the --quiet flag.
   251  
   252  By default we look over the last hour of data; use --since to modify or provide
   253  specific start and end times with --from and --to respectively.
   254  
   255  Notice that when using --from and --to then ensure to use RFC3339Nano time
   256  format, but without timezone at the end. The local timezone will be added
   257  automatically or if using --timezone flag.
   258  
   259  Example:
   260  
   261    logcli query
   262       --timezone=UTC
   263       --from="2021-01-19T10:00:00Z"
   264       --to="2021-01-19T20:00:00Z"
   265       --output=jsonl
   266       'my-query'
   267  
   268  The output is limited to 30 entries by default; use --limit to increase.
   269  
   270  While "query" does support metrics queries, its output contains multiple data
   271  points between the start and end query time. This output is used to build
   272  graphs, similar to what is seen in the Grafana Explore graph view. If you are
   273  querying metrics and just want the most recent data point (like what is seen in
   274  the Grafana Explore table view), then you should use the "instant-query" command
   275  instead.
   276  
   277  Flags:
   278        --help               Show context-sensitive help (also try --help-long and
   279                             --help-man).
   280        --version            Show application version.
   281    -q, --quiet              Suppress query metadata
   282        --stats              Show query statistics
   283    -o, --output=default     Specify output mode [default, raw, jsonl]. raw
   284                             suppresses log labels and timestamp.
   285    -z, --timezone=Local     Specify the timezone to use when formatting output
   286                             timestamps [Local, UTC]
   287        --cpuprofile=""      Specify the location for writing a CPU profile.
   288        --memprofile=""      Specify the location for writing a memory profile.
   289        --stdin              Take input logs from stdin
   290        --addr="http://localhost:3100"
   291                             Server address. Can also be set using LOKI_ADDR env
   292                             var.
   293        --username=""        Username for HTTP basic auth. Can also be set using
   294                             LOKI_USERNAME env var.
   295        --password=""        Password for HTTP basic auth. Can also be set using
   296                             LOKI_PASSWORD env var.
   297        --ca-cert=""         Path to the server Certificate Authority. Can also be
   298                             set using LOKI_CA_CERT_PATH env var.
   299        --tls-skip-verify    Server certificate TLS skip verify.
   300        --cert=""            Path to the client certificate. Can also be set using
   301                             LOKI_CLIENT_CERT_PATH env var.
   302        --key=""             Path to the client certificate key. Can also be set
   303                             using LOKI_CLIENT_KEY_PATH env var.
   304        --org-id=""          adds X-Scope-OrgID to API requests for representing
   305                             tenant ID. Useful for requesting tenant data when
   306                             bypassing an auth gateway.
   307        --limit=30           Limit on number of entries to print.
   308        --since=1h           Lookback window.
   309        --from=FROM          Start looking for logs at this absolute time
   310                             (inclusive)
   311        --to=TO              Stop looking for logs at this absolute time
   312                             (exclusive)
   313        --step=STEP          Query resolution step width, for metric queries.
   314                             Evaluate the query at the specified step over the
   315                             time range.
   316        --interval=INTERVAL  Query interval, for log queries. Return entries at
   317                             the specified interval, ignoring those between.
   318                             **This parameter is experimental, please see Issue
   319                             1779**
   320        --batch=1000         Query batch size to use until 'limit' is reached
   321        --forward            Scan forwards through logs.
   322        --no-labels          Do not print any labels
   323        --exclude-label=EXCLUDE-LABEL ...
   324                             Exclude labels given the provided key during output.
   325        --include-label=INCLUDE-LABEL ...
   326                             Include labels given the provided key during output.
   327        --labels-length=0    Set a fixed padding to labels
   328        --store-config=""    Execute the current query using a configured storage
   329                             from a given Loki configuration file.
   330        --colored-output     Show output with colored labels
   331    -t, --tail               Tail the logs
   332    -f, --follow             Alias for --tail
   333        --delay-for=0        Delay in tailing by number of seconds to accumulate
   334                             logs for re-ordering
   335  
   336  Args:
   337    <query>  eg '{foo="bar",baz=~".*blip"} |~ ".*error.*"'
   338  ```
   339  
   340  ### LogCLI labels command reference
   341  
   342  The output of `logcli help labels`:
   343  
   344  ```
   345  usage: logcli labels [<flags>] [<label>]
   346  
   347  Find values for a given label.
   348  
   349  Flags:
   350        --help             Show context-sensitive help (also try --help-long and
   351                           --help-man).
   352        --version          Show application version.
   353    -q, --quiet            Suppress query metadata
   354        --stats            Show query statistics
   355    -o, --output=default   Specify output mode [default, raw, jsonl]. raw
   356                           suppresses log labels and timestamp.
   357    -z, --timezone=Local   Specify the timezone to use when formatting output
   358                           timestamps [Local, UTC]
   359        --cpuprofile=""    Specify the location for writing a CPU profile.
   360        --memprofile=""    Specify the location for writing a memory profile.
   361        --stdin            Take input logs from stdin
   362        --addr="http://localhost:3100"
   363                           Server address. Can also be set using LOKI_ADDR env
   364                           var.
   365        --username=""      Username for HTTP basic auth. Can also be set using
   366                           LOKI_USERNAME env var.
   367        --password=""      Password for HTTP basic auth. Can also be set using
   368                           LOKI_PASSWORD env var.
   369        --ca-cert=""       Path to the server Certificate Authority. Can also be
   370                           set using LOKI_CA_CERT_PATH env var.
   371        --tls-skip-verify  Server certificate TLS skip verify.
   372        --cert=""          Path to the client certificate. Can also be set using
   373                           LOKI_CLIENT_CERT_PATH env var.
   374        --key=""           Path to the client certificate key. Can also be set
   375                           using LOKI_CLIENT_KEY_PATH env var.
   376        --org-id=""        adds X-Scope-OrgID to API requests for representing
   377                           tenant ID. Useful for requesting tenant data when
   378                           bypassing an auth gateway.
   379        --since=1h         Lookback window.
   380        --from=FROM        Start looking for labels at this absolute time
   381                           (inclusive)
   382        --to=TO            Stop looking for labels at this absolute time
   383                           (exclusive)
   384  
   385  Args:
   386    [<label>]  The name of the label.
   387  ```
   388  
   389  ### LogCLI series command reference
   390  
   391  The output of `logcli help series`:
   392  
   393  ```
   394  usage: logcli series [<flags>] <matcher>
   395  
   396  Run series query.
   397  
   398  The "series" command will take the provided label matcher and return all the log
   399  streams found in the time window.
   400  
   401  It is possible to send an empty label matcher '{}' to return all streams.
   402  
   403  Use the --analyze-labels flag to get a summary of the labels found in all
   404  streams. This is helpful to find high cardinality labels.
   405  
   406  Flags:
   407        --help             Show context-sensitive help (also try --help-long and
   408                           --help-man).
   409        --version          Show application version.
   410    -q, --quiet            Suppress query metadata
   411        --stats            Show query statistics
   412    -o, --output=default   Specify output mode [default, raw, jsonl]. raw
   413                           suppresses log labels and timestamp.
   414    -z, --timezone=Local   Specify the timezone to use when formatting output
   415                           timestamps [Local, UTC]
   416        --cpuprofile=""    Specify the location for writing a CPU profile.
   417        --memprofile=""    Specify the location for writing a memory profile.
   418        --stdin            Take input logs from stdin
   419        --addr="http://localhost:3100"
   420                           Server address. Can also be set using LOKI_ADDR env
   421                           var.
   422        --username=""      Username for HTTP basic auth. Can also be set using
   423                           LOKI_USERNAME env var.
   424        --password=""      Password for HTTP basic auth. Can also be set using
   425                           LOKI_PASSWORD env var.
   426        --ca-cert=""       Path to the server Certificate Authority. Can also be
   427                           set using LOKI_CA_CERT_PATH env var.
   428        --tls-skip-verify  Server certificate TLS skip verify.
   429        --cert=""          Path to the client certificate. Can also be set using
   430                           LOKI_CLIENT_CERT_PATH env var.
   431        --key=""           Path to the client certificate key. Can also be set
   432                           using LOKI_CLIENT_KEY_PATH env var.
   433        --org-id=""        adds X-Scope-OrgID to API requests for representing
   434                           tenant ID. Useful for requesting tenant data when
   435                           bypassing an auth gateway.
   436        --since=1h         Lookback window.
   437        --from=FROM        Start looking for logs at this absolute time
   438                           (inclusive)
   439        --to=TO            Stop looking for logs at this absolute time (exclusive)
   440        --analyze-labels   Printout a summary of labels including count of label
   441                           value combinations, useful for debugging high
   442                           cardinality series
   443  
   444  Args:
   445    <matcher>  eg '{foo="bar",baz=~".*blip"}'
   446  ```
   447  
   448  ### LogCLI `--stdin` usage
   449  
   450  You can consume log lines from your `stdin` instead of Loki servers.
   451  
   452  Say you have log files in your local, and just want to do run some LogQL queries for that, `--stdin` flag can help.
   453  
   454  **NOTE: Currently it doesn't support any type of metric queries**
   455  
   456  You may have to use `stdin` flag for several reasons
   457  1. Quick way to check and validate a LogQL expressions.
   458  2. Learn basics of LogQL with just Log files and `LogCLI`tool ( without needing set up Loki servers, Grafana etc.)
   459  3. Easy discussion on public forums. Like Q&A, Share the LogQL expressions.
   460  
   461  **NOTES on Usage**
   462  1. `--limits` flag doesn't have any meaning when using `--stdin` (use pager like `less` for that)
   463  1. Be aware there are no **labels** when using `--stdin`
   464     - So stream selector in the query is optional e.g just `|="timeout"|logfmt|level="error"` is same as `{foo="bar"}|="timeout|logfmt|level="error"`
   465  
   466  **Examples**
   467  1. Line filter - `cat mylog.log | logcli --stdin query '|="too many open connections"'`
   468  2. Label matcher - `echo 'msg="timeout happened" level="warning"' | logcli --stdin query '|logfmt|level="warning"'`
   469  3. Different parsers (logfmt, json, pattern, regexp) - `cat mylog.log | logcli --stdin query '|pattern <ip> - - <_> "<method> <uri> <_>" <status> <size> <_> "<agent>" <_>'`
   470  4. Line formatters - `cat mylog.log | logcli --stdin query '|logfmt|line_format "{{.query}} {{.duration}}"'`