github.com/Jeffail/benthos/v3@v3.65.0/website/docs/configuration/interpolation.md (about)

     1  ---
     2  title: Interpolation
     3  ---
     4  
     5  Benthos allows you to dynamically set config fields with environment variables anywhere within a config file using the syntax `${<variable-name>}` (or `${<variable-name>:<default-value>}` in order to specify a default value). This is useful for setting environment specific fields such as addresses:
     6  
     7  ```yaml
     8  input:
     9    kafka:
    10      addresses: [ "${BROKERS}" ]
    11      consumer_group: benthos_bridge_consumer
    12      topics: [ "haha_business" ]
    13  ```
    14  
    15  ```sh
    16  BROKERS="foo:9092,bar:9092" benthos -c ./config.yaml
    17  ```
    18  
    19  If a literal string is required that matches this pattern (`${foo}`) you can escape it with double brackets. For example, the string `${{foo}}` is read as the literal `${foo}`.
    20  
    21  ## Bloblang Queries
    22  
    23  Some Benthos fields also support [Bloblang][bloblang] function interpolations, which are much more powerful expressions that allow you to query the contents of messages and perform arithmetic. The syntax of a function interpolation is `${!<bloblang expression>}`, where the contents are a bloblang query (the right-hand-side of a bloblang map) including a range of [functions][bloblang_functions]. For example, with the following config:
    24  
    25  ```yaml
    26  output:
    27    kafka:
    28      addresses: [ "TODO:6379" ]
    29      topic: 'dope-${! json("topic") }'
    30  ```
    31  
    32  A message with the contents `{"topic":"foo","message":"hello world"}` would be routed to the Kafka topic `dope-foo`.
    33  
    34  If a literal string is required that matches this pattern (`${!foo}`) then, similar to environment variables, you can escape it with double brackets. For example, the string `${{!foo}}` would be read as the literal `${!foo}`.
    35  
    36  Bloblang supports arithmetic, boolean operators, coalesce and mapping expressions. For more in-depth details about the language [check out the docs][bloblang].
    37  
    38  ## Examples
    39  
    40  ### Reference Metadata
    41  
    42  A common usecase for interpolated functions is dynamic routing at the output level using metadata:
    43  
    44  ```yaml
    45  output:
    46    kafka:
    47      addresses: [ TODO ]
    48      topic: ${! meta("output_topic") }
    49      key: ${! meta("key") }
    50  ```
    51  
    52  ### Coalesce and Mapping
    53  
    54  Bloblang supports coalesce and mapping, which makes it easy to extract values from slightly varying data structures:
    55  
    56  ```yaml
    57  pipeline:
    58    processors:
    59      - cache:
    60          resource: foocache
    61          operator: set
    62          key: '${! json().message.(foo | bar).id }'
    63          value: '${! content() }'
    64  ```
    65  
    66  Here's a map of inputs to resulting values:
    67  
    68  ```
    69  {"foo":{"a":{"baz":"from_a"},"c":{"baz":"from_c"}}} -> from_a
    70  {"foo":{"b":{"baz":"from_b"},"c":{"baz":"from_c"}}} -> from_b
    71  {"foo":{"b":null,"c":{"baz":"from_c"}}}             -> from_c
    72  ```
    73  
    74  ### Delayed Processing
    75  
    76  We have a stream of JSON documents each with a unix timestamp field `doc.received_at` which is set when our platform receives it. We wish to only process messages an hour _after_ they were received. We can achieve this by running the `sleep` processor using an interpolation function to calculate the seconds needed to wait for:
    77  
    78  ```yaml
    79  pipeline:
    80    processors:
    81    - sleep:
    82        duration: '${! 3600 - ( timestamp_unix() - json("doc.created_at").number() ) }s'
    83  ```
    84  
    85  If the calculated result is less than or equal to zero the processor does not sleep at all. If the value of `doc.created_at` is a string then our method `.number()` will attempt to parse it into a number.
    86  
    87  [env_var_config]: https://github.com/Jeffail/benthos/blob/master/config/env/default.yaml
    88  [error_handling]: /docs/configuration/error_handling
    89  [field_paths]: /docs/configuration/field_paths
    90  [meta_proc]: /docs/components/processors/metadata
    91  [bloblang]: /docs/guides/bloblang/about
    92  [bloblang_functions]: /docs/guides/bloblang/about#functions