github.com/Jeffail/benthos/v3@v3.65.0/website/docs/configuration/interpolation.md (about) 1 --- 2 title: Interpolation 3 --- 4 5 Benthos allows you to dynamically set config fields with environment variables anywhere within a config file using the syntax `${<variable-name>}` (or `${<variable-name>:<default-value>}` in order to specify a default value). This is useful for setting environment specific fields such as addresses: 6 7 ```yaml 8 input: 9 kafka: 10 addresses: [ "${BROKERS}" ] 11 consumer_group: benthos_bridge_consumer 12 topics: [ "haha_business" ] 13 ``` 14 15 ```sh 16 BROKERS="foo:9092,bar:9092" benthos -c ./config.yaml 17 ``` 18 19 If a literal string is required that matches this pattern (`${foo}`) you can escape it with double brackets. For example, the string `${{foo}}` is read as the literal `${foo}`. 20 21 ## Bloblang Queries 22 23 Some Benthos fields also support [Bloblang][bloblang] function interpolations, which are much more powerful expressions that allow you to query the contents of messages and perform arithmetic. The syntax of a function interpolation is `${!<bloblang expression>}`, where the contents are a bloblang query (the right-hand-side of a bloblang map) including a range of [functions][bloblang_functions]. For example, with the following config: 24 25 ```yaml 26 output: 27 kafka: 28 addresses: [ "TODO:6379" ] 29 topic: 'dope-${! json("topic") }' 30 ``` 31 32 A message with the contents `{"topic":"foo","message":"hello world"}` would be routed to the Kafka topic `dope-foo`. 33 34 If a literal string is required that matches this pattern (`${!foo}`) then, similar to environment variables, you can escape it with double brackets. For example, the string `${{!foo}}` would be read as the literal `${!foo}`. 35 36 Bloblang supports arithmetic, boolean operators, coalesce and mapping expressions. For more in-depth details about the language [check out the docs][bloblang]. 37 38 ## Examples 39 40 ### Reference Metadata 41 42 A common usecase for interpolated functions is dynamic routing at the output level using metadata: 43 44 ```yaml 45 output: 46 kafka: 47 addresses: [ TODO ] 48 topic: ${! meta("output_topic") } 49 key: ${! meta("key") } 50 ``` 51 52 ### Coalesce and Mapping 53 54 Bloblang supports coalesce and mapping, which makes it easy to extract values from slightly varying data structures: 55 56 ```yaml 57 pipeline: 58 processors: 59 - cache: 60 resource: foocache 61 operator: set 62 key: '${! json().message.(foo | bar).id }' 63 value: '${! content() }' 64 ``` 65 66 Here's a map of inputs to resulting values: 67 68 ``` 69 {"foo":{"a":{"baz":"from_a"},"c":{"baz":"from_c"}}} -> from_a 70 {"foo":{"b":{"baz":"from_b"},"c":{"baz":"from_c"}}} -> from_b 71 {"foo":{"b":null,"c":{"baz":"from_c"}}} -> from_c 72 ``` 73 74 ### Delayed Processing 75 76 We have a stream of JSON documents each with a unix timestamp field `doc.received_at` which is set when our platform receives it. We wish to only process messages an hour _after_ they were received. We can achieve this by running the `sleep` processor using an interpolation function to calculate the seconds needed to wait for: 77 78 ```yaml 79 pipeline: 80 processors: 81 - sleep: 82 duration: '${! 3600 - ( timestamp_unix() - json("doc.created_at").number() ) }s' 83 ``` 84 85 If the calculated result is less than or equal to zero the processor does not sleep at all. If the value of `doc.created_at` is a string then our method `.number()` will attempt to parse it into a number. 86 87 [env_var_config]: https://github.com/Jeffail/benthos/blob/master/config/env/default.yaml 88 [error_handling]: /docs/configuration/error_handling 89 [field_paths]: /docs/configuration/field_paths 90 [meta_proc]: /docs/components/processors/metadata 91 [bloblang]: /docs/guides/bloblang/about 92 [bloblang_functions]: /docs/guides/bloblang/about#functions