github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/processors/jq.md (about) 1 --- 2 title: jq 3 type: processor 4 status: stable 5 categories: ["Mapping"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/processor/jq.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 19 Transforms and filters messages using jq queries. 20 21 22 <Tabs defaultValue="common" values={[ 23 { label: 'Common', value: 'common', }, 24 { label: 'Advanced', value: 'advanced', }, 25 ]}> 26 27 <TabItem value="common"> 28 29 ```yaml 30 # Common config fields, showing default values 31 label: "" 32 jq: 33 query: . 34 ``` 35 36 </TabItem> 37 <TabItem value="advanced"> 38 39 ```yaml 40 # All config fields, showing default values 41 label: "" 42 jq: 43 query: . 44 raw: false 45 output_raw: false 46 ``` 47 48 </TabItem> 49 </Tabs> 50 51 :::note Try out Bloblang 52 For better performance and improved capabilities try out native Benthos mapping with the [bloblang processor](/docs/components/processors/bloblang). 53 ::: 54 55 The provided query is executed on each message, targeting either the contents 56 as a structured JSON value or as a raw string using the field `raw`, 57 and the message is replaced with the query result. 58 59 Message metadata is also accessible within the query from the variable 60 `$metadata`. 61 62 This processor uses the [gojq library][gojq], and therefore does not require 63 jq to be installed as a dependency. However, this also means there are some 64 differences in how these queries are executed versus the jq cli which you can 65 [read about here][gojq-difference]. 66 67 If the query does not emit any value then the message is filtered, if the query 68 returns multiple values then the resulting message will be an array containing 69 all values. 70 71 The full query syntax is described in [jq's documentation][jq-docs]. 72 73 ## Error Handling 74 75 Queries can fail, in which case the message remains unchanged, errors are 76 logged, and the message is flagged as having failed, allowing you to use 77 [standard processor error handling patterns](/docs/configuration/error_handling). 78 79 ## Fields 80 81 ### `query` 82 83 The jq query to filter and transform messages with. 84 85 86 Type: `string` 87 Default: `"."` 88 89 ### `raw` 90 91 Whether to process the input as a raw string instead of as JSON. 92 93 94 Type: `bool` 95 Default: `false` 96 97 ### `output_raw` 98 99 Whether to output raw text (unquoted) instead of JSON strings when the emitted values are string types. 100 101 102 Type: `bool` 103 Default: `false` 104 105 ## Examples 106 107 <Tabs defaultValue="Mapping" values={[ 108 { label: 'Mapping', value: 'Mapping', }, 109 ]}> 110 111 <TabItem value="Mapping"> 112 113 114 When receiving JSON documents of the form: 115 116 ```json 117 { 118 "locations": [ 119 {"name": "Seattle", "state": "WA"}, 120 {"name": "New York", "state": "NY"}, 121 {"name": "Bellevue", "state": "WA"}, 122 {"name": "Olympia", "state": "WA"} 123 ] 124 } 125 ``` 126 127 We could collapse the location names from the state of Washington into a field `Cities`: 128 129 ```json 130 {"Cities": "Bellevue, Olympia, Seattle"} 131 ``` 132 133 With the following config: 134 135 ```yaml 136 pipeline: 137 processors: 138 - jq: 139 query: '{Cities: .locations | map(select(.state == "WA").name) | sort | join(", ") }' 140 ``` 141 142 </TabItem> 143 </Tabs> 144 145 [gojq]: https://github.com/itchyny/gojq 146 [gojq-difference]: https://github.com/itchyny/gojq#difference-to-jq 147 [jq-docs]: https://stedolan.github.io/jq/manual/ 148