github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/processors/process_map.md (about) 1 --- 2 title: process_map 3 type: processor 4 status: deprecated 5 --- 6 7 <!-- 8 THIS FILE IS AUTOGENERATED! 9 10 To make changes please edit the contents of: 11 lib/processor/process_map.go 12 --> 13 14 import Tabs from '@theme/Tabs'; 15 import TabItem from '@theme/TabItem'; 16 17 :::warning DEPRECATED 18 This component is deprecated and will be removed in the next major version release. Please consider moving onto [alternative components](#alternatives). 19 ::: 20 21 A processor that extracts and maps fields identified via 22 [dot path](/docs/configuration/field_paths) from the original payload into a new 23 object, applies a list of processors to the newly constructed object, and 24 finally maps the result back into the original payload. 25 26 27 <Tabs defaultValue="common" values={[ 28 { label: 'Common', value: 'common', }, 29 { label: 'Advanced', value: 'advanced', }, 30 ]}> 31 32 <TabItem value="common"> 33 34 ```yaml 35 # Common config fields, showing default values 36 label: "" 37 process_map: 38 conditions: [] 39 premap: {} 40 premap_optional: {} 41 processors: [] 42 postmap: {} 43 postmap_optional: {} 44 ``` 45 46 </TabItem> 47 <TabItem value="advanced"> 48 49 ```yaml 50 # All config fields, showing default values 51 label: "" 52 process_map: 53 conditions: [] 54 premap: {} 55 premap_optional: {} 56 processors: [] 57 postmap: {} 58 postmap_optional: {} 59 parts: [] 60 ``` 61 62 </TabItem> 63 </Tabs> 64 65 ## Alternatives 66 67 All functionality of this processor has been superseded by the 68 [branch](/docs/components/processors/branch) processor. 69 70 This processor is useful for performing processors on subsections of a payload. 71 For example, you could extract sections of a JSON object in order to construct 72 a reduced request object for an [`http`](/docs/components/processors/http) 73 processor, then map the result back into a field within the original object. 74 75 The order of stages of this processor are as follows: 76 77 - [Conditions](#conditions) are tested (if specified) against each message, 78 messages that do not pass will not be processed. 79 - Messages that are flagged for processing are mapped according to the 80 [premap](#premap) fields, creating a new object. If the premap stage fails 81 (targets are not found) the message will not be processed. 82 - Messages that are mapped are processed as a batch. 83 - After all child processors are applied to the mapped messages they are mapped 84 back into the original messages they originated from following the 85 [postmap](#postmap) fields. If the postmap stage fails the mapping is skipped 86 and the message payload remains as it started. 87 88 If the premap is empty then the full payload is sent to the processors, if the 89 postmap is empty then the processed result replaces the original contents 90 entirely. 91 92 ### Batch Ordering 93 94 This processor supports batched messages, but the list of processors to apply 95 must NOT change the ordering (or count) of the messages (do not use a 96 `group_by` processor, for example). 97 98 ### Error Handling 99 100 When premap, processing or postmap stages fail the underlying message will 101 remain unchanged, the errors are logged, and the message is flagged as having 102 failed, allowing you to use 103 [standard processor error handling patterns](/docs/configuration/error_handling) 104 for recovery. 105 106 ## Fields 107 108 ### `conditions` 109 110 A list of [conditions](/docs/components/conditions/about) to test against messages. If any condition fails then the message will not be mapped and processed. 111 112 113 Type: `array` 114 Default: `[]` 115 116 ```yaml 117 # Examples 118 119 conditions: 120 - bloblang: document.urls.length() > 0 121 ``` 122 123 ### `premap` 124 125 A map of source to destination [paths](/docs/configuration/field_paths) used to create a new object from the original. An empty (or dot `.`) path indicates the root of the object. If a map source is not found then the message will not be processed, for optional sources use the field [`premap_optional`](#premap_optional). 126 127 128 Type: `object` 129 Default: `{}` 130 131 ```yaml 132 # Examples 133 134 premap: 135 .: field.from.document 136 137 premap: 138 bar.baz: root.extra.baz 139 foo: root.body.foo 140 ``` 141 142 ### `premap_optional` 143 144 A map of optional source to destination [paths](/docs/configuration/field_paths) used to create a new object from the original. 145 146 147 Type: `object` 148 Default: `{}` 149 150 ### `processors` 151 152 A list of processors to apply to mapped payloads. 153 154 155 Type: `array` 156 Default: `[]` 157 158 ### `postmap` 159 160 A map of destination to source [paths](/docs/configuration/field_paths) used to map results from processing back into the original payload. An empty (or dot `.`) path indicates the root of the object. If a source is not found then the mapping is abandoned, for optional sources use the [`postmap_optional`](#postmap_optional) field. 161 162 163 Type: `object` 164 Default: `{}` 165 166 ```yaml 167 # Examples 168 169 postmap: 170 results.foo: . 171 ``` 172 173 ### `postmap_optional` 174 175 A map of optional destination to source [paths](/docs/configuration/field_paths) used to map results from processing back into the original payload. 176 177 178 Type: `object` 179 Default: `{}` 180 181 ### `parts` 182 183 An optional array of message indexes of a batch that the processor should apply to. 184 If left empty all messages are processed. This field is only applicable when 185 batching messages [at the input level](/docs/configuration/batching). 186 187 Indexes can be negative, and if so the part will be selected from the end 188 counting backwards starting from -1. 189 190 191 Type: `array` 192 Default: `[]` 193 194 ## Examples 195 196 Given a message payload of: 197 198 ```json 199 { 200 "doc": { 201 "id": "foo", 202 "title": "foo bar baz", 203 "description": "here's a thing", 204 "content": "this is a body" 205 } 206 } 207 ``` 208 209 We might wish to perform language detection on the `doc.content` field 210 by sending it to a hypothetical HTTP service. We do not wish to overwrite the 211 original document with the result, and instead want to place it within the path 212 `doc.language`, and so this is a good use case for `process_map`: 213 214 ```yaml 215 pipeline: 216 processors: 217 - process_map: 218 premap: 219 content: doc.content 220 processors: 221 - http: 222 url: http://localhost:1234 223 postmap: 224 doc.language: . 225 ``` 226 227 With the above config we would send our target HTTP service the payload 228 `{"content":"this is a body"}`, and whatever the service returns 229 will get mapped into our original document: 230 231 ```json 232 { 233 "doc": { 234 "id": "foo", 235 "title": "foo bar baz", 236 "description": "here's a thing", 237 "content": "this is a body", 238 "language": { 239 "code": "en", 240 "certainty": 0.2 241 } 242 } 243 } 244 ``` 245