github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/mongodb.md (about) 1 --- 2 title: mongodb 3 type: output 4 status: experimental 5 categories: ["Services"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/output/mongodb.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 :::caution EXPERIMENTAL 19 This component is experimental and therefore subject to change or removal outside of major version releases. 20 ::: 21 Inserts items into a MongoDB collection. 22 23 Introduced in version 3.43.0. 24 25 26 <Tabs defaultValue="common" values={[ 27 { label: 'Common', value: 'common', }, 28 { label: 'Advanced', value: 'advanced', }, 29 ]}> 30 31 <TabItem value="common"> 32 33 ```yaml 34 # Common config fields, showing default values 35 output: 36 label: "" 37 mongodb: 38 url: "" 39 database: "" 40 username: "" 41 password: "" 42 operation: update-one 43 collection: "" 44 write_concern: 45 w: "" 46 j: false 47 w_timeout: "" 48 document_map: "" 49 filter_map: "" 50 hint_map: "" 51 upsert: false 52 max_in_flight: 1 53 batching: 54 count: 0 55 byte_size: 0 56 period: "" 57 check: "" 58 ``` 59 60 </TabItem> 61 <TabItem value="advanced"> 62 63 ```yaml 64 # All config fields, showing default values 65 output: 66 label: "" 67 mongodb: 68 url: "" 69 database: "" 70 username: "" 71 password: "" 72 operation: update-one 73 collection: "" 74 write_concern: 75 w: "" 76 j: false 77 w_timeout: "" 78 document_map: "" 79 filter_map: "" 80 hint_map: "" 81 upsert: false 82 max_in_flight: 1 83 batching: 84 count: 0 85 byte_size: 0 86 period: "" 87 check: "" 88 processors: [] 89 max_retries: 3 90 backoff: 91 initial_interval: 1s 92 max_interval: 5s 93 max_elapsed_time: 30s 94 ``` 95 96 </TabItem> 97 </Tabs> 98 99 100 ## Performance 101 102 This output benefits from sending multiple messages in flight in parallel for 103 improved performance. You can tune the max number of in flight messages with the 104 field `max_in_flight`. 105 106 This output benefits from sending messages as a batch for improved performance. 107 Batches can be formed at both the input and output level. You can find out more 108 [in this doc](/docs/configuration/batching). 109 110 ## Fields 111 112 ### `url` 113 114 The URL of the target MongoDB DB. 115 116 117 Type: `string` 118 Default: `""` 119 120 ```yaml 121 # Examples 122 123 url: mongodb://localhost:27017 124 ``` 125 126 ### `database` 127 128 The name of the target MongoDB DB. 129 130 131 Type: `string` 132 Default: `""` 133 134 ### `username` 135 136 The username to connect to the database. 137 138 139 Type: `string` 140 Default: `""` 141 142 ### `password` 143 144 The password to connect to the database. 145 146 147 Type: `string` 148 Default: `""` 149 150 ### `operation` 151 152 The mongodb operation to perform. 153 154 155 Type: `string` 156 Default: `"update-one"` 157 Options: `insert-one`, `delete-one`, `delete-many`, `replace-one`, `update-one`. 158 159 ### `collection` 160 161 The name of the target collection in the MongoDB DB. 162 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 163 164 165 Type: `string` 166 Default: `""` 167 168 ### `write_concern` 169 170 The write concern settings for the mongo connection. 171 172 173 Type: `object` 174 175 ### `write_concern.w` 176 177 W requests acknowledgement that write operations propagate to the specified number of mongodb instances. 178 179 180 Type: `string` 181 Default: `""` 182 183 ### `write_concern.j` 184 185 J requests acknowledgement from MongoDB that write operations are written to the journal. 186 187 188 Type: `bool` 189 Default: `false` 190 191 ### `write_concern.w_timeout` 192 193 The write concern timeout. 194 195 196 Type: `string` 197 Default: `""` 198 199 ### `document_map` 200 201 A bloblang map representing the records in the mongo db. Used to generate the document for mongodb by mapping the fields in the message to the mongodb fields. The document map is required for the operations insert-one, replace-one and update-one. 202 203 204 Type: `string` 205 Default: `""` 206 207 ```yaml 208 # Examples 209 210 document_map: |- 211 root.a = this.foo 212 root.b = this.bar 213 ``` 214 215 ### `filter_map` 216 217 A bloblang map representing the filter for the mongo db command. The filter map is required for all operations except insert-one. It is used to find the document(s) for the operation. For example in a delete-one case, the filter map should have the fields required to locate the document to delete. 218 219 220 Type: `string` 221 Default: `""` 222 223 ```yaml 224 # Examples 225 226 filter_map: |- 227 root.a = this.foo 228 root.b = this.bar 229 ``` 230 231 ### `hint_map` 232 233 A bloblang map representing the hint for the mongo db command. This map is optional and is used with all operations except insert-one. It is used to improve performance of finding the documents in the mongodb. 234 235 236 Type: `string` 237 Default: `""` 238 239 ```yaml 240 # Examples 241 242 hint_map: |- 243 root.a = this.foo 244 root.b = this.bar 245 ``` 246 247 ### `upsert` 248 249 The upsert setting is optional and only applies for update-one and replace-one operations. If the filter specified in filter_map matches,the document is updated or replaced accordingly, otherwise it is created. 250 251 252 Type: `bool` 253 Default: `false` 254 Requires version 3.60.0 or newer 255 256 ### `max_in_flight` 257 258 The maximum number of messages to have in flight at a given time. Increase this to improve throughput. 259 260 261 Type: `int` 262 Default: `1` 263 264 ### `batching` 265 266 Allows you to configure a [batching policy](/docs/configuration/batching). 267 268 269 Type: `object` 270 271 ```yaml 272 # Examples 273 274 batching: 275 byte_size: 5000 276 count: 0 277 period: 1s 278 279 batching: 280 count: 10 281 period: 1s 282 283 batching: 284 check: this.contains("END BATCH") 285 count: 0 286 period: 1m 287 ``` 288 289 ### `batching.count` 290 291 A number of messages at which the batch should be flushed. If `0` disables count based batching. 292 293 294 Type: `int` 295 Default: `0` 296 297 ### `batching.byte_size` 298 299 An amount of bytes at which the batch should be flushed. If `0` disables size based batching. 300 301 302 Type: `int` 303 Default: `0` 304 305 ### `batching.period` 306 307 A period in which an incomplete batch should be flushed regardless of its size. 308 309 310 Type: `string` 311 Default: `""` 312 313 ```yaml 314 # Examples 315 316 period: 1s 317 318 period: 1m 319 320 period: 500ms 321 ``` 322 323 ### `batching.check` 324 325 A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch. 326 327 328 Type: `string` 329 Default: `""` 330 331 ```yaml 332 # Examples 333 334 check: this.type == "end_of_transaction" 335 ``` 336 337 ### `batching.processors` 338 339 A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op. 340 341 342 Type: `array` 343 Default: `[]` 344 345 ```yaml 346 # Examples 347 348 processors: 349 - archive: 350 format: lines 351 352 processors: 353 - archive: 354 format: json_array 355 356 processors: 357 - merge_json: {} 358 ``` 359 360 ### `max_retries` 361 362 The maximum number of retries before giving up on the request. If set to zero there is no discrete limit. 363 364 365 Type: `int` 366 Default: `3` 367 368 ### `backoff` 369 370 Control time intervals between retry attempts. 371 372 373 Type: `object` 374 375 ### `backoff.initial_interval` 376 377 The initial period to wait between retry attempts. 378 379 380 Type: `string` 381 Default: `"1s"` 382 383 ### `backoff.max_interval` 384 385 The maximum period to wait between retry attempts. 386 387 388 Type: `string` 389 Default: `"5s"` 390 391 ### `backoff.max_elapsed_time` 392 393 The maximum period to wait before retry attempts are abandoned. If zero then no limit is used. 394 395 396 Type: `string` 397 Default: `"30s"` 398 399