github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/aws_dynamodb.md (about) 1 --- 2 title: aws_dynamodb 3 type: output 4 status: stable 5 categories: ["Services","AWS"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/output/aws_dynamodb.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 19 Inserts items into a DynamoDB table. 20 21 Introduced in version 3.36.0. 22 23 24 <Tabs defaultValue="common" values={[ 25 { label: 'Common', value: 'common', }, 26 { label: 'Advanced', value: 'advanced', }, 27 ]}> 28 29 <TabItem value="common"> 30 31 ```yaml 32 # Common config fields, showing default values 33 output: 34 label: "" 35 aws_dynamodb: 36 table: "" 37 string_columns: {} 38 json_map_columns: {} 39 max_in_flight: 1 40 batching: 41 count: 0 42 byte_size: 0 43 period: "" 44 check: "" 45 region: eu-west-1 46 ``` 47 48 </TabItem> 49 <TabItem value="advanced"> 50 51 ```yaml 52 # All config fields, showing default values 53 output: 54 label: "" 55 aws_dynamodb: 56 table: "" 57 string_columns: {} 58 json_map_columns: {} 59 ttl: "" 60 ttl_key: "" 61 max_in_flight: 1 62 batching: 63 count: 0 64 byte_size: 0 65 period: "" 66 check: "" 67 processors: [] 68 region: eu-west-1 69 endpoint: "" 70 credentials: 71 profile: "" 72 id: "" 73 secret: "" 74 token: "" 75 role: "" 76 role_external_id: "" 77 max_retries: 3 78 backoff: 79 initial_interval: 1s 80 max_interval: 5s 81 max_elapsed_time: 30s 82 ``` 83 84 </TabItem> 85 </Tabs> 86 87 The field `string_columns` is a map of column names to string values, 88 where the values are 89 [function interpolated](/docs/configuration/interpolation#bloblang-queries) per message of a 90 batch. This allows you to populate string columns of an item by extracting 91 fields within the document payload or metadata like follows: 92 93 ```yaml 94 string_columns: 95 id: ${!json("id")} 96 title: ${!json("body.title")} 97 topic: ${!meta("kafka_topic")} 98 full_content: ${!content()} 99 ``` 100 101 The field `json_map_columns` is a map of column names to json paths, 102 where the [dot path](/docs/configuration/field_paths) is extracted from each document and 103 converted into a map value. Both an empty path and the path `.` are 104 interpreted as the root of the document. This allows you to populate map columns 105 of an item like follows: 106 107 ```yml 108 json_map_columns: 109 user: path.to.user 110 whole_document: . 111 ``` 112 113 A column name can be empty: 114 115 ```yml 116 json_map_columns: 117 "": . 118 ``` 119 120 In which case the top level document fields will be written at the root of the 121 item, potentially overwriting previously defined column values. If a path is not 122 found within a document the column will not be populated. 123 124 ### Credentials 125 126 By default Benthos will use a shared credentials file when connecting to AWS 127 services. It's also possible to set them explicitly at the component level, 128 allowing you to transfer data across accounts. You can find out more 129 [in this document](/docs/guides/cloud/aws). 130 131 ## Performance 132 133 This output benefits from sending multiple messages in flight in parallel for 134 improved performance. You can tune the max number of in flight messages with the 135 field `max_in_flight`. 136 137 This output benefits from sending messages as a batch for improved performance. 138 Batches can be formed at both the input and output level. You can find out more 139 [in this doc](/docs/configuration/batching). 140 141 ## Fields 142 143 ### `table` 144 145 The table to store messages in. 146 147 148 Type: `string` 149 Default: `""` 150 151 ### `string_columns` 152 153 A map of column keys to string values to store. 154 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 155 156 157 Type: `object` 158 Default: `{}` 159 160 ```yaml 161 # Examples 162 163 string_columns: 164 full_content: ${!content()} 165 id: ${!json("id")} 166 title: ${!json("body.title")} 167 topic: ${!meta("kafka_topic")} 168 ``` 169 170 ### `json_map_columns` 171 172 A map of column keys to [field paths](/docs/configuration/field_paths) pointing to value data within messages. 173 174 175 Type: `object` 176 Default: `{}` 177 178 ```yaml 179 # Examples 180 181 json_map_columns: 182 user: path.to.user 183 whole_document: . 184 185 json_map_columns: 186 "": . 187 ``` 188 189 ### `ttl` 190 191 An optional TTL to set for items, calculated from the moment the message is sent. 192 193 194 Type: `string` 195 Default: `""` 196 197 ### `ttl_key` 198 199 The column key to place the TTL value within. 200 201 202 Type: `string` 203 Default: `""` 204 205 ### `max_in_flight` 206 207 The maximum number of messages to have in flight at a given time. Increase this to improve throughput. 208 209 210 Type: `int` 211 Default: `1` 212 213 ### `batching` 214 215 Allows you to configure a [batching policy](/docs/configuration/batching). 216 217 218 Type: `object` 219 220 ```yaml 221 # Examples 222 223 batching: 224 byte_size: 5000 225 count: 0 226 period: 1s 227 228 batching: 229 count: 10 230 period: 1s 231 232 batching: 233 check: this.contains("END BATCH") 234 count: 0 235 period: 1m 236 ``` 237 238 ### `batching.count` 239 240 A number of messages at which the batch should be flushed. If `0` disables count based batching. 241 242 243 Type: `int` 244 Default: `0` 245 246 ### `batching.byte_size` 247 248 An amount of bytes at which the batch should be flushed. If `0` disables size based batching. 249 250 251 Type: `int` 252 Default: `0` 253 254 ### `batching.period` 255 256 A period in which an incomplete batch should be flushed regardless of its size. 257 258 259 Type: `string` 260 Default: `""` 261 262 ```yaml 263 # Examples 264 265 period: 1s 266 267 period: 1m 268 269 period: 500ms 270 ``` 271 272 ### `batching.check` 273 274 A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch. 275 276 277 Type: `string` 278 Default: `""` 279 280 ```yaml 281 # Examples 282 283 check: this.type == "end_of_transaction" 284 ``` 285 286 ### `batching.processors` 287 288 A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op. 289 290 291 Type: `array` 292 Default: `[]` 293 294 ```yaml 295 # Examples 296 297 processors: 298 - archive: 299 format: lines 300 301 processors: 302 - archive: 303 format: json_array 304 305 processors: 306 - merge_json: {} 307 ``` 308 309 ### `region` 310 311 The AWS region to target. 312 313 314 Type: `string` 315 Default: `"eu-west-1"` 316 317 ### `endpoint` 318 319 Allows you to specify a custom endpoint for the AWS API. 320 321 322 Type: `string` 323 Default: `""` 324 325 ### `credentials` 326 327 Optional manual configuration of AWS credentials to use. More information can be found [in this document](/docs/guides/cloud/aws). 328 329 330 Type: `object` 331 332 ### `credentials.profile` 333 334 A profile from `~/.aws/credentials` to use. 335 336 337 Type: `string` 338 Default: `""` 339 340 ### `credentials.id` 341 342 The ID of credentials to use. 343 344 345 Type: `string` 346 Default: `""` 347 348 ### `credentials.secret` 349 350 The secret for the credentials being used. 351 352 353 Type: `string` 354 Default: `""` 355 356 ### `credentials.token` 357 358 The token for the credentials being used, required when using short term credentials. 359 360 361 Type: `string` 362 Default: `""` 363 364 ### `credentials.role` 365 366 A role ARN to assume. 367 368 369 Type: `string` 370 Default: `""` 371 372 ### `credentials.role_external_id` 373 374 An external ID to provide when assuming a role. 375 376 377 Type: `string` 378 Default: `""` 379 380 ### `max_retries` 381 382 The maximum number of retries before giving up on the request. If set to zero there is no discrete limit. 383 384 385 Type: `int` 386 Default: `3` 387 388 ### `backoff` 389 390 Control time intervals between retry attempts. 391 392 393 Type: `object` 394 395 ### `backoff.initial_interval` 396 397 The initial period to wait between retry attempts. 398 399 400 Type: `string` 401 Default: `"1s"` 402 403 ### `backoff.max_interval` 404 405 The maximum period to wait between retry attempts. 406 407 408 Type: `string` 409 Default: `"5s"` 410 411 ### `backoff.max_elapsed_time` 412 413 The maximum period to wait before retry attempts are abandoned. If zero then no limit is used. 414 415 416 Type: `string` 417 Default: `"30s"` 418 419