github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/aws_dynamodb.md (about)

     1  ---
     2  title: aws_dynamodb
     3  type: output
     4  status: stable
     5  categories: ["Services","AWS"]
     6  ---
     7  
     8  <!--
     9       THIS FILE IS AUTOGENERATED!
    10  
    11       To make changes please edit the contents of:
    12       lib/output/aws_dynamodb.go
    13  -->
    14  
    15  import Tabs from '@theme/Tabs';
    16  import TabItem from '@theme/TabItem';
    17  
    18  
    19  Inserts items into a DynamoDB table.
    20  
    21  Introduced in version 3.36.0.
    22  
    23  
    24  <Tabs defaultValue="common" values={[
    25    { label: 'Common', value: 'common', },
    26    { label: 'Advanced', value: 'advanced', },
    27  ]}>
    28  
    29  <TabItem value="common">
    30  
    31  ```yaml
    32  # Common config fields, showing default values
    33  output:
    34    label: ""
    35    aws_dynamodb:
    36      table: ""
    37      string_columns: {}
    38      json_map_columns: {}
    39      max_in_flight: 1
    40      batching:
    41        count: 0
    42        byte_size: 0
    43        period: ""
    44        check: ""
    45      region: eu-west-1
    46  ```
    47  
    48  </TabItem>
    49  <TabItem value="advanced">
    50  
    51  ```yaml
    52  # All config fields, showing default values
    53  output:
    54    label: ""
    55    aws_dynamodb:
    56      table: ""
    57      string_columns: {}
    58      json_map_columns: {}
    59      ttl: ""
    60      ttl_key: ""
    61      max_in_flight: 1
    62      batching:
    63        count: 0
    64        byte_size: 0
    65        period: ""
    66        check: ""
    67        processors: []
    68      region: eu-west-1
    69      endpoint: ""
    70      credentials:
    71        profile: ""
    72        id: ""
    73        secret: ""
    74        token: ""
    75        role: ""
    76        role_external_id: ""
    77      max_retries: 3
    78      backoff:
    79        initial_interval: 1s
    80        max_interval: 5s
    81        max_elapsed_time: 30s
    82  ```
    83  
    84  </TabItem>
    85  </Tabs>
    86  
    87  The field `string_columns` is a map of column names to string values,
    88  where the values are
    89  [function interpolated](/docs/configuration/interpolation#bloblang-queries) per message of a
    90  batch. This allows you to populate string columns of an item by extracting
    91  fields within the document payload or metadata like follows:
    92  
    93  ```yaml
    94  string_columns:
    95    id: ${!json("id")}
    96    title: ${!json("body.title")}
    97    topic: ${!meta("kafka_topic")}
    98    full_content: ${!content()}
    99  ```
   100  
   101  The field `json_map_columns` is a map of column names to json paths,
   102  where the [dot path](/docs/configuration/field_paths) is extracted from each document and
   103  converted into a map value. Both an empty path and the path `.` are
   104  interpreted as the root of the document. This allows you to populate map columns
   105  of an item like follows:
   106  
   107  ```yml
   108  json_map_columns:
   109    user: path.to.user
   110    whole_document: .
   111  ```
   112  
   113  A column name can be empty:
   114  
   115  ```yml
   116  json_map_columns:
   117    "": .
   118  ```
   119  
   120  In which case the top level document fields will be written at the root of the
   121  item, potentially overwriting previously defined column values. If a path is not
   122  found within a document the column will not be populated.
   123  
   124  ### Credentials
   125  
   126  By default Benthos will use a shared credentials file when connecting to AWS
   127  services. It's also possible to set them explicitly at the component level,
   128  allowing you to transfer data across accounts. You can find out more
   129  [in this document](/docs/guides/cloud/aws).
   130  
   131  ## Performance
   132  
   133  This output benefits from sending multiple messages in flight in parallel for
   134  improved performance. You can tune the max number of in flight messages with the
   135  field `max_in_flight`.
   136  
   137  This output benefits from sending messages as a batch for improved performance.
   138  Batches can be formed at both the input and output level. You can find out more
   139  [in this doc](/docs/configuration/batching).
   140  
   141  ## Fields
   142  
   143  ### `table`
   144  
   145  The table to store messages in.
   146  
   147  
   148  Type: `string`  
   149  Default: `""`  
   150  
   151  ### `string_columns`
   152  
   153  A map of column keys to string values to store.
   154  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   155  
   156  
   157  Type: `object`  
   158  Default: `{}`  
   159  
   160  ```yaml
   161  # Examples
   162  
   163  string_columns:
   164    full_content: ${!content()}
   165    id: ${!json("id")}
   166    title: ${!json("body.title")}
   167    topic: ${!meta("kafka_topic")}
   168  ```
   169  
   170  ### `json_map_columns`
   171  
   172  A map of column keys to [field paths](/docs/configuration/field_paths) pointing to value data within messages.
   173  
   174  
   175  Type: `object`  
   176  Default: `{}`  
   177  
   178  ```yaml
   179  # Examples
   180  
   181  json_map_columns:
   182    user: path.to.user
   183    whole_document: .
   184  
   185  json_map_columns:
   186    "": .
   187  ```
   188  
   189  ### `ttl`
   190  
   191  An optional TTL to set for items, calculated from the moment the message is sent.
   192  
   193  
   194  Type: `string`  
   195  Default: `""`  
   196  
   197  ### `ttl_key`
   198  
   199  The column key to place the TTL value within.
   200  
   201  
   202  Type: `string`  
   203  Default: `""`  
   204  
   205  ### `max_in_flight`
   206  
   207  The maximum number of messages to have in flight at a given time. Increase this to improve throughput.
   208  
   209  
   210  Type: `int`  
   211  Default: `1`  
   212  
   213  ### `batching`
   214  
   215  Allows you to configure a [batching policy](/docs/configuration/batching).
   216  
   217  
   218  Type: `object`  
   219  
   220  ```yaml
   221  # Examples
   222  
   223  batching:
   224    byte_size: 5000
   225    count: 0
   226    period: 1s
   227  
   228  batching:
   229    count: 10
   230    period: 1s
   231  
   232  batching:
   233    check: this.contains("END BATCH")
   234    count: 0
   235    period: 1m
   236  ```
   237  
   238  ### `batching.count`
   239  
   240  A number of messages at which the batch should be flushed. If `0` disables count based batching.
   241  
   242  
   243  Type: `int`  
   244  Default: `0`  
   245  
   246  ### `batching.byte_size`
   247  
   248  An amount of bytes at which the batch should be flushed. If `0` disables size based batching.
   249  
   250  
   251  Type: `int`  
   252  Default: `0`  
   253  
   254  ### `batching.period`
   255  
   256  A period in which an incomplete batch should be flushed regardless of its size.
   257  
   258  
   259  Type: `string`  
   260  Default: `""`  
   261  
   262  ```yaml
   263  # Examples
   264  
   265  period: 1s
   266  
   267  period: 1m
   268  
   269  period: 500ms
   270  ```
   271  
   272  ### `batching.check`
   273  
   274  A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch.
   275  
   276  
   277  Type: `string`  
   278  Default: `""`  
   279  
   280  ```yaml
   281  # Examples
   282  
   283  check: this.type == "end_of_transaction"
   284  ```
   285  
   286  ### `batching.processors`
   287  
   288  A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op.
   289  
   290  
   291  Type: `array`  
   292  Default: `[]`  
   293  
   294  ```yaml
   295  # Examples
   296  
   297  processors:
   298    - archive:
   299        format: lines
   300  
   301  processors:
   302    - archive:
   303        format: json_array
   304  
   305  processors:
   306    - merge_json: {}
   307  ```
   308  
   309  ### `region`
   310  
   311  The AWS region to target.
   312  
   313  
   314  Type: `string`  
   315  Default: `"eu-west-1"`  
   316  
   317  ### `endpoint`
   318  
   319  Allows you to specify a custom endpoint for the AWS API.
   320  
   321  
   322  Type: `string`  
   323  Default: `""`  
   324  
   325  ### `credentials`
   326  
   327  Optional manual configuration of AWS credentials to use. More information can be found [in this document](/docs/guides/cloud/aws).
   328  
   329  
   330  Type: `object`  
   331  
   332  ### `credentials.profile`
   333  
   334  A profile from `~/.aws/credentials` to use.
   335  
   336  
   337  Type: `string`  
   338  Default: `""`  
   339  
   340  ### `credentials.id`
   341  
   342  The ID of credentials to use.
   343  
   344  
   345  Type: `string`  
   346  Default: `""`  
   347  
   348  ### `credentials.secret`
   349  
   350  The secret for the credentials being used.
   351  
   352  
   353  Type: `string`  
   354  Default: `""`  
   355  
   356  ### `credentials.token`
   357  
   358  The token for the credentials being used, required when using short term credentials.
   359  
   360  
   361  Type: `string`  
   362  Default: `""`  
   363  
   364  ### `credentials.role`
   365  
   366  A role ARN to assume.
   367  
   368  
   369  Type: `string`  
   370  Default: `""`  
   371  
   372  ### `credentials.role_external_id`
   373  
   374  An external ID to provide when assuming a role.
   375  
   376  
   377  Type: `string`  
   378  Default: `""`  
   379  
   380  ### `max_retries`
   381  
   382  The maximum number of retries before giving up on the request. If set to zero there is no discrete limit.
   383  
   384  
   385  Type: `int`  
   386  Default: `3`  
   387  
   388  ### `backoff`
   389  
   390  Control time intervals between retry attempts.
   391  
   392  
   393  Type: `object`  
   394  
   395  ### `backoff.initial_interval`
   396  
   397  The initial period to wait between retry attempts.
   398  
   399  
   400  Type: `string`  
   401  Default: `"1s"`  
   402  
   403  ### `backoff.max_interval`
   404  
   405  The maximum period to wait between retry attempts.
   406  
   407  
   408  Type: `string`  
   409  Default: `"5s"`  
   410  
   411  ### `backoff.max_elapsed_time`
   412  
   413  The maximum period to wait before retry attempts are abandoned. If zero then no limit is used.
   414  
   415  
   416  Type: `string`  
   417  Default: `"30s"`  
   418  
   419