github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/dynamodb.md (about)

     1  ---
     2  title: dynamodb
     3  type: output
     4  status: deprecated
     5  categories: ["Services","AWS"]
     6  ---
     7  
     8  <!--
     9       THIS FILE IS AUTOGENERATED!
    10  
    11       To make changes please edit the contents of:
    12       lib/output/dynamodb.go
    13  -->
    14  
    15  import Tabs from '@theme/Tabs';
    16  import TabItem from '@theme/TabItem';
    17  
    18  :::warning DEPRECATED
    19  This component is deprecated and will be removed in the next major version release. Please consider moving onto [alternative components](#alternatives).
    20  :::
    21  
    22  Inserts items into a DynamoDB table.
    23  
    24  
    25  <Tabs defaultValue="common" values={[
    26    { label: 'Common', value: 'common', },
    27    { label: 'Advanced', value: 'advanced', },
    28  ]}>
    29  
    30  <TabItem value="common">
    31  
    32  ```yaml
    33  # Common config fields, showing default values
    34  output:
    35    label: ""
    36    dynamodb:
    37      table: ""
    38      string_columns: {}
    39      json_map_columns: {}
    40      max_in_flight: 1
    41      batching:
    42        count: 0
    43        byte_size: 0
    44        period: ""
    45        check: ""
    46      region: eu-west-1
    47  ```
    48  
    49  </TabItem>
    50  <TabItem value="advanced">
    51  
    52  ```yaml
    53  # All config fields, showing default values
    54  output:
    55    label: ""
    56    dynamodb:
    57      table: ""
    58      string_columns: {}
    59      json_map_columns: {}
    60      ttl: ""
    61      ttl_key: ""
    62      max_in_flight: 1
    63      batching:
    64        count: 0
    65        byte_size: 0
    66        period: ""
    67        check: ""
    68        processors: []
    69      region: eu-west-1
    70      endpoint: ""
    71      credentials:
    72        profile: ""
    73        id: ""
    74        secret: ""
    75        token: ""
    76        role: ""
    77        role_external_id: ""
    78      max_retries: 3
    79      backoff:
    80        initial_interval: 1s
    81        max_interval: 5s
    82        max_elapsed_time: 30s
    83  ```
    84  
    85  </TabItem>
    86  </Tabs>
    87  
    88  ## Alternatives
    89  
    90  This output has been renamed to [`aws_dynamodb`](/docs/components/outputs/aws_dynamodb).
    91  
    92  The field `string_columns` is a map of column names to string values,
    93  where the values are
    94  [function interpolated](/docs/configuration/interpolation#bloblang-queries) per message of a
    95  batch. This allows you to populate string columns of an item by extracting
    96  fields within the document payload or metadata like follows:
    97  
    98  ```yml
    99  string_columns:
   100    id: ${!json("id")}
   101    title: ${!json("body.title")}
   102    topic: ${!meta("kafka_topic")}
   103    full_content: ${!content()}
   104  ```
   105  
   106  The field `json_map_columns` is a map of column names to json paths,
   107  where the [dot path](/docs/configuration/field_paths) is extracted from each document and
   108  converted into a map value. Both an empty path and the path `.` are
   109  interpreted as the root of the document. This allows you to populate map columns
   110  of an item like follows:
   111  
   112  ```yml
   113  json_map_columns:
   114    user: path.to.user
   115    whole_document: .
   116  ```
   117  
   118  A column name can be empty:
   119  
   120  ```yml
   121  json_map_columns:
   122    "": .
   123  ```
   124  
   125  In which case the top level document fields will be written at the root of the
   126  item, potentially overwriting previously defined column values. If a path is not
   127  found within a document the column will not be populated.
   128  
   129  ### Credentials
   130  
   131  By default Benthos will use a shared credentials file when connecting to AWS
   132  services. It's also possible to set them explicitly at the component level,
   133  allowing you to transfer data across accounts. You can find out more
   134  [in this document](/docs/guides/cloud/aws).
   135  
   136  ## Performance
   137  
   138  This output benefits from sending multiple messages in flight in parallel for
   139  improved performance. You can tune the max number of in flight messages with the
   140  field `max_in_flight`.
   141  
   142  This output benefits from sending messages as a batch for improved performance.
   143  Batches can be formed at both the input and output level. You can find out more
   144  [in this doc](/docs/configuration/batching).
   145  
   146  ## Fields
   147  
   148  ### `table`
   149  
   150  The table to store messages in.
   151  
   152  
   153  Type: `string`  
   154  Default: `""`  
   155  
   156  ### `string_columns`
   157  
   158  A map of column keys to string values to store.
   159  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   160  
   161  
   162  Type: `object`  
   163  Default: `{}`  
   164  
   165  ```yaml
   166  # Examples
   167  
   168  string_columns:
   169    full_content: ${!content()}
   170    id: ${!json("id")}
   171    title: ${!json("body.title")}
   172    topic: ${!meta("kafka_topic")}
   173  ```
   174  
   175  ### `json_map_columns`
   176  
   177  A map of column keys to [field paths](/docs/configuration/field_paths) pointing to value data within messages.
   178  
   179  
   180  Type: `object`  
   181  Default: `{}`  
   182  
   183  ```yaml
   184  # Examples
   185  
   186  json_map_columns:
   187    user: path.to.user
   188    whole_document: .
   189  
   190  json_map_columns:
   191    "": .
   192  ```
   193  
   194  ### `ttl`
   195  
   196  An optional TTL to set for items, calculated from the moment the message is sent.
   197  
   198  
   199  Type: `string`  
   200  Default: `""`  
   201  
   202  ### `ttl_key`
   203  
   204  The column key to place the TTL value within.
   205  
   206  
   207  Type: `string`  
   208  Default: `""`  
   209  
   210  ### `max_in_flight`
   211  
   212  The maximum number of messages to have in flight at a given time. Increase this to improve throughput.
   213  
   214  
   215  Type: `int`  
   216  Default: `1`  
   217  
   218  ### `batching`
   219  
   220  Allows you to configure a [batching policy](/docs/configuration/batching).
   221  
   222  
   223  Type: `object`  
   224  
   225  ```yaml
   226  # Examples
   227  
   228  batching:
   229    byte_size: 5000
   230    count: 0
   231    period: 1s
   232  
   233  batching:
   234    count: 10
   235    period: 1s
   236  
   237  batching:
   238    check: this.contains("END BATCH")
   239    count: 0
   240    period: 1m
   241  ```
   242  
   243  ### `batching.count`
   244  
   245  A number of messages at which the batch should be flushed. If `0` disables count based batching.
   246  
   247  
   248  Type: `int`  
   249  Default: `0`  
   250  
   251  ### `batching.byte_size`
   252  
   253  An amount of bytes at which the batch should be flushed. If `0` disables size based batching.
   254  
   255  
   256  Type: `int`  
   257  Default: `0`  
   258  
   259  ### `batching.period`
   260  
   261  A period in which an incomplete batch should be flushed regardless of its size.
   262  
   263  
   264  Type: `string`  
   265  Default: `""`  
   266  
   267  ```yaml
   268  # Examples
   269  
   270  period: 1s
   271  
   272  period: 1m
   273  
   274  period: 500ms
   275  ```
   276  
   277  ### `batching.check`
   278  
   279  A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch.
   280  
   281  
   282  Type: `string`  
   283  Default: `""`  
   284  
   285  ```yaml
   286  # Examples
   287  
   288  check: this.type == "end_of_transaction"
   289  ```
   290  
   291  ### `batching.processors`
   292  
   293  A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op.
   294  
   295  
   296  Type: `array`  
   297  Default: `[]`  
   298  
   299  ```yaml
   300  # Examples
   301  
   302  processors:
   303    - archive:
   304        format: lines
   305  
   306  processors:
   307    - archive:
   308        format: json_array
   309  
   310  processors:
   311    - merge_json: {}
   312  ```
   313  
   314  ### `region`
   315  
   316  The AWS region to target.
   317  
   318  
   319  Type: `string`  
   320  Default: `"eu-west-1"`  
   321  
   322  ### `endpoint`
   323  
   324  Allows you to specify a custom endpoint for the AWS API.
   325  
   326  
   327  Type: `string`  
   328  Default: `""`  
   329  
   330  ### `credentials`
   331  
   332  Optional manual configuration of AWS credentials to use. More information can be found [in this document](/docs/guides/cloud/aws).
   333  
   334  
   335  Type: `object`  
   336  
   337  ### `credentials.profile`
   338  
   339  A profile from `~/.aws/credentials` to use.
   340  
   341  
   342  Type: `string`  
   343  Default: `""`  
   344  
   345  ### `credentials.id`
   346  
   347  The ID of credentials to use.
   348  
   349  
   350  Type: `string`  
   351  Default: `""`  
   352  
   353  ### `credentials.secret`
   354  
   355  The secret for the credentials being used.
   356  
   357  
   358  Type: `string`  
   359  Default: `""`  
   360  
   361  ### `credentials.token`
   362  
   363  The token for the credentials being used, required when using short term credentials.
   364  
   365  
   366  Type: `string`  
   367  Default: `""`  
   368  
   369  ### `credentials.role`
   370  
   371  A role ARN to assume.
   372  
   373  
   374  Type: `string`  
   375  Default: `""`  
   376  
   377  ### `credentials.role_external_id`
   378  
   379  An external ID to provide when assuming a role.
   380  
   381  
   382  Type: `string`  
   383  Default: `""`  
   384  
   385  ### `max_retries`
   386  
   387  The maximum number of retries before giving up on the request. If set to zero there is no discrete limit.
   388  
   389  
   390  Type: `int`  
   391  Default: `3`  
   392  
   393  ### `backoff`
   394  
   395  Control time intervals between retry attempts.
   396  
   397  
   398  Type: `object`  
   399  
   400  ### `backoff.initial_interval`
   401  
   402  The initial period to wait between retry attempts.
   403  
   404  
   405  Type: `string`  
   406  Default: `"1s"`  
   407  
   408  ### `backoff.max_interval`
   409  
   410  The maximum period to wait between retry attempts.
   411  
   412  
   413  Type: `string`  
   414  Default: `"5s"`  
   415  
   416  ### `backoff.max_elapsed_time`
   417  
   418  The maximum period to wait before retry attempts are abandoned. If zero then no limit is used.
   419  
   420  
   421  Type: `string`  
   422  Default: `"30s"`  
   423  
   424