github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/aws_sqs.md (about)

     1  ---
     2  title: aws_sqs
     3  type: output
     4  status: stable
     5  categories: ["Services","AWS"]
     6  ---
     7  
     8  <!--
     9       THIS FILE IS AUTOGENERATED!
    10  
    11       To make changes please edit the contents of:
    12       lib/output/aws_sqs.go
    13  -->
    14  
    15  import Tabs from '@theme/Tabs';
    16  import TabItem from '@theme/TabItem';
    17  
    18  
    19  Sends messages to an SQS queue.
    20  
    21  Introduced in version 3.36.0.
    22  
    23  
    24  <Tabs defaultValue="common" values={[
    25    { label: 'Common', value: 'common', },
    26    { label: 'Advanced', value: 'advanced', },
    27  ]}>
    28  
    29  <TabItem value="common">
    30  
    31  ```yaml
    32  # Common config fields, showing default values
    33  output:
    34    label: ""
    35    aws_sqs:
    36      url: ""
    37      message_group_id: ""
    38      message_deduplication_id: ""
    39      max_in_flight: 1
    40      metadata:
    41        exclude_prefixes: []
    42      batching:
    43        count: 0
    44        byte_size: 0
    45        period: ""
    46        check: ""
    47      region: eu-west-1
    48  ```
    49  
    50  </TabItem>
    51  <TabItem value="advanced">
    52  
    53  ```yaml
    54  # All config fields, showing default values
    55  output:
    56    label: ""
    57    aws_sqs:
    58      url: ""
    59      message_group_id: ""
    60      message_deduplication_id: ""
    61      max_in_flight: 1
    62      metadata:
    63        exclude_prefixes: []
    64      batching:
    65        count: 0
    66        byte_size: 0
    67        period: ""
    68        check: ""
    69        processors: []
    70      region: eu-west-1
    71      endpoint: ""
    72      credentials:
    73        profile: ""
    74        id: ""
    75        secret: ""
    76        token: ""
    77        role: ""
    78        role_external_id: ""
    79      max_retries: 0
    80      backoff:
    81        initial_interval: 1s
    82        max_interval: 5s
    83        max_elapsed_time: 30s
    84  ```
    85  
    86  </TabItem>
    87  </Tabs>
    88  
    89  Metadata values are sent along with the payload as attributes with the data type
    90  String. If the number of metadata values in a message exceeds the message
    91  attribute limit (10) then the top ten keys ordered alphabetically will be
    92  selected.
    93  
    94  The fields `message_group_id` and `message_deduplication_id` can be
    95  set dynamically using
    96  [function interpolations](/docs/configuration/interpolation#bloblang-queries), which are
    97  resolved individually for each message of a batch.
    98  
    99  ### Credentials
   100  
   101  By default Benthos will use a shared credentials file when connecting to AWS
   102  services. It's also possible to set them explicitly at the component level,
   103  allowing you to transfer data across accounts. You can find out more
   104  [in this document](/docs/guides/cloud/aws).
   105  
   106  ## Performance
   107  
   108  This output benefits from sending multiple messages in flight in parallel for
   109  improved performance. You can tune the max number of in flight messages with the
   110  field `max_in_flight`.
   111  
   112  This output benefits from sending messages as a batch for improved performance.
   113  Batches can be formed at both the input and output level. You can find out more
   114  [in this doc](/docs/configuration/batching).
   115  
   116  ## Fields
   117  
   118  ### `url`
   119  
   120  The URL of the target SQS queue.
   121  
   122  
   123  Type: `string`  
   124  Default: `""`  
   125  
   126  ### `message_group_id`
   127  
   128  An optional group ID to set for messages.
   129  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   130  
   131  
   132  Type: `string`  
   133  Default: `""`  
   134  
   135  ### `message_deduplication_id`
   136  
   137  An optional deduplication ID to set for messages.
   138  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   139  
   140  
   141  Type: `string`  
   142  Default: `""`  
   143  
   144  ### `max_in_flight`
   145  
   146  The maximum number of messages to have in flight at a given time. Increase this to improve throughput.
   147  
   148  
   149  Type: `int`  
   150  Default: `1`  
   151  
   152  ### `metadata`
   153  
   154  Specify criteria for which metadata values are sent as headers.
   155  
   156  
   157  Type: `object`  
   158  
   159  ### `metadata.exclude_prefixes`
   160  
   161  Provide a list of explicit metadata key prefixes to be excluded when adding metadata to sent messages.
   162  
   163  
   164  Type: `array`  
   165  Default: `[]`  
   166  
   167  ### `batching`
   168  
   169  Allows you to configure a [batching policy](/docs/configuration/batching).
   170  
   171  
   172  Type: `object`  
   173  
   174  ```yaml
   175  # Examples
   176  
   177  batching:
   178    byte_size: 5000
   179    count: 0
   180    period: 1s
   181  
   182  batching:
   183    count: 10
   184    period: 1s
   185  
   186  batching:
   187    check: this.contains("END BATCH")
   188    count: 0
   189    period: 1m
   190  ```
   191  
   192  ### `batching.count`
   193  
   194  A number of messages at which the batch should be flushed. If `0` disables count based batching.
   195  
   196  
   197  Type: `int`  
   198  Default: `0`  
   199  
   200  ### `batching.byte_size`
   201  
   202  An amount of bytes at which the batch should be flushed. If `0` disables size based batching.
   203  
   204  
   205  Type: `int`  
   206  Default: `0`  
   207  
   208  ### `batching.period`
   209  
   210  A period in which an incomplete batch should be flushed regardless of its size.
   211  
   212  
   213  Type: `string`  
   214  Default: `""`  
   215  
   216  ```yaml
   217  # Examples
   218  
   219  period: 1s
   220  
   221  period: 1m
   222  
   223  period: 500ms
   224  ```
   225  
   226  ### `batching.check`
   227  
   228  A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch.
   229  
   230  
   231  Type: `string`  
   232  Default: `""`  
   233  
   234  ```yaml
   235  # Examples
   236  
   237  check: this.type == "end_of_transaction"
   238  ```
   239  
   240  ### `batching.processors`
   241  
   242  A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op.
   243  
   244  
   245  Type: `array`  
   246  Default: `[]`  
   247  
   248  ```yaml
   249  # Examples
   250  
   251  processors:
   252    - archive:
   253        format: lines
   254  
   255  processors:
   256    - archive:
   257        format: json_array
   258  
   259  processors:
   260    - merge_json: {}
   261  ```
   262  
   263  ### `region`
   264  
   265  The AWS region to target.
   266  
   267  
   268  Type: `string`  
   269  Default: `"eu-west-1"`  
   270  
   271  ### `endpoint`
   272  
   273  Allows you to specify a custom endpoint for the AWS API.
   274  
   275  
   276  Type: `string`  
   277  Default: `""`  
   278  
   279  ### `credentials`
   280  
   281  Optional manual configuration of AWS credentials to use. More information can be found [in this document](/docs/guides/cloud/aws).
   282  
   283  
   284  Type: `object`  
   285  
   286  ### `credentials.profile`
   287  
   288  A profile from `~/.aws/credentials` to use.
   289  
   290  
   291  Type: `string`  
   292  Default: `""`  
   293  
   294  ### `credentials.id`
   295  
   296  The ID of credentials to use.
   297  
   298  
   299  Type: `string`  
   300  Default: `""`  
   301  
   302  ### `credentials.secret`
   303  
   304  The secret for the credentials being used.
   305  
   306  
   307  Type: `string`  
   308  Default: `""`  
   309  
   310  ### `credentials.token`
   311  
   312  The token for the credentials being used, required when using short term credentials.
   313  
   314  
   315  Type: `string`  
   316  Default: `""`  
   317  
   318  ### `credentials.role`
   319  
   320  A role ARN to assume.
   321  
   322  
   323  Type: `string`  
   324  Default: `""`  
   325  
   326  ### `credentials.role_external_id`
   327  
   328  An external ID to provide when assuming a role.
   329  
   330  
   331  Type: `string`  
   332  Default: `""`  
   333  
   334  ### `max_retries`
   335  
   336  The maximum number of retries before giving up on the request. If set to zero there is no discrete limit.
   337  
   338  
   339  Type: `int`  
   340  Default: `0`  
   341  
   342  ### `backoff`
   343  
   344  Control time intervals between retry attempts.
   345  
   346  
   347  Type: `object`  
   348  
   349  ### `backoff.initial_interval`
   350  
   351  The initial period to wait between retry attempts.
   352  
   353  
   354  Type: `string`  
   355  Default: `"1s"`  
   356  
   357  ### `backoff.max_interval`
   358  
   359  The maximum period to wait between retry attempts.
   360  
   361  
   362  Type: `string`  
   363  Default: `"5s"`  
   364  
   365  ### `backoff.max_elapsed_time`
   366  
   367  The maximum period to wait before retry attempts are abandoned. If zero then no limit is used.
   368  
   369  
   370  Type: `string`  
   371  Default: `"30s"`  
   372  
   373