github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/aws_sqs.md (about) 1 --- 2 title: aws_sqs 3 type: output 4 status: stable 5 categories: ["Services","AWS"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/output/aws_sqs.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 19 Sends messages to an SQS queue. 20 21 Introduced in version 3.36.0. 22 23 24 <Tabs defaultValue="common" values={[ 25 { label: 'Common', value: 'common', }, 26 { label: 'Advanced', value: 'advanced', }, 27 ]}> 28 29 <TabItem value="common"> 30 31 ```yaml 32 # Common config fields, showing default values 33 output: 34 label: "" 35 aws_sqs: 36 url: "" 37 message_group_id: "" 38 message_deduplication_id: "" 39 max_in_flight: 1 40 metadata: 41 exclude_prefixes: [] 42 batching: 43 count: 0 44 byte_size: 0 45 period: "" 46 check: "" 47 region: eu-west-1 48 ``` 49 50 </TabItem> 51 <TabItem value="advanced"> 52 53 ```yaml 54 # All config fields, showing default values 55 output: 56 label: "" 57 aws_sqs: 58 url: "" 59 message_group_id: "" 60 message_deduplication_id: "" 61 max_in_flight: 1 62 metadata: 63 exclude_prefixes: [] 64 batching: 65 count: 0 66 byte_size: 0 67 period: "" 68 check: "" 69 processors: [] 70 region: eu-west-1 71 endpoint: "" 72 credentials: 73 profile: "" 74 id: "" 75 secret: "" 76 token: "" 77 role: "" 78 role_external_id: "" 79 max_retries: 0 80 backoff: 81 initial_interval: 1s 82 max_interval: 5s 83 max_elapsed_time: 30s 84 ``` 85 86 </TabItem> 87 </Tabs> 88 89 Metadata values are sent along with the payload as attributes with the data type 90 String. If the number of metadata values in a message exceeds the message 91 attribute limit (10) then the top ten keys ordered alphabetically will be 92 selected. 93 94 The fields `message_group_id` and `message_deduplication_id` can be 95 set dynamically using 96 [function interpolations](/docs/configuration/interpolation#bloblang-queries), which are 97 resolved individually for each message of a batch. 98 99 ### Credentials 100 101 By default Benthos will use a shared credentials file when connecting to AWS 102 services. It's also possible to set them explicitly at the component level, 103 allowing you to transfer data across accounts. You can find out more 104 [in this document](/docs/guides/cloud/aws). 105 106 ## Performance 107 108 This output benefits from sending multiple messages in flight in parallel for 109 improved performance. You can tune the max number of in flight messages with the 110 field `max_in_flight`. 111 112 This output benefits from sending messages as a batch for improved performance. 113 Batches can be formed at both the input and output level. You can find out more 114 [in this doc](/docs/configuration/batching). 115 116 ## Fields 117 118 ### `url` 119 120 The URL of the target SQS queue. 121 122 123 Type: `string` 124 Default: `""` 125 126 ### `message_group_id` 127 128 An optional group ID to set for messages. 129 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 130 131 132 Type: `string` 133 Default: `""` 134 135 ### `message_deduplication_id` 136 137 An optional deduplication ID to set for messages. 138 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 139 140 141 Type: `string` 142 Default: `""` 143 144 ### `max_in_flight` 145 146 The maximum number of messages to have in flight at a given time. Increase this to improve throughput. 147 148 149 Type: `int` 150 Default: `1` 151 152 ### `metadata` 153 154 Specify criteria for which metadata values are sent as headers. 155 156 157 Type: `object` 158 159 ### `metadata.exclude_prefixes` 160 161 Provide a list of explicit metadata key prefixes to be excluded when adding metadata to sent messages. 162 163 164 Type: `array` 165 Default: `[]` 166 167 ### `batching` 168 169 Allows you to configure a [batching policy](/docs/configuration/batching). 170 171 172 Type: `object` 173 174 ```yaml 175 # Examples 176 177 batching: 178 byte_size: 5000 179 count: 0 180 period: 1s 181 182 batching: 183 count: 10 184 period: 1s 185 186 batching: 187 check: this.contains("END BATCH") 188 count: 0 189 period: 1m 190 ``` 191 192 ### `batching.count` 193 194 A number of messages at which the batch should be flushed. If `0` disables count based batching. 195 196 197 Type: `int` 198 Default: `0` 199 200 ### `batching.byte_size` 201 202 An amount of bytes at which the batch should be flushed. If `0` disables size based batching. 203 204 205 Type: `int` 206 Default: `0` 207 208 ### `batching.period` 209 210 A period in which an incomplete batch should be flushed regardless of its size. 211 212 213 Type: `string` 214 Default: `""` 215 216 ```yaml 217 # Examples 218 219 period: 1s 220 221 period: 1m 222 223 period: 500ms 224 ``` 225 226 ### `batching.check` 227 228 A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch. 229 230 231 Type: `string` 232 Default: `""` 233 234 ```yaml 235 # Examples 236 237 check: this.type == "end_of_transaction" 238 ``` 239 240 ### `batching.processors` 241 242 A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op. 243 244 245 Type: `array` 246 Default: `[]` 247 248 ```yaml 249 # Examples 250 251 processors: 252 - archive: 253 format: lines 254 255 processors: 256 - archive: 257 format: json_array 258 259 processors: 260 - merge_json: {} 261 ``` 262 263 ### `region` 264 265 The AWS region to target. 266 267 268 Type: `string` 269 Default: `"eu-west-1"` 270 271 ### `endpoint` 272 273 Allows you to specify a custom endpoint for the AWS API. 274 275 276 Type: `string` 277 Default: `""` 278 279 ### `credentials` 280 281 Optional manual configuration of AWS credentials to use. More information can be found [in this document](/docs/guides/cloud/aws). 282 283 284 Type: `object` 285 286 ### `credentials.profile` 287 288 A profile from `~/.aws/credentials` to use. 289 290 291 Type: `string` 292 Default: `""` 293 294 ### `credentials.id` 295 296 The ID of credentials to use. 297 298 299 Type: `string` 300 Default: `""` 301 302 ### `credentials.secret` 303 304 The secret for the credentials being used. 305 306 307 Type: `string` 308 Default: `""` 309 310 ### `credentials.token` 311 312 The token for the credentials being used, required when using short term credentials. 313 314 315 Type: `string` 316 Default: `""` 317 318 ### `credentials.role` 319 320 A role ARN to assume. 321 322 323 Type: `string` 324 Default: `""` 325 326 ### `credentials.role_external_id` 327 328 An external ID to provide when assuming a role. 329 330 331 Type: `string` 332 Default: `""` 333 334 ### `max_retries` 335 336 The maximum number of retries before giving up on the request. If set to zero there is no discrete limit. 337 338 339 Type: `int` 340 Default: `0` 341 342 ### `backoff` 343 344 Control time intervals between retry attempts. 345 346 347 Type: `object` 348 349 ### `backoff.initial_interval` 350 351 The initial period to wait between retry attempts. 352 353 354 Type: `string` 355 Default: `"1s"` 356 357 ### `backoff.max_interval` 358 359 The maximum period to wait between retry attempts. 360 361 362 Type: `string` 363 Default: `"5s"` 364 365 ### `backoff.max_elapsed_time` 366 367 The maximum period to wait before retry attempts are abandoned. If zero then no limit is used. 368 369 370 Type: `string` 371 Default: `"30s"` 372 373