github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/sqs.md (about) 1 --- 2 title: sqs 3 type: output 4 status: deprecated 5 categories: ["Services","AWS"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/output/sqs.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 :::warning DEPRECATED 19 This component is deprecated and will be removed in the next major version release. Please consider moving onto [alternative components](#alternatives). 20 ::: 21 22 Sends messages to an SQS queue. 23 24 25 <Tabs defaultValue="common" values={[ 26 { label: 'Common', value: 'common', }, 27 { label: 'Advanced', value: 'advanced', }, 28 ]}> 29 30 <TabItem value="common"> 31 32 ```yaml 33 # Common config fields, showing default values 34 output: 35 label: "" 36 sqs: 37 url: "" 38 message_group_id: "" 39 message_deduplication_id: "" 40 max_in_flight: 1 41 metadata: 42 exclude_prefixes: [] 43 batching: 44 count: 0 45 byte_size: 0 46 period: "" 47 check: "" 48 region: eu-west-1 49 ``` 50 51 </TabItem> 52 <TabItem value="advanced"> 53 54 ```yaml 55 # All config fields, showing default values 56 output: 57 label: "" 58 sqs: 59 url: "" 60 message_group_id: "" 61 message_deduplication_id: "" 62 max_in_flight: 1 63 metadata: 64 exclude_prefixes: [] 65 batching: 66 count: 0 67 byte_size: 0 68 period: "" 69 check: "" 70 processors: [] 71 region: eu-west-1 72 endpoint: "" 73 credentials: 74 profile: "" 75 id: "" 76 secret: "" 77 token: "" 78 role: "" 79 role_external_id: "" 80 max_retries: 0 81 backoff: 82 initial_interval: 1s 83 max_interval: 5s 84 max_elapsed_time: 30s 85 ``` 86 87 </TabItem> 88 </Tabs> 89 90 ## Alternatives 91 92 This output has been renamed to [`aws_sqs`](/docs/components/outputs/aws_sqs). 93 94 Metadata values are sent along with the payload as attributes with the data type 95 String. If the number of metadata values in a message exceeds the message 96 attribute limit (10) then the top ten keys ordered alphabetically will be 97 selected. 98 99 The fields `message_group_id` and `message_deduplication_id` can be 100 set dynamically using 101 [function interpolations](/docs/configuration/interpolation#bloblang-queries), which are 102 resolved individually for each message of a batch. 103 104 ### Credentials 105 106 By default Benthos will use a shared credentials file when connecting to AWS 107 services. It's also possible to set them explicitly at the component level, 108 allowing you to transfer data across accounts. You can find out more 109 [in this document](/docs/guides/cloud/aws). 110 111 ## Performance 112 113 This output benefits from sending multiple messages in flight in parallel for 114 improved performance. You can tune the max number of in flight messages with the 115 field `max_in_flight`. 116 117 This output benefits from sending messages as a batch for improved performance. 118 Batches can be formed at both the input and output level. You can find out more 119 [in this doc](/docs/configuration/batching). 120 121 ## Fields 122 123 ### `url` 124 125 The URL of the target SQS queue. 126 127 128 Type: `string` 129 Default: `""` 130 131 ### `message_group_id` 132 133 An optional group ID to set for messages. 134 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 135 136 137 Type: `string` 138 Default: `""` 139 140 ### `message_deduplication_id` 141 142 An optional deduplication ID to set for messages. 143 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 144 145 146 Type: `string` 147 Default: `""` 148 149 ### `max_in_flight` 150 151 The maximum number of messages to have in flight at a given time. Increase this to improve throughput. 152 153 154 Type: `int` 155 Default: `1` 156 157 ### `metadata` 158 159 Specify criteria for which metadata values are sent as headers. 160 161 162 Type: `object` 163 164 ### `metadata.exclude_prefixes` 165 166 Provide a list of explicit metadata key prefixes to be excluded when adding metadata to sent messages. 167 168 169 Type: `array` 170 Default: `[]` 171 172 ### `batching` 173 174 Allows you to configure a [batching policy](/docs/configuration/batching). 175 176 177 Type: `object` 178 179 ```yaml 180 # Examples 181 182 batching: 183 byte_size: 5000 184 count: 0 185 period: 1s 186 187 batching: 188 count: 10 189 period: 1s 190 191 batching: 192 check: this.contains("END BATCH") 193 count: 0 194 period: 1m 195 ``` 196 197 ### `batching.count` 198 199 A number of messages at which the batch should be flushed. If `0` disables count based batching. 200 201 202 Type: `int` 203 Default: `0` 204 205 ### `batching.byte_size` 206 207 An amount of bytes at which the batch should be flushed. If `0` disables size based batching. 208 209 210 Type: `int` 211 Default: `0` 212 213 ### `batching.period` 214 215 A period in which an incomplete batch should be flushed regardless of its size. 216 217 218 Type: `string` 219 Default: `""` 220 221 ```yaml 222 # Examples 223 224 period: 1s 225 226 period: 1m 227 228 period: 500ms 229 ``` 230 231 ### `batching.check` 232 233 A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch. 234 235 236 Type: `string` 237 Default: `""` 238 239 ```yaml 240 # Examples 241 242 check: this.type == "end_of_transaction" 243 ``` 244 245 ### `batching.processors` 246 247 A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op. 248 249 250 Type: `array` 251 Default: `[]` 252 253 ```yaml 254 # Examples 255 256 processors: 257 - archive: 258 format: lines 259 260 processors: 261 - archive: 262 format: json_array 263 264 processors: 265 - merge_json: {} 266 ``` 267 268 ### `region` 269 270 The AWS region to target. 271 272 273 Type: `string` 274 Default: `"eu-west-1"` 275 276 ### `endpoint` 277 278 Allows you to specify a custom endpoint for the AWS API. 279 280 281 Type: `string` 282 Default: `""` 283 284 ### `credentials` 285 286 Optional manual configuration of AWS credentials to use. More information can be found [in this document](/docs/guides/cloud/aws). 287 288 289 Type: `object` 290 291 ### `credentials.profile` 292 293 A profile from `~/.aws/credentials` to use. 294 295 296 Type: `string` 297 Default: `""` 298 299 ### `credentials.id` 300 301 The ID of credentials to use. 302 303 304 Type: `string` 305 Default: `""` 306 307 ### `credentials.secret` 308 309 The secret for the credentials being used. 310 311 312 Type: `string` 313 Default: `""` 314 315 ### `credentials.token` 316 317 The token for the credentials being used, required when using short term credentials. 318 319 320 Type: `string` 321 Default: `""` 322 323 ### `credentials.role` 324 325 A role ARN to assume. 326 327 328 Type: `string` 329 Default: `""` 330 331 ### `credentials.role_external_id` 332 333 An external ID to provide when assuming a role. 334 335 336 Type: `string` 337 Default: `""` 338 339 ### `max_retries` 340 341 The maximum number of retries before giving up on the request. If set to zero there is no discrete limit. 342 343 344 Type: `int` 345 Default: `0` 346 347 ### `backoff` 348 349 Control time intervals between retry attempts. 350 351 352 Type: `object` 353 354 ### `backoff.initial_interval` 355 356 The initial period to wait between retry attempts. 357 358 359 Type: `string` 360 Default: `"1s"` 361 362 ### `backoff.max_interval` 363 364 The maximum period to wait between retry attempts. 365 366 367 Type: `string` 368 Default: `"5s"` 369 370 ### `backoff.max_elapsed_time` 371 372 The maximum period to wait before retry attempts are abandoned. If zero then no limit is used. 373 374 375 Type: `string` 376 Default: `"30s"` 377 378