github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/azure_queue_storage.md (about)

     1  ---
     2  title: azure_queue_storage
     3  type: output
     4  status: beta
     5  categories: ["Services","Azure"]
     6  ---
     7  
     8  <!--
     9       THIS FILE IS AUTOGENERATED!
    10  
    11       To make changes please edit the contents of:
    12       lib/output/azure_queue_storage.go
    13  -->
    14  
    15  import Tabs from '@theme/Tabs';
    16  import TabItem from '@theme/TabItem';
    17  
    18  :::caution BETA
    19  This component is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with the component is found.
    20  :::
    21  
    22  Sends messages to an Azure Storage Queue.
    23  
    24  Introduced in version 3.36.0.
    25  
    26  
    27  <Tabs defaultValue="common" values={[
    28    { label: 'Common', value: 'common', },
    29    { label: 'Advanced', value: 'advanced', },
    30  ]}>
    31  
    32  <TabItem value="common">
    33  
    34  ```yaml
    35  # Common config fields, showing default values
    36  output:
    37    label: ""
    38    azure_queue_storage:
    39      storage_account: ""
    40      storage_access_key: ""
    41      storage_connection_string: ""
    42      queue_name: ""
    43      max_in_flight: 1
    44      batching:
    45        count: 0
    46        byte_size: 0
    47        period: ""
    48        check: ""
    49  ```
    50  
    51  </TabItem>
    52  <TabItem value="advanced">
    53  
    54  ```yaml
    55  # All config fields, showing default values
    56  output:
    57    label: ""
    58    azure_queue_storage:
    59      storage_account: ""
    60      storage_access_key: ""
    61      storage_connection_string: ""
    62      queue_name: ""
    63      ttl: ""
    64      max_in_flight: 1
    65      batching:
    66        count: 0
    67        byte_size: 0
    68        period: ""
    69        check: ""
    70        processors: []
    71  ```
    72  
    73  </TabItem>
    74  </Tabs>
    75  
    76  Only one authentication method is required, `storage_connection_string` or `storage_account` and `storage_access_key`. If both are set then the `storage_connection_string` is given priority.
    77  
    78  In order to set the `queue_name` you can use function interpolations described [here](/docs/configuration/interpolation#bloblang-queries), which are calculated per message of a batch.
    79  
    80  ## Performance
    81  
    82  This output benefits from sending multiple messages in flight in parallel for
    83  improved performance. You can tune the max number of in flight messages with the
    84  field `max_in_flight`.
    85  
    86  This output benefits from sending messages as a batch for improved performance.
    87  Batches can be formed at both the input and output level. You can find out more
    88  [in this doc](/docs/configuration/batching).
    89  
    90  ## Fields
    91  
    92  ### `storage_account`
    93  
    94  The storage account to upload messages to. This field is ignored if `storage_connection_string` is set.
    95  
    96  
    97  Type: `string`  
    98  Default: `""`  
    99  
   100  ### `storage_access_key`
   101  
   102  The storage account access key. This field is ignored if `storage_connection_string` is set.
   103  
   104  
   105  Type: `string`  
   106  Default: `""`  
   107  
   108  ### `storage_connection_string`
   109  
   110  A storage account connection string. This field is required if `storage_account` and `storage_access_key` are not set.
   111  
   112  
   113  Type: `string`  
   114  Default: `""`  
   115  
   116  ### `queue_name`
   117  
   118  The name of the target Queue Storage queue.
   119  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   120  
   121  
   122  Type: `string`  
   123  Default: `""`  
   124  
   125  ### `ttl`
   126  
   127  The TTL of each individual message as a duration string. Defaults to 0, meaning no retention period is set
   128  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   129  
   130  
   131  Type: `string`  
   132  Default: `""`  
   133  
   134  ```yaml
   135  # Examples
   136  
   137  ttl: 60s
   138  
   139  ttl: 5m
   140  
   141  ttl: 36h
   142  ```
   143  
   144  ### `max_in_flight`
   145  
   146  The maximum number of messages to have in flight at a given time. Increase this to improve throughput.
   147  
   148  
   149  Type: `int`  
   150  Default: `1`  
   151  Requires version 3.45.0 or newer  
   152  
   153  ### `batching`
   154  
   155  Allows you to configure a [batching policy](/docs/configuration/batching).
   156  
   157  
   158  Type: `object`  
   159  
   160  ```yaml
   161  # Examples
   162  
   163  batching:
   164    byte_size: 5000
   165    count: 0
   166    period: 1s
   167  
   168  batching:
   169    count: 10
   170    period: 1s
   171  
   172  batching:
   173    check: this.contains("END BATCH")
   174    count: 0
   175    period: 1m
   176  ```
   177  
   178  ### `batching.count`
   179  
   180  A number of messages at which the batch should be flushed. If `0` disables count based batching.
   181  
   182  
   183  Type: `int`  
   184  Default: `0`  
   185  
   186  ### `batching.byte_size`
   187  
   188  An amount of bytes at which the batch should be flushed. If `0` disables size based batching.
   189  
   190  
   191  Type: `int`  
   192  Default: `0`  
   193  
   194  ### `batching.period`
   195  
   196  A period in which an incomplete batch should be flushed regardless of its size.
   197  
   198  
   199  Type: `string`  
   200  Default: `""`  
   201  
   202  ```yaml
   203  # Examples
   204  
   205  period: 1s
   206  
   207  period: 1m
   208  
   209  period: 500ms
   210  ```
   211  
   212  ### `batching.check`
   213  
   214  A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch.
   215  
   216  
   217  Type: `string`  
   218  Default: `""`  
   219  
   220  ```yaml
   221  # Examples
   222  
   223  check: this.type == "end_of_transaction"
   224  ```
   225  
   226  ### `batching.processors`
   227  
   228  A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op.
   229  
   230  
   231  Type: `array`  
   232  Default: `[]`  
   233  
   234  ```yaml
   235  # Examples
   236  
   237  processors:
   238    - archive:
   239        format: lines
   240  
   241  processors:
   242    - archive:
   243        format: json_array
   244  
   245  processors:
   246    - merge_json: {}
   247  ```
   248  
   249