github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/table_storage.md (about)

     1  ---
     2  title: table_storage
     3  type: output
     4  status: deprecated
     5  categories: ["Services","Azure"]
     6  ---
     7  
     8  <!--
     9       THIS FILE IS AUTOGENERATED!
    10  
    11       To make changes please edit the contents of:
    12       lib/output/table_storage.go
    13  -->
    14  
    15  import Tabs from '@theme/Tabs';
    16  import TabItem from '@theme/TabItem';
    17  
    18  :::warning DEPRECATED
    19  This component is deprecated and will be removed in the next major version release. Please consider moving onto [alternative components](#alternatives).
    20  :::
    21  This component has been renamed to [`azure_table_storage`](/docs/components/outputs/azure_table_storage).
    22  
    23  
    24  <Tabs defaultValue="common" values={[
    25    { label: 'Common', value: 'common', },
    26    { label: 'Advanced', value: 'advanced', },
    27  ]}>
    28  
    29  <TabItem value="common">
    30  
    31  ```yaml
    32  # Common config fields, showing default values
    33  output:
    34    label: ""
    35    table_storage:
    36      storage_account: ""
    37      storage_access_key: ""
    38      storage_connection_string: ""
    39      table_name: ""
    40      partition_key: ""
    41      row_key: ""
    42      properties: {}
    43      max_in_flight: 1
    44      batching:
    45        count: 0
    46        byte_size: 0
    47        period: ""
    48        check: ""
    49  ```
    50  
    51  </TabItem>
    52  <TabItem value="advanced">
    53  
    54  ```yaml
    55  # All config fields, showing default values
    56  output:
    57    label: ""
    58    table_storage:
    59      storage_account: ""
    60      storage_access_key: ""
    61      storage_connection_string: ""
    62      table_name: ""
    63      partition_key: ""
    64      row_key: ""
    65      properties: {}
    66      insert_type: INSERT
    67      max_in_flight: 1
    68      timeout: 5s
    69      batching:
    70        count: 0
    71        byte_size: 0
    72        period: ""
    73        check: ""
    74        processors: []
    75  ```
    76  
    77  </TabItem>
    78  </Tabs>
    79  
    80  
    81  ## Performance
    82  
    83  This output benefits from sending multiple messages in flight in parallel for
    84  improved performance. You can tune the max number of in flight messages with the
    85  field `max_in_flight`.
    86  
    87  This output benefits from sending messages as a batch for improved performance.
    88  Batches can be formed at both the input and output level. You can find out more
    89  [in this doc](/docs/configuration/batching).
    90  
    91  ## Fields
    92  
    93  ### `storage_account`
    94  
    95  The storage account to upload messages to. This field is ignored if `storage_connection_string` is set.
    96  
    97  
    98  Type: `string`  
    99  Default: `""`  
   100  
   101  ### `storage_access_key`
   102  
   103  The storage account access key. This field is ignored if `storage_connection_string` is set.
   104  
   105  
   106  Type: `string`  
   107  Default: `""`  
   108  
   109  ### `storage_connection_string`
   110  
   111  A storage account connection string. This field is required if `storage_account` and `storage_access_key` are not set.
   112  
   113  
   114  Type: `string`  
   115  Default: `""`  
   116  
   117  ### `table_name`
   118  
   119  The table to store messages into.
   120  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   121  
   122  
   123  Type: `string`  
   124  Default: `""`  
   125  
   126  ```yaml
   127  # Examples
   128  
   129  table_name: ${!meta("kafka_topic")}
   130  ```
   131  
   132  ### `partition_key`
   133  
   134  The partition key.
   135  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   136  
   137  
   138  Type: `string`  
   139  Default: `""`  
   140  
   141  ```yaml
   142  # Examples
   143  
   144  partition_key: ${!json("date")}
   145  ```
   146  
   147  ### `row_key`
   148  
   149  The row key.
   150  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   151  
   152  
   153  Type: `string`  
   154  Default: `""`  
   155  
   156  ```yaml
   157  # Examples
   158  
   159  row_key: ${!json("device")}-${!uuid_v4()}
   160  ```
   161  
   162  ### `properties`
   163  
   164  A map of properties to store into the table.
   165  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   166  
   167  
   168  Type: `object`  
   169  Default: `{}`  
   170  
   171  ### `insert_type`
   172  
   173  Type of insert operation
   174  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   175  
   176  
   177  Type: `string`  
   178  Default: `"INSERT"`  
   179  Options: `INSERT`, `INSERT_MERGE`, `INSERT_REPLACE`.
   180  
   181  ### `max_in_flight`
   182  
   183  The maximum number of messages to have in flight at a given time. Increase this to improve throughput.
   184  
   185  
   186  Type: `int`  
   187  Default: `1`  
   188  
   189  ### `timeout`
   190  
   191  The maximum period to wait on an upload before abandoning it and reattempting.
   192  
   193  
   194  Type: `string`  
   195  Default: `"5s"`  
   196  
   197  ### `batching`
   198  
   199  Allows you to configure a [batching policy](/docs/configuration/batching).
   200  
   201  
   202  Type: `object`  
   203  
   204  ```yaml
   205  # Examples
   206  
   207  batching:
   208    byte_size: 5000
   209    count: 0
   210    period: 1s
   211  
   212  batching:
   213    count: 10
   214    period: 1s
   215  
   216  batching:
   217    check: this.contains("END BATCH")
   218    count: 0
   219    period: 1m
   220  ```
   221  
   222  ### `batching.count`
   223  
   224  A number of messages at which the batch should be flushed. If `0` disables count based batching.
   225  
   226  
   227  Type: `int`  
   228  Default: `0`  
   229  
   230  ### `batching.byte_size`
   231  
   232  An amount of bytes at which the batch should be flushed. If `0` disables size based batching.
   233  
   234  
   235  Type: `int`  
   236  Default: `0`  
   237  
   238  ### `batching.period`
   239  
   240  A period in which an incomplete batch should be flushed regardless of its size.
   241  
   242  
   243  Type: `string`  
   244  Default: `""`  
   245  
   246  ```yaml
   247  # Examples
   248  
   249  period: 1s
   250  
   251  period: 1m
   252  
   253  period: 500ms
   254  ```
   255  
   256  ### `batching.check`
   257  
   258  A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch.
   259  
   260  
   261  Type: `string`  
   262  Default: `""`  
   263  
   264  ```yaml
   265  # Examples
   266  
   267  check: this.type == "end_of_transaction"
   268  ```
   269  
   270  ### `batching.processors`
   271  
   272  A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op.
   273  
   274  
   275  Type: `array`  
   276  Default: `[]`  
   277  
   278  ```yaml
   279  # Examples
   280  
   281  processors:
   282    - archive:
   283        format: lines
   284  
   285  processors:
   286    - archive:
   287        format: json_array
   288  
   289  processors:
   290    - merge_json: {}
   291  ```
   292  
   293