github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/table_storage.md (about) 1 --- 2 title: table_storage 3 type: output 4 status: deprecated 5 categories: ["Services","Azure"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/output/table_storage.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 :::warning DEPRECATED 19 This component is deprecated and will be removed in the next major version release. Please consider moving onto [alternative components](#alternatives). 20 ::: 21 This component has been renamed to [`azure_table_storage`](/docs/components/outputs/azure_table_storage). 22 23 24 <Tabs defaultValue="common" values={[ 25 { label: 'Common', value: 'common', }, 26 { label: 'Advanced', value: 'advanced', }, 27 ]}> 28 29 <TabItem value="common"> 30 31 ```yaml 32 # Common config fields, showing default values 33 output: 34 label: "" 35 table_storage: 36 storage_account: "" 37 storage_access_key: "" 38 storage_connection_string: "" 39 table_name: "" 40 partition_key: "" 41 row_key: "" 42 properties: {} 43 max_in_flight: 1 44 batching: 45 count: 0 46 byte_size: 0 47 period: "" 48 check: "" 49 ``` 50 51 </TabItem> 52 <TabItem value="advanced"> 53 54 ```yaml 55 # All config fields, showing default values 56 output: 57 label: "" 58 table_storage: 59 storage_account: "" 60 storage_access_key: "" 61 storage_connection_string: "" 62 table_name: "" 63 partition_key: "" 64 row_key: "" 65 properties: {} 66 insert_type: INSERT 67 max_in_flight: 1 68 timeout: 5s 69 batching: 70 count: 0 71 byte_size: 0 72 period: "" 73 check: "" 74 processors: [] 75 ``` 76 77 </TabItem> 78 </Tabs> 79 80 81 ## Performance 82 83 This output benefits from sending multiple messages in flight in parallel for 84 improved performance. You can tune the max number of in flight messages with the 85 field `max_in_flight`. 86 87 This output benefits from sending messages as a batch for improved performance. 88 Batches can be formed at both the input and output level. You can find out more 89 [in this doc](/docs/configuration/batching). 90 91 ## Fields 92 93 ### `storage_account` 94 95 The storage account to upload messages to. This field is ignored if `storage_connection_string` is set. 96 97 98 Type: `string` 99 Default: `""` 100 101 ### `storage_access_key` 102 103 The storage account access key. This field is ignored if `storage_connection_string` is set. 104 105 106 Type: `string` 107 Default: `""` 108 109 ### `storage_connection_string` 110 111 A storage account connection string. This field is required if `storage_account` and `storage_access_key` are not set. 112 113 114 Type: `string` 115 Default: `""` 116 117 ### `table_name` 118 119 The table to store messages into. 120 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 121 122 123 Type: `string` 124 Default: `""` 125 126 ```yaml 127 # Examples 128 129 table_name: ${!meta("kafka_topic")} 130 ``` 131 132 ### `partition_key` 133 134 The partition key. 135 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 136 137 138 Type: `string` 139 Default: `""` 140 141 ```yaml 142 # Examples 143 144 partition_key: ${!json("date")} 145 ``` 146 147 ### `row_key` 148 149 The row key. 150 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 151 152 153 Type: `string` 154 Default: `""` 155 156 ```yaml 157 # Examples 158 159 row_key: ${!json("device")}-${!uuid_v4()} 160 ``` 161 162 ### `properties` 163 164 A map of properties to store into the table. 165 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 166 167 168 Type: `object` 169 Default: `{}` 170 171 ### `insert_type` 172 173 Type of insert operation 174 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 175 176 177 Type: `string` 178 Default: `"INSERT"` 179 Options: `INSERT`, `INSERT_MERGE`, `INSERT_REPLACE`. 180 181 ### `max_in_flight` 182 183 The maximum number of messages to have in flight at a given time. Increase this to improve throughput. 184 185 186 Type: `int` 187 Default: `1` 188 189 ### `timeout` 190 191 The maximum period to wait on an upload before abandoning it and reattempting. 192 193 194 Type: `string` 195 Default: `"5s"` 196 197 ### `batching` 198 199 Allows you to configure a [batching policy](/docs/configuration/batching). 200 201 202 Type: `object` 203 204 ```yaml 205 # Examples 206 207 batching: 208 byte_size: 5000 209 count: 0 210 period: 1s 211 212 batching: 213 count: 10 214 period: 1s 215 216 batching: 217 check: this.contains("END BATCH") 218 count: 0 219 period: 1m 220 ``` 221 222 ### `batching.count` 223 224 A number of messages at which the batch should be flushed. If `0` disables count based batching. 225 226 227 Type: `int` 228 Default: `0` 229 230 ### `batching.byte_size` 231 232 An amount of bytes at which the batch should be flushed. If `0` disables size based batching. 233 234 235 Type: `int` 236 Default: `0` 237 238 ### `batching.period` 239 240 A period in which an incomplete batch should be flushed regardless of its size. 241 242 243 Type: `string` 244 Default: `""` 245 246 ```yaml 247 # Examples 248 249 period: 1s 250 251 period: 1m 252 253 period: 500ms 254 ``` 255 256 ### `batching.check` 257 258 A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch. 259 260 261 Type: `string` 262 Default: `""` 263 264 ```yaml 265 # Examples 266 267 check: this.type == "end_of_transaction" 268 ``` 269 270 ### `batching.processors` 271 272 A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op. 273 274 275 Type: `array` 276 Default: `[]` 277 278 ```yaml 279 # Examples 280 281 processors: 282 - archive: 283 format: lines 284 285 processors: 286 - archive: 287 format: json_array 288 289 processors: 290 - merge_json: {} 291 ``` 292 293