github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/cassandra.md (about) 1 --- 2 title: cassandra 3 type: output 4 status: beta 5 --- 6 7 <!-- 8 THIS FILE IS AUTOGENERATED! 9 10 To make changes please edit the contents of: 11 lib/output/cassandra.go 12 --> 13 14 import Tabs from '@theme/Tabs'; 15 import TabItem from '@theme/TabItem'; 16 17 :::caution BETA 18 This component is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with the component is found. 19 ::: 20 21 Runs a query against a Cassandra database for each message in order to insert data. 22 23 24 <Tabs defaultValue="common" values={[ 25 { label: 'Common', value: 'common', }, 26 { label: 'Advanced', value: 'advanced', }, 27 ]}> 28 29 <TabItem value="common"> 30 31 ```yaml 32 # Common config fields, showing default values 33 output: 34 label: "" 35 cassandra: 36 addresses: [] 37 query: "" 38 args_mapping: "" 39 timeout: 600ms 40 max_in_flight: 1 41 batching: 42 count: 0 43 byte_size: 0 44 period: "" 45 check: "" 46 ``` 47 48 </TabItem> 49 <TabItem value="advanced"> 50 51 ```yaml 52 # All config fields, showing default values 53 output: 54 label: "" 55 cassandra: 56 addresses: [] 57 tls: 58 enabled: false 59 skip_cert_verify: false 60 enable_renegotiation: false 61 root_cas: "" 62 root_cas_file: "" 63 client_certs: [] 64 password_authenticator: 65 enabled: false 66 username: "" 67 password: "" 68 disable_initial_host_lookup: false 69 query: "" 70 args_mapping: "" 71 consistency: QUORUM 72 max_retries: 3 73 backoff: 74 initial_interval: 1s 75 max_interval: 5s 76 timeout: 600ms 77 max_in_flight: 1 78 batching: 79 count: 0 80 byte_size: 0 81 period: "" 82 check: "" 83 processors: [] 84 ``` 85 86 </TabItem> 87 </Tabs> 88 89 Query arguments can be set using [interpolation functions](/docs/configuration/interpolation#bloblang-queries) in the `args` field or by creating a bloblang array for the fields using the `args_mapping` field. 90 91 When populating timestamp columns the value must either be a string in ISO 8601 format (2006-01-02T15:04:05Z07:00), or an integer representing unix time in seconds. 92 93 ## Performance 94 95 This output benefits from sending multiple messages in flight in parallel for 96 improved performance. You can tune the max number of in flight messages with the 97 field `max_in_flight`. 98 99 This output benefits from sending messages as a batch for improved performance. 100 Batches can be formed at both the input and output level. You can find out more 101 [in this doc](/docs/configuration/batching). 102 103 ## Examples 104 105 <Tabs defaultValue="Basic Inserts" values={[ 106 { label: 'Basic Inserts', value: 'Basic Inserts', }, 107 { label: 'Insert JSON Documents', value: 'Insert JSON Documents', }, 108 ]}> 109 110 <TabItem value="Basic Inserts"> 111 112 If we were to create a table with some basic columns with `CREATE TABLE foo.bar (id int primary key, content text, created_at timestamp);`, and were processing JSON documents of the form `{"id":"342354354","content":"hello world","timestamp":1605219406}`, we could populate our table with the following config: 113 114 ```yaml 115 output: 116 cassandra: 117 addresses: 118 - localhost:9042 119 query: 'INSERT INTO foo.bar (id, content, created_at) VALUES (?, ?, ?)' 120 args_mapping: | 121 root = [ 122 this.id, 123 this.content, 124 this.timestamp 125 ] 126 batching: 127 count: 500 128 ``` 129 130 </TabItem> 131 <TabItem value="Insert JSON Documents"> 132 133 The following example inserts JSON documents into the table `footable` of the keyspace `foospace` using INSERT JSON (https://cassandra.apache.org/doc/latest/cql/json.html#insert-json). 134 135 ```yaml 136 output: 137 cassandra: 138 addresses: 139 - localhost:9042 140 query: 'INSERT INTO foospace.footable JSON ?' 141 args_mapping: 'root = [ this ]' 142 batching: 143 count: 500 144 ``` 145 146 </TabItem> 147 </Tabs> 148 149 ## Fields 150 151 ### `addresses` 152 153 A list of Cassandra nodes to connect to. Multiple comma separated addresses can be specified on a single line. 154 155 156 Type: `array` 157 Default: `[]` 158 159 ```yaml 160 # Examples 161 162 addresses: 163 - localhost:9042 164 165 addresses: 166 - foo:9042 167 - bar:9042 168 169 addresses: 170 - foo:9042,bar:9042 171 ``` 172 173 ### `tls` 174 175 Custom TLS settings can be used to override system defaults. 176 177 178 Type: `object` 179 180 ### `tls.enabled` 181 182 Whether custom TLS settings are enabled. 183 184 185 Type: `bool` 186 Default: `false` 187 188 ### `tls.skip_cert_verify` 189 190 Whether to skip server side certificate verification. 191 192 193 Type: `bool` 194 Default: `false` 195 196 ### `tls.enable_renegotiation` 197 198 Whether to allow the remote server to repeatedly request renegotiation. Enable this option if you're seeing the error message `local error: tls: no renegotiation`. 199 200 201 Type: `bool` 202 Default: `false` 203 Requires version 3.45.0 or newer 204 205 ### `tls.root_cas` 206 207 An optional root certificate authority to use. This is a string, representing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate. 208 209 210 Type: `string` 211 Default: `""` 212 213 ```yaml 214 # Examples 215 216 root_cas: |- 217 -----BEGIN CERTIFICATE----- 218 ... 219 -----END CERTIFICATE----- 220 ``` 221 222 ### `tls.root_cas_file` 223 224 An optional path of a root certificate authority file to use. This is a file, often with a .pem extension, containing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate. 225 226 227 Type: `string` 228 Default: `""` 229 230 ```yaml 231 # Examples 232 233 root_cas_file: ./root_cas.pem 234 ``` 235 236 ### `tls.client_certs` 237 238 A list of client certificates to use. For each certificate either the fields `cert` and `key`, or `cert_file` and `key_file` should be specified, but not both. 239 240 241 Type: `array` 242 Default: `[]` 243 244 ```yaml 245 # Examples 246 247 client_certs: 248 - cert: foo 249 key: bar 250 251 client_certs: 252 - cert_file: ./example.pem 253 key_file: ./example.key 254 ``` 255 256 ### `tls.client_certs[].cert` 257 258 A plain text certificate to use. 259 260 261 Type: `string` 262 Default: `""` 263 264 ### `tls.client_certs[].key` 265 266 A plain text certificate key to use. 267 268 269 Type: `string` 270 Default: `""` 271 272 ### `tls.client_certs[].cert_file` 273 274 The path to a certificate to use. 275 276 277 Type: `string` 278 Default: `""` 279 280 ### `tls.client_certs[].key_file` 281 282 The path of a certificate key to use. 283 284 285 Type: `string` 286 Default: `""` 287 288 ### `password_authenticator` 289 290 An object containing the username and password. 291 292 293 Type: `object` 294 295 ### `password_authenticator.enabled` 296 297 Whether to use password authentication. 298 299 300 Type: `bool` 301 Default: `false` 302 303 ### `password_authenticator.username` 304 305 A username. 306 307 308 Type: `string` 309 Default: `""` 310 311 ### `password_authenticator.password` 312 313 A password. 314 315 316 Type: `string` 317 Default: `""` 318 319 ### `disable_initial_host_lookup` 320 321 If enabled the driver will not attempt to get host info from the system.peers table. This can speed up queries but will mean that data_centre, rack and token information will not be available. 322 323 324 Type: `bool` 325 Default: `false` 326 327 ### `query` 328 329 A query to execute for each message. 330 331 332 Type: `string` 333 Default: `""` 334 335 ### `args_mapping` 336 337 A [Bloblang mapping](/docs/guides/bloblang/about) that can be used to provide arguments to Cassandra queries. The result of the query must be an array containing a matching number of elements to the query arguments. 338 339 340 Type: `string` 341 Default: `""` 342 Requires version 3.55.0 or newer 343 344 ### `consistency` 345 346 The consistency level to use. 347 348 349 Type: `string` 350 Default: `"QUORUM"` 351 Options: `ANY`, `ONE`, `TWO`, `THREE`, `QUORUM`, `ALL`, `LOCAL_QUORUM`, `EACH_QUORUM`, `LOCAL_ONE`. 352 353 ### `max_retries` 354 355 The maximum number of retries before giving up on a request. 356 357 358 Type: `int` 359 Default: `3` 360 361 ### `backoff` 362 363 Control time intervals between retry attempts. 364 365 366 Type: `object` 367 368 ### `backoff.initial_interval` 369 370 The initial period to wait between retry attempts. 371 372 373 Type: `string` 374 Default: `"1s"` 375 376 ### `backoff.max_interval` 377 378 The maximum period to wait between retry attempts. 379 380 381 Type: `string` 382 Default: `"5s"` 383 384 ### `timeout` 385 386 The client connection timeout. 387 388 389 Type: `string` 390 Default: `"600ms"` 391 Requires version 3.63.0 or newer 392 393 ### `max_in_flight` 394 395 The maximum number of messages to have in flight at a given time. Increase this to improve throughput. 396 397 398 Type: `int` 399 Default: `1` 400 401 ### `batching` 402 403 Allows you to configure a [batching policy](/docs/configuration/batching). 404 405 406 Type: `object` 407 408 ```yaml 409 # Examples 410 411 batching: 412 byte_size: 5000 413 count: 0 414 period: 1s 415 416 batching: 417 count: 10 418 period: 1s 419 420 batching: 421 check: this.contains("END BATCH") 422 count: 0 423 period: 1m 424 ``` 425 426 ### `batching.count` 427 428 A number of messages at which the batch should be flushed. If `0` disables count based batching. 429 430 431 Type: `int` 432 Default: `0` 433 434 ### `batching.byte_size` 435 436 An amount of bytes at which the batch should be flushed. If `0` disables size based batching. 437 438 439 Type: `int` 440 Default: `0` 441 442 ### `batching.period` 443 444 A period in which an incomplete batch should be flushed regardless of its size. 445 446 447 Type: `string` 448 Default: `""` 449 450 ```yaml 451 # Examples 452 453 period: 1s 454 455 period: 1m 456 457 period: 500ms 458 ``` 459 460 ### `batching.check` 461 462 A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch. 463 464 465 Type: `string` 466 Default: `""` 467 468 ```yaml 469 # Examples 470 471 check: this.type == "end_of_transaction" 472 ``` 473 474 ### `batching.processors` 475 476 A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op. 477 478 479 Type: `array` 480 Default: `[]` 481 482 ```yaml 483 # Examples 484 485 processors: 486 - archive: 487 format: lines 488 489 processors: 490 - archive: 491 format: json_array 492 493 processors: 494 - merge_json: {} 495 ``` 496 497