github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/inputs/redis_streams.md (about) 1 --- 2 title: redis_streams 3 type: input 4 status: stable 5 categories: ["Services"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/input/redis_streams.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 19 Pulls messages from Redis (v5.0+) streams with the XREADGROUP command. The 20 `client_id` should be unique for each consumer of a group. 21 22 23 <Tabs defaultValue="common" values={[ 24 { label: 'Common', value: 'common', }, 25 { label: 'Advanced', value: 'advanced', }, 26 ]}> 27 28 <TabItem value="common"> 29 30 ```yaml 31 # Common config fields, showing default values 32 input: 33 label: "" 34 redis_streams: 35 url: tcp://localhost:6379 36 body_key: body 37 streams: 38 - benthos_stream 39 limit: 10 40 client_id: benthos_consumer 41 consumer_group: benthos_group 42 ``` 43 44 </TabItem> 45 <TabItem value="advanced"> 46 47 ```yaml 48 # All config fields, showing default values 49 input: 50 label: "" 51 redis_streams: 52 url: tcp://localhost:6379 53 kind: simple 54 master: "" 55 tls: 56 enabled: false 57 skip_cert_verify: false 58 enable_renegotiation: false 59 root_cas: "" 60 root_cas_file: "" 61 client_certs: [] 62 body_key: body 63 streams: 64 - benthos_stream 65 limit: 10 66 client_id: benthos_consumer 67 consumer_group: benthos_group 68 create_streams: true 69 start_from_oldest: true 70 commit_period: 1s 71 timeout: 1s 72 ``` 73 74 </TabItem> 75 </Tabs> 76 77 Redis stream entries are key/value pairs, as such it is necessary to specify the 78 key that contains the body of the message. All other keys/value pairs are saved 79 as metadata fields. 80 81 ## Fields 82 83 ### `url` 84 85 The URL of the target Redis server. Database is optional and is supplied as the URL path. The scheme `tcp` is equivalent to `redis`. 86 87 88 Type: `string` 89 Default: `"tcp://localhost:6379"` 90 91 ```yaml 92 # Examples 93 94 url: :6397 95 96 url: localhost:6397 97 98 url: redis://localhost:6379 99 100 url: redis://:foopassword@redisplace:6379 101 102 url: redis://localhost:6379/1 103 104 url: redis://localhost:6379/1,redis://localhost:6380/1 105 ``` 106 107 ### `kind` 108 109 Specifies a simple, cluster-aware, or failover-aware redis client. 110 111 112 Type: `string` 113 Default: `"simple"` 114 115 ```yaml 116 # Examples 117 118 kind: simple 119 120 kind: cluster 121 122 kind: failover 123 ``` 124 125 ### `master` 126 127 Name of the redis master when `kind` is `failover` 128 129 130 Type: `string` 131 Default: `""` 132 133 ```yaml 134 # Examples 135 136 master: mymaster 137 ``` 138 139 ### `tls` 140 141 Custom TLS settings can be used to override system defaults. 142 143 **Troubleshooting** 144 145 Some cloud hosted instances of Redis (such as Azure Cache) might need some hand holding in order to establish stable connections. Unfortunately, it is often the case that TLS issues will manifest as generic error messages such as "i/o timeout". If you're using TLS and are seeing connectivity problems consider setting `enable_renegotiation` to `true`, and ensuring that the server supports at least TLS version 1.2. 146 147 148 Type: `object` 149 150 ### `tls.enabled` 151 152 Whether custom TLS settings are enabled. 153 154 155 Type: `bool` 156 Default: `false` 157 158 ### `tls.skip_cert_verify` 159 160 Whether to skip server side certificate verification. 161 162 163 Type: `bool` 164 Default: `false` 165 166 ### `tls.enable_renegotiation` 167 168 Whether to allow the remote server to repeatedly request renegotiation. Enable this option if you're seeing the error message `local error: tls: no renegotiation`. 169 170 171 Type: `bool` 172 Default: `false` 173 Requires version 3.45.0 or newer 174 175 ### `tls.root_cas` 176 177 An optional root certificate authority to use. This is a string, representing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate. 178 179 180 Type: `string` 181 Default: `""` 182 183 ```yaml 184 # Examples 185 186 root_cas: |- 187 -----BEGIN CERTIFICATE----- 188 ... 189 -----END CERTIFICATE----- 190 ``` 191 192 ### `tls.root_cas_file` 193 194 An optional path of a root certificate authority file to use. This is a file, often with a .pem extension, containing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate. 195 196 197 Type: `string` 198 Default: `""` 199 200 ```yaml 201 # Examples 202 203 root_cas_file: ./root_cas.pem 204 ``` 205 206 ### `tls.client_certs` 207 208 A list of client certificates to use. For each certificate either the fields `cert` and `key`, or `cert_file` and `key_file` should be specified, but not both. 209 210 211 Type: `array` 212 Default: `[]` 213 214 ```yaml 215 # Examples 216 217 client_certs: 218 - cert: foo 219 key: bar 220 221 client_certs: 222 - cert_file: ./example.pem 223 key_file: ./example.key 224 ``` 225 226 ### `tls.client_certs[].cert` 227 228 A plain text certificate to use. 229 230 231 Type: `string` 232 Default: `""` 233 234 ### `tls.client_certs[].key` 235 236 A plain text certificate key to use. 237 238 239 Type: `string` 240 Default: `""` 241 242 ### `tls.client_certs[].cert_file` 243 244 The path to a certificate to use. 245 246 247 Type: `string` 248 Default: `""` 249 250 ### `tls.client_certs[].key_file` 251 252 The path of a certificate key to use. 253 254 255 Type: `string` 256 Default: `""` 257 258 ### `body_key` 259 260 The field key to extract the raw message from. All other keys will be stored in the message as metadata. 261 262 263 Type: `string` 264 Default: `"body"` 265 266 ### `streams` 267 268 A list of streams to consume from. 269 270 271 Type: `array` 272 Default: `["benthos_stream"]` 273 274 ### `limit` 275 276 The maximum number of messages to consume from a single request. 277 278 279 Type: `int` 280 Default: `10` 281 282 ### `client_id` 283 284 An identifier for the client connection. 285 286 287 Type: `string` 288 Default: `"benthos_consumer"` 289 290 ### `consumer_group` 291 292 An identifier for the consumer group of the stream. 293 294 295 Type: `string` 296 Default: `"benthos_group"` 297 298 ### `create_streams` 299 300 Create subscribed streams if they do not exist (MKSTREAM option). 301 302 303 Type: `bool` 304 Default: `true` 305 306 ### `start_from_oldest` 307 308 If an offset is not found for a stream, determines whether to consume from the oldest available offset, otherwise messages are consumed from the latest offset. 309 310 311 Type: `bool` 312 Default: `true` 313 314 ### `commit_period` 315 316 The period of time between each commit of the current offset. Offsets are always committed during shutdown. 317 318 319 Type: `string` 320 Default: `"1s"` 321 322 ### `timeout` 323 324 The length of time to poll for new messages before reattempting. 325 326 327 Type: `string` 328 Default: `"1s"` 329 330