github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/sql_raw.md (about) 1 --- 2 title: sql_raw 3 type: output 4 status: stable 5 categories: ["Services"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/output/sql_raw.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 Executes an arbitrary SQL query for each message. 19 20 Introduced in version 3.65.0. 21 22 23 <Tabs defaultValue="common" values={[ 24 { label: 'Common', value: 'common', }, 25 { label: 'Advanced', value: 'advanced', }, 26 ]}> 27 28 <TabItem value="common"> 29 30 ```yaml 31 # Common config fields, showing default values 32 output: 33 label: "" 34 sql_raw: 35 driver: "" 36 dsn: "" 37 query: "" 38 args_mapping: "" 39 max_in_flight: 64 40 batching: 41 count: 0 42 byte_size: 0 43 period: "" 44 check: "" 45 ``` 46 47 </TabItem> 48 <TabItem value="advanced"> 49 50 ```yaml 51 # All config fields, showing default values 52 output: 53 label: "" 54 sql_raw: 55 driver: "" 56 dsn: "" 57 query: "" 58 args_mapping: "" 59 max_in_flight: 64 60 batching: 61 count: 0 62 byte_size: 0 63 period: "" 64 check: "" 65 processors: [] 66 ``` 67 68 </TabItem> 69 </Tabs> 70 71 ## Examples 72 73 <Tabs defaultValue="Table Insert (MySQL)" values={[ 74 { label: 'Table Insert (MySQL)', value: 'Table Insert (MySQL)', }, 75 ]}> 76 77 <TabItem value="Table Insert (MySQL)"> 78 79 80 Here we insert rows into a database by populating the columns id, name and topic with values extracted from messages and metadata: 81 82 ```yaml 83 output: 84 sql_raw: 85 driver: mysql 86 dsn: foouser:foopassword@tcp(localhost:3306)/foodb 87 query: "INSERT INTO footable (id, name, topic) VALUES (?, ?, ?);" 88 args_mapping: | 89 root = [ 90 this.user.id, 91 this.user.name, 92 meta("kafka_topic"), 93 ] 94 ``` 95 96 </TabItem> 97 </Tabs> 98 99 ## Fields 100 101 ### `driver` 102 103 A database [driver](#drivers) to use. 104 105 106 Type: `string` 107 Options: `mysql`, `postgres`, `clickhouse`, `mssql`. 108 109 ### `dsn` 110 111 A Data Source Name to identify the target database. 112 113 #### Drivers 114 115 The following is a list of supported drivers and their respective DSN formats: 116 117 | Driver | Data Source Name Format | 118 |---|---| 119 | `clickhouse` | [`tcp://[netloc][:port][?param1=value1&...¶mN=valueN]`](https://github.com/ClickHouse/clickhouse-go#dsn) 120 | `mysql` | `[username[:password]@][protocol[(address)]]/dbname[?param1=value1&...¶mN=valueN]` | 121 | `postgres` | `postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]` | 122 | `mssql` | `sqlserver://[user[:password]@][netloc][:port][?database=dbname¶m1=value1&...]` | 123 124 Please note that the `postgres` driver enforces SSL by default, you 125 can override this with the parameter `sslmode=disable` if required. 126 127 128 Type: `string` 129 130 ```yaml 131 # Examples 132 133 dsn: tcp://host1:9000?username=user&password=qwerty&database=clicks&read_timeout=10&write_timeout=20&alt_hosts=host2:9000,host3:9000 134 135 dsn: foouser:foopassword@tcp(localhost:3306)/foodb 136 137 dsn: postgres://foouser:foopass@localhost:5432/foodb?sslmode=disable 138 ``` 139 140 ### `query` 141 142 The query to execute. 143 144 145 Type: `string` 146 147 ```yaml 148 # Examples 149 150 query: INSERT INTO footable (foo, bar, baz) VALUES (?, ?, ?); 151 ``` 152 153 ### `args_mapping` 154 155 An optional [Bloblang mapping](/docs/guides/bloblang/about) which should evaluate to an array of values matching in size to the number of placeholder arguments in the field `query`. 156 157 158 Type: `string` 159 160 ```yaml 161 # Examples 162 163 args_mapping: root = [ this.cat.meow, this.doc.woofs[0] ] 164 165 args_mapping: root = [ meta("user.id") ] 166 ``` 167 168 ### `max_in_flight` 169 170 The maximum number of inserts to run in parallel. 171 172 173 Type: `int` 174 Default: `64` 175 176 ### `batching` 177 178 Allows you to configure a [batching policy](/docs/configuration/batching). 179 180 181 Type: `object` 182 183 ```yaml 184 # Examples 185 186 batching: 187 byte_size: 5000 188 count: 0 189 period: 1s 190 191 batching: 192 count: 10 193 period: 1s 194 195 batching: 196 check: this.contains("END BATCH") 197 count: 0 198 period: 1m 199 ``` 200 201 ### `batching.count` 202 203 A number of messages at which the batch should be flushed. If `0` disables count based batching. 204 205 206 Type: `int` 207 Default: `0` 208 209 ### `batching.byte_size` 210 211 An amount of bytes at which the batch should be flushed. If `0` disables size based batching. 212 213 214 Type: `int` 215 Default: `0` 216 217 ### `batching.period` 218 219 A period in which an incomplete batch should be flushed regardless of its size. 220 221 222 Type: `string` 223 Default: `""` 224 225 ```yaml 226 # Examples 227 228 period: 1s 229 230 period: 1m 231 232 period: 500ms 233 ``` 234 235 ### `batching.check` 236 237 A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch. 238 239 240 Type: `string` 241 Default: `""` 242 243 ```yaml 244 # Examples 245 246 check: this.type == "end_of_transaction" 247 ``` 248 249 ### `batching.processors` 250 251 A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op. 252 253 254 Type: `array` 255 256 ```yaml 257 # Examples 258 259 processors: 260 - archive: 261 format: lines 262 263 processors: 264 - archive: 265 format: json_array 266 267 processors: 268 - merge_json: {} 269 ``` 270 271