github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/processors/sql.md (about) 1 --- 2 title: sql 3 type: processor 4 status: stable 5 categories: ["Integration"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/processor/sql.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 19 Runs an SQL prepared query against a target database for each message and, for 20 queries that return rows, replaces it with the result according to a 21 [codec](#result-codecs). 22 23 24 <Tabs defaultValue="common" values={[ 25 { label: 'Common', value: 'common', }, 26 { label: 'Advanced', value: 'advanced', }, 27 ]}> 28 29 <TabItem value="common"> 30 31 ```yaml 32 # Common config fields, showing default values 33 label: "" 34 sql: 35 driver: mysql 36 data_source_name: "" 37 query: "" 38 args_mapping: "" 39 result_codec: none 40 ``` 41 42 </TabItem> 43 <TabItem value="advanced"> 44 45 ```yaml 46 # All config fields, showing default values 47 label: "" 48 sql: 49 driver: mysql 50 data_source_name: "" 51 query: "" 52 unsafe_dynamic_query: false 53 args_mapping: "" 54 result_codec: none 55 ``` 56 57 </TabItem> 58 </Tabs> 59 60 ## Alternatives 61 62 For basic inserts or select queries use use either the [`sql_insert`](/docs/components/processors/sql_insert) or the [`sql_select`](/docs/components/processors/sql_select) processor. 63 64 For more complex queries use the [`sql_raw`](/docs/components/processors/sql_raw) processor. 65 66 ## Examples 67 68 <Tabs defaultValue="Table Insert (MySQL)" values={[ 69 { label: 'Table Insert (MySQL)', value: 'Table Insert (MySQL)', }, 70 { label: 'Table Query (PostgreSQL)', value: 'Table Query (PostgreSQL)', }, 71 ]}> 72 73 <TabItem value="Table Insert (MySQL)"> 74 75 76 The following example inserts rows into the table footable with the columns foo, 77 bar and baz populated with values extracted from messages: 78 79 ```yaml 80 pipeline: 81 processors: 82 - sql: 83 driver: mysql 84 data_source_name: foouser:foopassword@tcp(localhost:3306)/foodb 85 query: "INSERT INTO footable (foo, bar, baz) VALUES (?, ?, ?);" 86 args_mapping: '[ document.foo, document.bar, meta("kafka_topic") ]' 87 ``` 88 89 </TabItem> 90 <TabItem value="Table Query (PostgreSQL)"> 91 92 93 Here we query a database for columns of footable that share a `user_id` 94 with the message `user.id`. The `result_codec` is set to 95 `json_array` and a [`branch` processor](/docs/components/processors/branch) 96 is used in order to insert the resulting array into the original message at the 97 path `foo_rows`: 98 99 ```yaml 100 pipeline: 101 processors: 102 - branch: 103 processors: 104 - sql: 105 driver: postgres 106 result_codec: json_array 107 data_source_name: postgres://foouser:foopass@localhost:5432/testdb?sslmode=disable 108 query: "SELECT * FROM footable WHERE user_id = $1;" 109 args_mapping: '[ this.user.id ]' 110 result_map: 'root.foo_rows = this' 111 ``` 112 113 </TabItem> 114 </Tabs> 115 116 ## Fields 117 118 ### `driver` 119 120 A database [driver](#drivers) to use. 121 122 123 Type: `string` 124 Default: `"mysql"` 125 Options: `mysql`, `postgres`, `clickhouse`, `mssql`. 126 127 ### `data_source_name` 128 129 A Data Source Name to identify the target database. 130 131 132 Type: `string` 133 Default: `""` 134 135 ```yaml 136 # Examples 137 138 data_source_name: tcp://host1:9000?username=user&password=qwerty&database=clicks&read_timeout=10&write_timeout=20&alt_hosts=host2:9000,host3:9000 139 140 data_source_name: foouser:foopassword@tcp(localhost:3306)/foodb 141 142 data_source_name: postgres://foouser:foopass@localhost:5432/foodb?sslmode=disable 143 ``` 144 145 ### `query` 146 147 The query to run against the database. 148 149 150 Type: `string` 151 Default: `""` 152 153 ```yaml 154 # Examples 155 156 query: INSERT INTO footable (foo, bar, baz) VALUES (?, ?, ?); 157 ``` 158 159 ### `unsafe_dynamic_query` 160 161 Whether to enable dynamic queries that support interpolation functions. WARNING: This feature opens up the possibility of SQL injection attacks and is considered unsafe. 162 163 164 Type: `bool` 165 Default: `false` 166 167 ### `args_mapping` 168 169 A [Bloblang mapping](/docs/guides/bloblang/about) that produces the arguments for the query. The mapping must return an array containing the number of arguments in the query. 170 171 172 Type: `string` 173 Default: `""` 174 Requires version 3.47.0 or newer 175 176 ```yaml 177 # Examples 178 179 args_mapping: '[ this.foo, this.bar.not_empty().catch(null), meta("baz") ]' 180 181 args_mapping: root = [ uuid_v4() ].merge(this.document.args) 182 ``` 183 184 ### `result_codec` 185 186 A [codec](#result-codecs) to determine how resulting rows are converted into messages. 187 188 189 Type: `string` 190 Default: `"none"` 191 Options: `none`, `json_array`. 192 193 ## Result Codecs 194 195 When a query returns rows they are serialised according to a chosen codec, and 196 the message contents are replaced with the serialised result. 197 198 ### `none` 199 200 The result of the query is ignored and the message remains unchanged. If your 201 query does not return rows then this is the appropriate codec. 202 203 ### `json_array` 204 205 The resulting rows are serialised into an array of JSON objects, where each 206 object represents a row, where the key is the column name and the value is that 207 columns value in the row. 208