github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/processors/sql.md (about)

     1  ---
     2  title: sql
     3  type: processor
     4  status: stable
     5  categories: ["Integration"]
     6  ---
     7  
     8  <!--
     9       THIS FILE IS AUTOGENERATED!
    10  
    11       To make changes please edit the contents of:
    12       lib/processor/sql.go
    13  -->
    14  
    15  import Tabs from '@theme/Tabs';
    16  import TabItem from '@theme/TabItem';
    17  
    18  
    19  Runs an SQL prepared query against a target database for each message and, for
    20  queries that return rows, replaces it with the result according to a
    21  [codec](#result-codecs).
    22  
    23  
    24  <Tabs defaultValue="common" values={[
    25    { label: 'Common', value: 'common', },
    26    { label: 'Advanced', value: 'advanced', },
    27  ]}>
    28  
    29  <TabItem value="common">
    30  
    31  ```yaml
    32  # Common config fields, showing default values
    33  label: ""
    34  sql:
    35    driver: mysql
    36    data_source_name: ""
    37    query: ""
    38    args_mapping: ""
    39    result_codec: none
    40  ```
    41  
    42  </TabItem>
    43  <TabItem value="advanced">
    44  
    45  ```yaml
    46  # All config fields, showing default values
    47  label: ""
    48  sql:
    49    driver: mysql
    50    data_source_name: ""
    51    query: ""
    52    unsafe_dynamic_query: false
    53    args_mapping: ""
    54    result_codec: none
    55  ```
    56  
    57  </TabItem>
    58  </Tabs>
    59  
    60  ## Alternatives
    61  
    62  For basic inserts or select queries use use either the [`sql_insert`](/docs/components/processors/sql_insert) or the [`sql_select`](/docs/components/processors/sql_select) processor.
    63  
    64  For more complex queries use the [`sql_raw`](/docs/components/processors/sql_raw) processor.
    65  
    66  ## Examples
    67  
    68  <Tabs defaultValue="Table Insert (MySQL)" values={[
    69  { label: 'Table Insert (MySQL)', value: 'Table Insert (MySQL)', },
    70  { label: 'Table Query (PostgreSQL)', value: 'Table Query (PostgreSQL)', },
    71  ]}>
    72  
    73  <TabItem value="Table Insert (MySQL)">
    74  
    75  
    76  The following example inserts rows into the table footable with the columns foo,
    77  bar and baz populated with values extracted from messages:
    78  
    79  ```yaml
    80  pipeline:
    81    processors:
    82      - sql:
    83          driver: mysql
    84          data_source_name: foouser:foopassword@tcp(localhost:3306)/foodb
    85          query: "INSERT INTO footable (foo, bar, baz) VALUES (?, ?, ?);"
    86          args_mapping: '[ document.foo, document.bar, meta("kafka_topic") ]'
    87  ```
    88  
    89  </TabItem>
    90  <TabItem value="Table Query (PostgreSQL)">
    91  
    92  
    93  Here we query a database for columns of footable that share a `user_id`
    94  with the message `user.id`. The `result_codec` is set to
    95  `json_array` and a [`branch` processor](/docs/components/processors/branch)
    96  is used in order to insert the resulting array into the original message at the
    97  path `foo_rows`:
    98  
    99  ```yaml
   100  pipeline:
   101    processors:
   102      - branch:
   103          processors:
   104            - sql:
   105                driver: postgres
   106                result_codec: json_array
   107                data_source_name: postgres://foouser:foopass@localhost:5432/testdb?sslmode=disable
   108                query: "SELECT * FROM footable WHERE user_id = $1;"
   109                args_mapping: '[ this.user.id ]'
   110          result_map: 'root.foo_rows = this'
   111  ```
   112  
   113  </TabItem>
   114  </Tabs>
   115  
   116  ## Fields
   117  
   118  ### `driver`
   119  
   120  A database [driver](#drivers) to use.
   121  
   122  
   123  Type: `string`  
   124  Default: `"mysql"`  
   125  Options: `mysql`, `postgres`, `clickhouse`, `mssql`.
   126  
   127  ### `data_source_name`
   128  
   129  A Data Source Name to identify the target database.
   130  
   131  
   132  Type: `string`  
   133  Default: `""`  
   134  
   135  ```yaml
   136  # Examples
   137  
   138  data_source_name: tcp://host1:9000?username=user&password=qwerty&database=clicks&read_timeout=10&write_timeout=20&alt_hosts=host2:9000,host3:9000
   139  
   140  data_source_name: foouser:foopassword@tcp(localhost:3306)/foodb
   141  
   142  data_source_name: postgres://foouser:foopass@localhost:5432/foodb?sslmode=disable
   143  ```
   144  
   145  ### `query`
   146  
   147  The query to run against the database.
   148  
   149  
   150  Type: `string`  
   151  Default: `""`  
   152  
   153  ```yaml
   154  # Examples
   155  
   156  query: INSERT INTO footable (foo, bar, baz) VALUES (?, ?, ?);
   157  ```
   158  
   159  ### `unsafe_dynamic_query`
   160  
   161  Whether to enable dynamic queries that support interpolation functions. WARNING: This feature opens up the possibility of SQL injection attacks and is considered unsafe.
   162  
   163  
   164  Type: `bool`  
   165  Default: `false`  
   166  
   167  ### `args_mapping`
   168  
   169  A [Bloblang mapping](/docs/guides/bloblang/about) that produces the arguments for the query. The mapping must return an array containing the number of arguments in the query.
   170  
   171  
   172  Type: `string`  
   173  Default: `""`  
   174  Requires version 3.47.0 or newer  
   175  
   176  ```yaml
   177  # Examples
   178  
   179  args_mapping: '[ this.foo, this.bar.not_empty().catch(null), meta("baz") ]'
   180  
   181  args_mapping: root = [ uuid_v4() ].merge(this.document.args)
   182  ```
   183  
   184  ### `result_codec`
   185  
   186  A [codec](#result-codecs) to determine how resulting rows are converted into messages.
   187  
   188  
   189  Type: `string`  
   190  Default: `"none"`  
   191  Options: `none`, `json_array`.
   192  
   193  ## Result Codecs
   194  
   195  When a query returns rows they are serialised according to a chosen codec, and
   196  the message contents are replaced with the serialised result.
   197  
   198  ### `none`
   199  
   200  The result of the query is ignored and the message remains unchanged. If your
   201  query does not return rows then this is the appropriate codec.
   202  
   203  ### `json_array`
   204  
   205  The resulting rows are serialised into an array of JSON objects, where each
   206  object represents a row, where the key is the column name and the value is that
   207  columns value in the row.
   208