github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/sql_raw.md (about)

     1  ---
     2  title: sql_raw
     3  type: output
     4  status: stable
     5  categories: ["Services"]
     6  ---
     7  
     8  <!--
     9       THIS FILE IS AUTOGENERATED!
    10  
    11       To make changes please edit the contents of:
    12       lib/output/sql_raw.go
    13  -->
    14  
    15  import Tabs from '@theme/Tabs';
    16  import TabItem from '@theme/TabItem';
    17  
    18  Executes an arbitrary SQL query for each message.
    19  
    20  Introduced in version 3.65.0.
    21  
    22  
    23  <Tabs defaultValue="common" values={[
    24    { label: 'Common', value: 'common', },
    25    { label: 'Advanced', value: 'advanced', },
    26  ]}>
    27  
    28  <TabItem value="common">
    29  
    30  ```yaml
    31  # Common config fields, showing default values
    32  output:
    33    label: ""
    34    sql_raw:
    35      driver: ""
    36      dsn: ""
    37      query: ""
    38      args_mapping: ""
    39      max_in_flight: 64
    40      batching:
    41        count: 0
    42        byte_size: 0
    43        period: ""
    44        check: ""
    45  ```
    46  
    47  </TabItem>
    48  <TabItem value="advanced">
    49  
    50  ```yaml
    51  # All config fields, showing default values
    52  output:
    53    label: ""
    54    sql_raw:
    55      driver: ""
    56      dsn: ""
    57      query: ""
    58      args_mapping: ""
    59      max_in_flight: 64
    60      batching:
    61        count: 0
    62        byte_size: 0
    63        period: ""
    64        check: ""
    65        processors: []
    66  ```
    67  
    68  </TabItem>
    69  </Tabs>
    70  
    71  ## Examples
    72  
    73  <Tabs defaultValue="Table Insert (MySQL)" values={[
    74  { label: 'Table Insert (MySQL)', value: 'Table Insert (MySQL)', },
    75  ]}>
    76  
    77  <TabItem value="Table Insert (MySQL)">
    78  
    79  
    80  Here we insert rows into a database by populating the columns id, name and topic with values extracted from messages and metadata:
    81  
    82  ```yaml
    83  output:
    84    sql_raw:
    85      driver: mysql
    86      dsn: foouser:foopassword@tcp(localhost:3306)/foodb
    87      query: "INSERT INTO footable (id, name, topic) VALUES (?, ?, ?);"
    88      args_mapping: |
    89        root = [
    90          this.user.id,
    91          this.user.name,
    92          meta("kafka_topic"),
    93        ]
    94  ```
    95  
    96  </TabItem>
    97  </Tabs>
    98  
    99  ## Fields
   100  
   101  ### `driver`
   102  
   103  A database [driver](#drivers) to use.
   104  
   105  
   106  Type: `string`  
   107  Options: `mysql`, `postgres`, `clickhouse`, `mssql`.
   108  
   109  ### `dsn`
   110  
   111  A Data Source Name to identify the target database.
   112  
   113  #### Drivers
   114  
   115  The following is a list of supported drivers and their respective DSN formats:
   116  
   117  | Driver | Data Source Name Format |
   118  |---|---|
   119  | `clickhouse` | [`tcp://[netloc][:port][?param1=value1&...&paramN=valueN]`](https://github.com/ClickHouse/clickhouse-go#dsn)
   120  | `mysql` | `[username[:password]@][protocol[(address)]]/dbname[?param1=value1&...&paramN=valueN]` |
   121  | `postgres` | `postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]` |
   122  | `mssql` | `sqlserver://[user[:password]@][netloc][:port][?database=dbname&param1=value1&...]` |
   123  
   124  Please note that the `postgres` driver enforces SSL by default, you
   125  can override this with the parameter `sslmode=disable` if required.
   126  
   127  
   128  Type: `string`  
   129  
   130  ```yaml
   131  # Examples
   132  
   133  dsn: tcp://host1:9000?username=user&password=qwerty&database=clicks&read_timeout=10&write_timeout=20&alt_hosts=host2:9000,host3:9000
   134  
   135  dsn: foouser:foopassword@tcp(localhost:3306)/foodb
   136  
   137  dsn: postgres://foouser:foopass@localhost:5432/foodb?sslmode=disable
   138  ```
   139  
   140  ### `query`
   141  
   142  The query to execute.
   143  
   144  
   145  Type: `string`  
   146  
   147  ```yaml
   148  # Examples
   149  
   150  query: INSERT INTO footable (foo, bar, baz) VALUES (?, ?, ?);
   151  ```
   152  
   153  ### `args_mapping`
   154  
   155  An optional [Bloblang mapping](/docs/guides/bloblang/about) which should evaluate to an array of values matching in size to the number of placeholder arguments in the field `query`.
   156  
   157  
   158  Type: `string`  
   159  
   160  ```yaml
   161  # Examples
   162  
   163  args_mapping: root = [ this.cat.meow, this.doc.woofs[0] ]
   164  
   165  args_mapping: root = [ meta("user.id") ]
   166  ```
   167  
   168  ### `max_in_flight`
   169  
   170  The maximum number of inserts to run in parallel.
   171  
   172  
   173  Type: `int`  
   174  Default: `64`  
   175  
   176  ### `batching`
   177  
   178  Allows you to configure a [batching policy](/docs/configuration/batching).
   179  
   180  
   181  Type: `object`  
   182  
   183  ```yaml
   184  # Examples
   185  
   186  batching:
   187    byte_size: 5000
   188    count: 0
   189    period: 1s
   190  
   191  batching:
   192    count: 10
   193    period: 1s
   194  
   195  batching:
   196    check: this.contains("END BATCH")
   197    count: 0
   198    period: 1m
   199  ```
   200  
   201  ### `batching.count`
   202  
   203  A number of messages at which the batch should be flushed. If `0` disables count based batching.
   204  
   205  
   206  Type: `int`  
   207  Default: `0`  
   208  
   209  ### `batching.byte_size`
   210  
   211  An amount of bytes at which the batch should be flushed. If `0` disables size based batching.
   212  
   213  
   214  Type: `int`  
   215  Default: `0`  
   216  
   217  ### `batching.period`
   218  
   219  A period in which an incomplete batch should be flushed regardless of its size.
   220  
   221  
   222  Type: `string`  
   223  Default: `""`  
   224  
   225  ```yaml
   226  # Examples
   227  
   228  period: 1s
   229  
   230  period: 1m
   231  
   232  period: 500ms
   233  ```
   234  
   235  ### `batching.check`
   236  
   237  A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch.
   238  
   239  
   240  Type: `string`  
   241  Default: `""`  
   242  
   243  ```yaml
   244  # Examples
   245  
   246  check: this.type == "end_of_transaction"
   247  ```
   248  
   249  ### `batching.processors`
   250  
   251  A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op.
   252  
   253  
   254  Type: `array`  
   255  
   256  ```yaml
   257  # Examples
   258  
   259  processors:
   260    - archive:
   261        format: lines
   262  
   263  processors:
   264    - archive:
   265        format: json_array
   266  
   267  processors:
   268    - merge_json: {}
   269  ```
   270  
   271