github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/sql_insert.md (about)

     1  ---
     2  title: sql_insert
     3  type: output
     4  status: stable
     5  categories: ["Services"]
     6  ---
     7  
     8  <!--
     9       THIS FILE IS AUTOGENERATED!
    10  
    11       To make changes please edit the contents of:
    12       lib/output/sql_insert.go
    13  -->
    14  
    15  import Tabs from '@theme/Tabs';
    16  import TabItem from '@theme/TabItem';
    17  
    18  Inserts a row into an SQL database for each message.
    19  
    20  Introduced in version 3.59.0.
    21  
    22  
    23  <Tabs defaultValue="common" values={[
    24    { label: 'Common', value: 'common', },
    25    { label: 'Advanced', value: 'advanced', },
    26  ]}>
    27  
    28  <TabItem value="common">
    29  
    30  ```yaml
    31  # Common config fields, showing default values
    32  output:
    33    label: ""
    34    sql_insert:
    35      driver: ""
    36      dsn: ""
    37      table: ""
    38      columns: []
    39      args_mapping: ""
    40      max_in_flight: 64
    41      batching:
    42        count: 0
    43        byte_size: 0
    44        period: ""
    45        check: ""
    46  ```
    47  
    48  </TabItem>
    49  <TabItem value="advanced">
    50  
    51  ```yaml
    52  # All config fields, showing default values
    53  output:
    54    label: ""
    55    sql_insert:
    56      driver: ""
    57      dsn: ""
    58      table: ""
    59      columns: []
    60      args_mapping: ""
    61      prefix: ""
    62      suffix: ""
    63      max_in_flight: 64
    64      batching:
    65        count: 0
    66        byte_size: 0
    67        period: ""
    68        check: ""
    69        processors: []
    70  ```
    71  
    72  </TabItem>
    73  </Tabs>
    74  
    75  ## Examples
    76  
    77  <Tabs defaultValue="Table Insert (MySQL)" values={[
    78  { label: 'Table Insert (MySQL)', value: 'Table Insert (MySQL)', },
    79  ]}>
    80  
    81  <TabItem value="Table Insert (MySQL)">
    82  
    83  
    84  Here we insert rows into a database by populating the columns id, name and topic with values extracted from messages and metadata:
    85  
    86  ```yaml
    87  output:
    88    sql_insert:
    89      driver: mysql
    90      dsn: foouser:foopassword@tcp(localhost:3306)/foodb
    91      table: footable
    92      columns: [ id, name, topic ]
    93      args_mapping: |
    94        root = [
    95          this.user.id,
    96          this.user.name,
    97          meta("kafka_topic"),
    98        ]
    99  ```
   100  
   101  </TabItem>
   102  </Tabs>
   103  
   104  ## Fields
   105  
   106  ### `driver`
   107  
   108  A database [driver](#drivers) to use.
   109  
   110  
   111  Type: `string`  
   112  Options: `mysql`, `postgres`, `clickhouse`, `mssql`.
   113  
   114  ### `dsn`
   115  
   116  A Data Source Name to identify the target database.
   117  
   118  #### Drivers
   119  
   120  The following is a list of supported drivers and their respective DSN formats:
   121  
   122  | Driver | Data Source Name Format |
   123  |---|---|
   124  | `clickhouse` | [`tcp://[netloc][:port][?param1=value1&...&paramN=valueN]`](https://github.com/ClickHouse/clickhouse-go#dsn)
   125  | `mysql` | `[username[:password]@][protocol[(address)]]/dbname[?param1=value1&...&paramN=valueN]` |
   126  | `postgres` | `postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]` |
   127  | `mssql` | `sqlserver://[user[:password]@][netloc][:port][?database=dbname&param1=value1&...]` |
   128  
   129  Please note that the `postgres` driver enforces SSL by default, you
   130  can override this with the parameter `sslmode=disable` if required.
   131  
   132  
   133  Type: `string`  
   134  
   135  ```yaml
   136  # Examples
   137  
   138  dsn: tcp://host1:9000?username=user&password=qwerty&database=clicks&read_timeout=10&write_timeout=20&alt_hosts=host2:9000,host3:9000
   139  
   140  dsn: foouser:foopassword@tcp(localhost:3306)/foodb
   141  
   142  dsn: postgres://foouser:foopass@localhost:5432/foodb?sslmode=disable
   143  ```
   144  
   145  ### `table`
   146  
   147  The table to insert to.
   148  
   149  
   150  Type: `string`  
   151  
   152  ```yaml
   153  # Examples
   154  
   155  table: foo
   156  ```
   157  
   158  ### `columns`
   159  
   160  A list of columns to insert.
   161  
   162  
   163  Type: `array`  
   164  
   165  ```yaml
   166  # Examples
   167  
   168  columns:
   169    - foo
   170    - bar
   171    - baz
   172  ```
   173  
   174  ### `args_mapping`
   175  
   176  A [Bloblang mapping](/docs/guides/bloblang/about) which should evaluate to an array of values matching in size to the number of columns specified.
   177  
   178  
   179  Type: `string`  
   180  
   181  ```yaml
   182  # Examples
   183  
   184  args_mapping: root = [ this.cat.meow, this.doc.woofs[0] ]
   185  
   186  args_mapping: root = [ meta("user.id") ]
   187  ```
   188  
   189  ### `prefix`
   190  
   191  An optional prefix to prepend to the insert query (before INSERT).
   192  
   193  
   194  Type: `string`  
   195  
   196  ### `suffix`
   197  
   198  An optional suffix to append to the insert query.
   199  
   200  
   201  Type: `string`  
   202  
   203  ```yaml
   204  # Examples
   205  
   206  suffix: ON CONFLICT (name) DO NOTHING
   207  ```
   208  
   209  ### `max_in_flight`
   210  
   211  The maximum number of inserts to run in parallel.
   212  
   213  
   214  Type: `int`  
   215  Default: `64`  
   216  
   217  ### `batching`
   218  
   219  Allows you to configure a [batching policy](/docs/configuration/batching).
   220  
   221  
   222  Type: `object`  
   223  
   224  ```yaml
   225  # Examples
   226  
   227  batching:
   228    byte_size: 5000
   229    count: 0
   230    period: 1s
   231  
   232  batching:
   233    count: 10
   234    period: 1s
   235  
   236  batching:
   237    check: this.contains("END BATCH")
   238    count: 0
   239    period: 1m
   240  ```
   241  
   242  ### `batching.count`
   243  
   244  A number of messages at which the batch should be flushed. If `0` disables count based batching.
   245  
   246  
   247  Type: `int`  
   248  Default: `0`  
   249  
   250  ### `batching.byte_size`
   251  
   252  An amount of bytes at which the batch should be flushed. If `0` disables size based batching.
   253  
   254  
   255  Type: `int`  
   256  Default: `0`  
   257  
   258  ### `batching.period`
   259  
   260  A period in which an incomplete batch should be flushed regardless of its size.
   261  
   262  
   263  Type: `string`  
   264  Default: `""`  
   265  
   266  ```yaml
   267  # Examples
   268  
   269  period: 1s
   270  
   271  period: 1m
   272  
   273  period: 500ms
   274  ```
   275  
   276  ### `batching.check`
   277  
   278  A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message should end a batch.
   279  
   280  
   281  Type: `string`  
   282  Default: `""`  
   283  
   284  ```yaml
   285  # Examples
   286  
   287  check: this.type == "end_of_transaction"
   288  ```
   289  
   290  ### `batching.processors`
   291  
   292  A list of [processors](/docs/components/processors/about) to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op.
   293  
   294  
   295  Type: `array`  
   296  
   297  ```yaml
   298  # Examples
   299  
   300  processors:
   301    - archive:
   302        format: lines
   303  
   304  processors:
   305    - archive:
   306        format: json_array
   307  
   308  processors:
   309    - merge_json: {}
   310  ```
   311  
   312