github.com/Jeffail/benthos/v3@v3.65.0/website/docs/configuration/dynamic_inputs_and_outputs.md (about)

     1  ---
     2  title: Dynamic Inputs and Outputs
     3  ---
     4  
     5  It is possible to have sets of inputs and outputs in Benthos that can be added,
     6  updated and removed during runtime with the [dynamic fan in][dynamic_inputs] and
     7  [dynamic fan out][dynamic_outputs] types.
     8  
     9  Dynamic inputs and outputs are each identified by unique string labels, which
    10  are specified when adding them either in configuration or via the HTTP API. The
    11  labels are useful when querying which types are active.
    12  
    13  ## API
    14  
    15  The API for dynamic types (both inputs and outputs) is a collection of HTTP REST
    16  endpoints:
    17  
    18  ### `/inputs`
    19  
    20  Returns a JSON object that maps input labels to an object containing details
    21  about the input, including uptime and configuration. If the input has terminated
    22  naturally the uptime will be set to `stopped`.
    23  
    24  ``` json
    25  {
    26    "<string, input_label>": {
    27      "uptime": "<string>",
    28      "config": <object>
    29    },
    30    ...
    31  }
    32  ```
    33  
    34  ### `/inputs/{input_label}`
    35  
    36  GET returns the configuration of the input idenfified by `input_label`.
    37  
    38  POST sets the input `input_label` to the body of the request parsed as a JSON
    39  configuration. If the input label already exists the previous input is first
    40  stopped and removed.
    41  
    42  DELETE stops and removes the input identified by `input_label`.
    43  
    44  ### `/outputs`
    45  
    46  Returns a JSON object that maps output labels to an object containing details
    47  about the output, including uptime and configuration. If the output has
    48  terminated naturally the uptime will be set to `stopped`.
    49  
    50  ``` json
    51  {
    52    "<string, output_label>": {
    53      "uptime": "<string>",
    54      "config": <object>
    55    },
    56    ...
    57  }
    58  ```
    59  
    60  ### `/outputs/{output_label}`
    61  
    62  GET returns the configuration of the output idenfified by `output_label`.
    63  
    64  POST sets the output `output_label` to the body of the request parsed as a JSON
    65  configuration. If the output label already exists the previous output is first
    66  stopped and removed.
    67  
    68  DELETE stops and removes the output identified by `output_label`.
    69  
    70  A custom prefix can be set for these endpoints in configuration.
    71  
    72  ## Applications
    73  
    74  Dynamic types are useful when a platforms data streams might need to change
    75  regularly and automatically. It is also useful for triggering batches of
    76  platform data, e.g. a cron job can be created to send hourly curl requests that
    77  adds a dynamic input to read a file of sample data:
    78  
    79  ``` sh
    80  curl http://localhost:4195/inputs/read_sample -d @- << EOF
    81  {
    82  	"file": {
    83  		"path": "/tmp/line_delim_sample_data.txt"
    84  	}
    85  }
    86  EOF
    87  ```
    88  
    89  Some inputs have a finite lifetime, e.g. `s3` without an SQS queue configured
    90  will close once the whole bucket has been read. When a dynamic types lifetime
    91  ends the `uptime` field of an input listing will be set to `stopped`. You can
    92  use this to write tools that trigger new inputs (to move onto the next bucket,
    93  for example).
    94  
    95  [dynamic_inputs]: /docs/components/inputs/dynamic
    96  [dynamic_outputs]: /docs/components/outputs/dynamic