github.com/Jeffail/benthos/v3@v3.65.0/website/docs/guides/streams_mode/streams_api.md (about) 1 --- 2 title: Streams API 3 --- 4 5 When Benthos is run in `streams` mode it will open up an HTTP REST API for creating and managing independent streams of data instead of creating a single stream. 6 7 Each stream has its own input, buffer, pipeline and output sections which contains an isolated stream of data with its own lifetime. A stream config cannot include [resources][resources], and instead these should be created and modified using the `/resources/{type}/{id}` endpoint. 8 9 A walkthrough on using this API [can be found here][streams-api-walkthrough]. 10 11 ## API 12 13 ### GET `/ready` 14 15 Returns a 200 OK response if all active streams are connected to their respective inputs and outputs at the time of the request. Otherwise, a 503 response is returned along with a message naming the faulty stream. 16 17 If zero streams are active this endpoint still returns a 200 OK response. 18 19 ### GET `/streams` 20 21 Returns a map of existing streams by their unique identifiers to an object showing their status and uptime. 22 23 #### Response 200 24 25 ```json 26 { 27 "<string, stream id>": { 28 "active": "<bool, whether the stream is running>", 29 "uptime": "<float, uptime in seconds>", 30 "uptime_str": "<string, human readable string of uptime>" 31 } 32 } 33 ``` 34 35 ### POST `/streams` 36 37 Sets the entire collection of streams to the body of the request. Streams that exist but aren't within the request body are *removed*, streams that exist already and are in the request body are updated, other streams within the request body are created. 38 39 ```json 40 { 41 "<string, stream id>": "<object, a standard Benthos stream configuration>" 42 } 43 ``` 44 45 #### Response 200 46 47 The streams were updated successfully. 48 49 #### Response 400 50 51 A configuration was invalid, or has linting errors. If linting errors were detected then a JSON response is provided of the form: 52 53 ```json 54 { 55 "linting_errors": [ 56 "<a description of the error" 57 ] 58 } 59 ``` 60 61 If you wish for the streams API to proceed with configurations that contain linting errors then you can override this check by setting the URL param `chilled` to `true`, e.g. `/streams?chilled=true`. 62 63 ### POST `/streams/{id}` 64 65 Create a new stream identified by `id` by posting a body containing the stream configuration in either JSON or YAML format. The configuration should be a standard Benthos configuration containing the sections `input`, `buffer`, `pipeline` and `output`. 66 67 #### Request Body Example 68 69 URL: `/streams/foo` 70 71 ```yaml 72 input: 73 file: 74 paths: [ /tmp/input.ndjson ] 75 pipeline: 76 processors: 77 - bloblang: root = content().uppercase() 78 output: 79 file: 80 path: /tmp/output.ndjson 81 ``` 82 83 #### Response 200 84 85 The stream was created successfully. 86 87 #### Response 400 88 89 The configuration was invalid, or has linting errors. If linting errors were detected then a JSON response is provided of the form: 90 91 ```json 92 { 93 "linting_errors": [ 94 "<a description of the error" 95 ] 96 } 97 ``` 98 99 If you wish for the streams API to proceed with configurations that contain linting errors then you can override this check by setting the URL param `chilled` to `true`, e.g. `/streams/foo?chilled=true`. 100 101 ### GET `/streams/{id}` 102 103 Read the details of an existing stream identified by `id`. 104 105 #### Response 200 106 107 ```json 108 { 109 "active": "<bool, whether the stream is running>", 110 "uptime": "<float, uptime in seconds>", 111 "uptime_str": "<string, human readable string of uptime>", 112 "config": "<object, the configuration of the stream>" 113 } 114 ``` 115 116 ### PUT `/streams/{id}` 117 118 Update an existing stream identified by `id` by posting a body containing the new stream configuration in either JSON or YAML format. The configuration should be a standard Benthos configuration containing the sections `input`, `buffer`, `pipeline` and `output`. 119 120 The previous stream will be shut down before and a new stream will take its place. 121 122 #### Response 200 123 124 The stream was updated successfully. 125 126 #### Response 400 127 128 The configuration was invalid, or has linting errors. If linting errors were detected then a JSON response is provided of the form: 129 130 ```json 131 { 132 "linting_errors": [ 133 "<a description of the error" 134 ] 135 } 136 ``` 137 138 If you wish for the streams API to proceed with configurations that contain linting errors then you can override this check by setting the URL param `chilled` to `true`, e.g. `/streams/foo?chilled=true`. 139 140 ### PATCH `/streams/{id}` 141 142 Update an existing stream identified by `id` by posting a body containing only changes to be made to the existing configuration. The existing configuration will be patched with the new fields and the stream restarted with the result. 143 144 #### Response 200 145 146 The stream was patched successfully. 147 148 ### DELETE `/streams/{id}` 149 150 Attempt to shut down and remove a stream identified by `id`. 151 152 #### Response 200 153 154 The stream was found, shut down and removed successfully. 155 156 ### GET `/streams/{id}/stats` 157 158 Read the metrics of an existing stream as a hierarchical JSON object. 159 160 #### Response 200 161 162 The stream was found. 163 164 ### POST `/resources/{type}/{id}` 165 166 Add or modify a resource component configuration of a given `type` identified by a unique `id`. The configuration must be in JSON or YAML format and must only contain configuration fields for the component. 167 168 Valid component types are `cache`, `input`, `output`, `processor` and `rate_limit`. 169 170 #### Request Body Example 171 172 URL: `/resources/cache/foo` 173 174 ```yml 175 redis: 176 url: http://localhost:6379 177 expiration: 1h 178 ``` 179 180 #### Response 200 181 182 The resource was created successfully. 183 184 #### Response 400 185 186 The configuration was invalid, or has linting errors. If linting errors were detected then a JSON response is provided of the form: 187 188 ```json 189 { 190 "linting_errors": [ 191 "<a description of the error" 192 ] 193 } 194 ``` 195 196 If you wish for the streams API to proceed with configurations that contain linting errors then you can override this check by setting the URL param `chilled` to `true`, e.g. `/resources/cache/foo?chilled=true`. 197 198 [streams-api-walkthrough]: /docs/guides/streams_mode/using_rest_api 199 [resources]: /docs/configuration/resources