github.com/Jeffail/benthos/v3@v3.65.0/website/docs/configuration/resources.md (about) 1 --- 2 title: Resources 3 --- 4 5 Resources are components within Benthos that are declared with a unique label and can be referenced any number of times within a configuration. Only one instance of each named resource is created, but it is safe to use it in multiple places as they can be shared without consequence. 6 7 Some components such as caches and rate limits can _only_ be created as a resource. However, for components where it's optional there are a few reasons why it might be advantageous to do so. 8 9 ```yaml 10 input: 11 resource: foo 12 13 pipeline: 14 processors: 15 - resource: bar 16 - cache: 17 operator: set 18 resource: baz 19 key: ${! json("id") } 20 value: ${! content() } 21 22 output: 23 resource: buz 24 25 input_resources: 26 - label: foo 27 file: 28 paths: [ ./in.txt ] 29 30 processor_resources: 31 - label: bar 32 bloblang: 'root = content.lowercase()' 33 34 cache_resources: 35 - label: baz 36 memory: 37 ttl: 300 38 39 output_resources: 40 - label: buz 41 file: 42 path: ./out.txt 43 ``` 44 45 ## Reusability 46 47 Sometimes it's necessary to use a rather large component multiple times. Instead of copy/pasting the configuration or using YAML anchors you can define your component as a resource. 48 49 In the following example we want to make an HTTP request with our payloads. Occasionally the payload might get rejected due to garbage within its contents, and so we catch these rejected requests, attempt to "cleanse" the contents and try to make the same HTTP request again. Since the HTTP request component is quite large (and likely to change over time) we make sure to avoid duplicating it by defining it as a resource `get_foo`: 50 51 ```yaml 52 pipeline: 53 processors: 54 - resource: get_foo 55 - catch: 56 - bloblang: | 57 root = this 58 root.content = this.content.strip_html() 59 - resource: get_foo 60 61 processor_resources: 62 - label: get_foo 63 http: 64 url: http://example.com/foo 65 verb: POST 66 headers: 67 SomeThing: "set-to-this" 68 SomeThingElse: "set-to-something-else" 69 ``` 70 71 ## Feature Toggling 72 73 ### With Environment Variables 74 75 There are two ways of using resources for feature toggling, the first is to define your feature components with unique names and then apply the old switcheroo with environment variables to select the one you wish to execute: 76 77 ```yaml 78 pipeline: 79 processors: 80 - resource: ${FEATURE_REQUEST} 81 82 processor_resources: 83 - label: get_foo 84 http: 85 url: http://example.com/foo 86 verb: POST 87 headers: 88 SomeThing: "set-to-this" 89 SomeThingElse: "set-to-something-else" 90 91 - label: get_bar 92 http: 93 url: http://example.com/bar 94 verb: PUT 95 headers: 96 Desires: "are-empty" 97 ``` 98 99 Then when you execute Benthos use the environment variable to choose your resource: `FEATURE_REQUEST=get_foo benthos -c ./your_config.yaml`. 100 101 ### With Imports 102 103 However, Benthos allows you to import resources from separate files with the cli flag `-r` or `-resources`, which can be a useful way to switch out resources with common names based on your chosen environment. For example, with a main configuration file `config.yaml`: 104 105 ```yaml 106 pipeline: 107 processors: 108 - resource: get_foo 109 ``` 110 111 And then two resource files, one stored at the path `./staging/request.yaml`: 112 113 ```yaml 114 processor_resources: 115 - label: get_foo 116 http: 117 url: http://example.com/foo 118 verb: POST 119 headers: 120 SomeThing: "set-to-this" 121 SomeThingElse: "set-to-something-else" 122 ``` 123 124 And another stored at the path `./production/request.yaml`: 125 126 ```yaml 127 processor_resources: 128 - label: get_foo 129 http: 130 url: http://example.com/bar 131 verb: PUT 132 headers: 133 Desires: "are-empty" 134 ``` 135 136 We can select our chosen resource by changing which file we import, either running: 137 138 ```sh 139 benthos -r ./staging/request.yaml -c ./config.yaml 140 ``` 141 142 Or: 143 144 ```sh 145 benthos -r ./production/request.yaml -c ./config.yaml 146 ``` 147 148 These flags also support wildcards, which allows you to import an entire directory of resource files like `benthos -r "./staging/*.yaml" -c ./config.yaml`.