github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/processors/cache.md (about) 1 --- 2 title: cache 3 type: processor 4 status: stable 5 categories: ["Integration"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/processor/cache.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 19 Performs operations against a [cache resource](/docs/components/caches/about) for each message, allowing you to store or retrieve data within message payloads. 20 21 22 <Tabs defaultValue="common" values={[ 23 { label: 'Common', value: 'common', }, 24 { label: 'Advanced', value: 'advanced', }, 25 ]}> 26 27 <TabItem value="common"> 28 29 ```yaml 30 # Common config fields, showing default values 31 label: "" 32 cache: 33 resource: "" 34 operator: set 35 key: "" 36 value: "" 37 ``` 38 39 </TabItem> 40 <TabItem value="advanced"> 41 42 ```yaml 43 # All config fields, showing default values 44 label: "" 45 cache: 46 resource: "" 47 operator: set 48 key: "" 49 value: "" 50 ttl: "" 51 parts: [] 52 ``` 53 54 </TabItem> 55 </Tabs> 56 57 This processor will interpolate functions within the `key` and `value` fields individually for each message. This allows you to specify dynamic keys and values based on the contents of the message payloads and metadata. You can find a list of functions [here](/docs/configuration/interpolation#bloblang-queries). 58 59 ## Examples 60 61 <Tabs defaultValue="Deduplication" values={[ 62 { label: 'Deduplication', value: 'Deduplication', }, 63 { label: 'Hydration', value: 'Hydration', }, 64 ]}> 65 66 <TabItem value="Deduplication"> 67 68 69 Deduplication can be done using the add operator with a key extracted from the 70 message payload, since it fails when a key already exists we can remove the 71 duplicates using a 72 [`bloblang` processor](/docs/components/processors/bloblang): 73 74 ```yaml 75 pipeline: 76 processors: 77 - cache: 78 resource: foocache 79 operator: add 80 key: '${! json("message.id") }' 81 value: "storeme" 82 - bloblang: root = if errored() { deleted() } 83 84 cache_resources: 85 - label: foocache 86 redis: 87 url: tcp://TODO:6379 88 ``` 89 90 </TabItem> 91 <TabItem value="Hydration"> 92 93 94 It's possible to enrich payloads with content previously stored in a cache by 95 using the [`branch`](/docs/components/processors/branch) processor: 96 97 ```yaml 98 pipeline: 99 processors: 100 - branch: 101 processors: 102 - cache: 103 resource: foocache 104 operator: get 105 key: '${! json("message.document_id") }' 106 result_map: 'root.message.document = this' 107 108 # NOTE: If the data stored in the cache is not valid JSON then use 109 # something like this instead: 110 # result_map: 'root.message.document = content().string()' 111 112 cache_resources: 113 - label: foocache 114 memcached: 115 addresses: [ "TODO:11211" ] 116 ``` 117 118 </TabItem> 119 </Tabs> 120 121 ## Fields 122 123 ### `resource` 124 125 The [`cache` resource](/docs/components/caches/about) to target with this processor. 126 127 128 Type: `string` 129 Default: `""` 130 131 ### `operator` 132 133 The [operation](#operators) to perform with the cache. 134 135 136 Type: `string` 137 Default: `"set"` 138 Options: `set`, `add`, `get`, `delete`. 139 140 ### `key` 141 142 A key to use with the cache. 143 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 144 145 146 Type: `string` 147 Default: `""` 148 149 ### `value` 150 151 A value to use with the cache (when applicable). 152 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 153 154 155 Type: `string` 156 Default: `""` 157 158 ### `ttl` 159 160 The TTL of each individual item as a duration string. After this period an item will be eligible for removal during the next compaction. Not all caches support per-key TTLs, and those that do not will fall back to their generally configured TTL setting. 161 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 162 163 164 Type: `string` 165 Default: `""` 166 Requires version 3.33.0 or newer 167 168 ```yaml 169 # Examples 170 171 ttl: 60s 172 173 ttl: 5m 174 175 ttl: 36h 176 ``` 177 178 ### `parts` 179 180 An optional array of message indexes of a batch that the processor should apply to. 181 If left empty all messages are processed. This field is only applicable when 182 batching messages [at the input level](/docs/configuration/batching). 183 184 Indexes can be negative, and if so the part will be selected from the end 185 counting backwards starting from -1. 186 187 188 Type: `array` 189 Default: `[]` 190 191 ## Operators 192 193 ### `set` 194 195 Set a key in the cache to a value. If the key already exists the contents are 196 overridden. 197 198 ### `add` 199 200 Set a key in the cache to a value. If the key already exists the action fails 201 with a 'key already exists' error, which can be detected with 202 [processor error handling](/docs/configuration/error_handling). 203 204 ### `get` 205 206 Retrieve the contents of a cached key and replace the original message payload 207 with the result. If the key does not exist the action fails with an error, which 208 can be detected with [processor error handling](/docs/configuration/error_handling). 209 210 ### `delete` 211 212 Delete a key and its contents from the cache. If the key does not exist the 213 action is a no-op and will not fail with an error. 214