github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/processors/group_by.md (about) 1 --- 2 title: group_by 3 type: processor 4 status: stable 5 categories: ["Composition"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/processor/group_by.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 19 Splits a [batch of messages](/docs/configuration/batching/) into N batches, where each resulting batch contains a group of messages determined by a [Bloblang query](/docs/guides/bloblang/about/). 20 21 ```yaml 22 # Config fields, showing default values 23 label: "" 24 group_by: [] 25 ``` 26 27 Once the groups are established a list of processors are applied to their respective grouped batch, which can be used to label the batch as per their grouping. Messages that do not pass the check of any specified group are placed in their own group. 28 29 The functionality of this processor depends on being applied across messages 30 that are batched. You can find out more about batching [in this doc](/docs/configuration/batching). 31 32 ## Fields 33 34 ### `[].check` 35 36 A [Bloblang query](/docs/guides/bloblang/about/) that should return a boolean value indicating whether a message belongs to a given group. 37 38 39 Type: `string` 40 Default: `""` 41 42 ```yaml 43 # Examples 44 45 check: this.type == "foo" 46 47 check: this.contents.urls.contains("https://benthos.dev/") 48 49 check: "true" 50 ``` 51 52 ### `[].processors` 53 54 A list of [processors](/docs/components/processors/about/) to execute on the newly formed group. 55 56 57 Type: `array` 58 Default: `[]` 59 60 ## Examples 61 62 <Tabs defaultValue="Grouped Processing" values={[ 63 { label: 'Grouped Processing', value: 'Grouped Processing', }, 64 ]}> 65 66 <TabItem value="Grouped Processing"> 67 68 Imagine we have a batch of messages that we wish to split into a group of foos and everything else, which should be sent to different output destinations based on those groupings. We also need to send the foos as a tar gzip archive. For this purpose we can use the `group_by` processor with a [`switch`](/docs/components/outputs/switch) output: 69 70 ```yaml 71 pipeline: 72 processors: 73 - group_by: 74 - check: content().contains("this is a foo") 75 processors: 76 - archive: 77 format: tar 78 - compress: 79 algorithm: gzip 80 - bloblang: 'meta grouping = "foo"' 81 82 output: 83 switch: 84 cases: 85 - check: meta("grouping") == "foo" 86 output: 87 gcp_pubsub: 88 project: foo_prod 89 topic: only_the_foos 90 - output: 91 gcp_pubsub: 92 project: somewhere_else 93 topic: no_foos_here 94 ``` 95 96 </TabItem> 97 </Tabs> 98 99