github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/gcp_pubsub.md (about) 1 --- 2 title: gcp_pubsub 3 type: output 4 status: stable 5 categories: ["Services","GCP"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/output/gcp_pubsub.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 19 Sends messages to a GCP Cloud Pub/Sub topic. [Metadata](/docs/configuration/metadata) from messages are sent as attributes. 20 21 22 <Tabs defaultValue="common" values={[ 23 { label: 'Common', value: 'common', }, 24 { label: 'Advanced', value: 'advanced', }, 25 ]}> 26 27 <TabItem value="common"> 28 29 ```yaml 30 # Common config fields, showing default values 31 output: 32 label: "" 33 gcp_pubsub: 34 project: "" 35 topic: "" 36 max_in_flight: 1 37 metadata: 38 exclude_prefixes: [] 39 ``` 40 41 </TabItem> 42 <TabItem value="advanced"> 43 44 ```yaml 45 # All config fields, showing default values 46 output: 47 label: "" 48 gcp_pubsub: 49 project: "" 50 topic: "" 51 max_in_flight: 1 52 publish_timeout: 60s 53 ordering_key: "" 54 metadata: 55 exclude_prefixes: [] 56 ``` 57 58 </TabItem> 59 </Tabs> 60 61 For information on how to set up credentials check out [this guide](https://cloud.google.com/docs/authentication/production). 62 63 ### Troubleshooting 64 65 If you're consistently seeing `Failed to send message to gcp_pubsub: context deadline exceeded` error logs without any further information it is possible that you are encountering https://github.com/Jeffail/benthos/issues/1042, which occurs when metadata values contain characters that are not valid utf-8. This can frequently occur when consuming from Kafka as the key metadata field may be populated with an arbitrary binary value, but this issue is not exclusive to Kafka. 66 67 If you are blocked by this issue then a work around is to delete either the specific problematic keys: 68 69 ```yaml 70 pipeline: 71 processors: 72 - bloblang: | 73 meta kafka_key = deleted() 74 ``` 75 76 Or delete all keys with: 77 78 ```yaml 79 pipeline: 80 processors: 81 - bloblang: meta = deleted() 82 ``` 83 84 ## Performance 85 86 This output benefits from sending multiple messages in flight in parallel for 87 improved performance. You can tune the max number of in flight messages with the 88 field `max_in_flight`. 89 90 ## Fields 91 92 ### `project` 93 94 The project ID of the topic to publish to. 95 96 97 Type: `string` 98 Default: `""` 99 100 ### `topic` 101 102 The topic to publish to. 103 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 104 105 106 Type: `string` 107 Default: `""` 108 109 ### `max_in_flight` 110 111 The maximum number of messages to have in flight at a given time. Increase this to improve throughput. 112 113 114 Type: `int` 115 Default: `1` 116 117 ### `publish_timeout` 118 119 The maximum length of time to wait before abandoning a publish attempt for a message. 120 121 122 Type: `string` 123 Default: `"60s"` 124 125 ```yaml 126 # Examples 127 128 publish_timeout: 10s 129 130 publish_timeout: 5m 131 132 publish_timeout: 60m 133 ``` 134 135 ### `ordering_key` 136 137 The ordering key to use for publishing messages. 138 This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries). 139 140 141 Type: `string` 142 Default: `""` 143 144 ### `metadata` 145 146 Specify criteria for which metadata values are sent as attributes. 147 148 149 Type: `object` 150 151 ### `metadata.exclude_prefixes` 152 153 Provide a list of explicit metadata key prefixes to be excluded when adding metadata to sent messages. 154 155 156 Type: `array` 157 Default: `[]` 158 159