github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/outputs/gcp_pubsub.md (about)

     1  ---
     2  title: gcp_pubsub
     3  type: output
     4  status: stable
     5  categories: ["Services","GCP"]
     6  ---
     7  
     8  <!--
     9       THIS FILE IS AUTOGENERATED!
    10  
    11       To make changes please edit the contents of:
    12       lib/output/gcp_pubsub.go
    13  -->
    14  
    15  import Tabs from '@theme/Tabs';
    16  import TabItem from '@theme/TabItem';
    17  
    18  
    19  Sends messages to a GCP Cloud Pub/Sub topic. [Metadata](/docs/configuration/metadata) from messages are sent as attributes.
    20  
    21  
    22  <Tabs defaultValue="common" values={[
    23    { label: 'Common', value: 'common', },
    24    { label: 'Advanced', value: 'advanced', },
    25  ]}>
    26  
    27  <TabItem value="common">
    28  
    29  ```yaml
    30  # Common config fields, showing default values
    31  output:
    32    label: ""
    33    gcp_pubsub:
    34      project: ""
    35      topic: ""
    36      max_in_flight: 1
    37      metadata:
    38        exclude_prefixes: []
    39  ```
    40  
    41  </TabItem>
    42  <TabItem value="advanced">
    43  
    44  ```yaml
    45  # All config fields, showing default values
    46  output:
    47    label: ""
    48    gcp_pubsub:
    49      project: ""
    50      topic: ""
    51      max_in_flight: 1
    52      publish_timeout: 60s
    53      ordering_key: ""
    54      metadata:
    55        exclude_prefixes: []
    56  ```
    57  
    58  </TabItem>
    59  </Tabs>
    60  
    61  For information on how to set up credentials check out [this guide](https://cloud.google.com/docs/authentication/production).
    62  
    63  ### Troubleshooting
    64  
    65  If you're consistently seeing `Failed to send message to gcp_pubsub: context deadline exceeded` error logs without any further information it is possible that you are encountering https://github.com/Jeffail/benthos/issues/1042, which occurs when metadata values contain characters that are not valid utf-8. This can frequently occur when consuming from Kafka as the key metadata field may be populated with an arbitrary binary value, but this issue is not exclusive to Kafka.
    66  
    67  If you are blocked by this issue then a work around is to delete either the specific problematic keys:
    68  
    69  ```yaml
    70  pipeline:
    71    processors:
    72      - bloblang: |
    73          meta kafka_key = deleted()
    74  ```
    75  
    76  Or delete all keys with:
    77  
    78  ```yaml
    79  pipeline:
    80    processors:
    81      - bloblang: meta = deleted()
    82  ```
    83  
    84  ## Performance
    85  
    86  This output benefits from sending multiple messages in flight in parallel for
    87  improved performance. You can tune the max number of in flight messages with the
    88  field `max_in_flight`.
    89  
    90  ## Fields
    91  
    92  ### `project`
    93  
    94  The project ID of the topic to publish to.
    95  
    96  
    97  Type: `string`  
    98  Default: `""`  
    99  
   100  ### `topic`
   101  
   102  The topic to publish to.
   103  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   104  
   105  
   106  Type: `string`  
   107  Default: `""`  
   108  
   109  ### `max_in_flight`
   110  
   111  The maximum number of messages to have in flight at a given time. Increase this to improve throughput.
   112  
   113  
   114  Type: `int`  
   115  Default: `1`  
   116  
   117  ### `publish_timeout`
   118  
   119  The maximum length of time to wait before abandoning a publish attempt for a message.
   120  
   121  
   122  Type: `string`  
   123  Default: `"60s"`  
   124  
   125  ```yaml
   126  # Examples
   127  
   128  publish_timeout: 10s
   129  
   130  publish_timeout: 5m
   131  
   132  publish_timeout: 60m
   133  ```
   134  
   135  ### `ordering_key`
   136  
   137  The ordering key to use for publishing messages.
   138  This field supports [interpolation functions](/docs/configuration/interpolation#bloblang-queries).
   139  
   140  
   141  Type: `string`  
   142  Default: `""`  
   143  
   144  ### `metadata`
   145  
   146  Specify criteria for which metadata values are sent as attributes.
   147  
   148  
   149  Type: `object`  
   150  
   151  ### `metadata.exclude_prefixes`
   152  
   153  Provide a list of explicit metadata key prefixes to be excluded when adding metadata to sent messages.
   154  
   155  
   156  Type: `array`  
   157  Default: `[]`  
   158  
   159