github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/processors/protobuf.md

---
title: protobuf
type: processor
status: beta
categories: ["Parsing"]
---

<!--
     THIS FILE IS AUTOGENERATED!

     To make changes please edit the contents of:
     lib/processor/protobuf.go
-->

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

:::caution BETA
This component is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with the component is found.
:::

Performs conversions to or from a protobuf message. This processor uses
reflection, meaning conversions can be made directly from the target .proto
files.


<Tabs defaultValue="common" values={[
  { label: 'Common', value: 'common', },
  { label: 'Advanced', value: 'advanced', },
]}>

<TabItem value="common">

```yaml
# Common config fields, showing default values
label: ""
protobuf:
  operator: to_json
  message: ""
  import_paths: []
```

</TabItem>
<TabItem value="advanced">

```yaml
# All config fields, showing default values
label: ""
protobuf:
  operator: to_json
  message: ""
  import_paths: []
  parts: []
```

</TabItem>
</Tabs>

The main functionality of this processor is to map to and from JSON documents,
you can read more about JSON mapping of protobuf messages here:
[https://developers.google.com/protocol-buffers/docs/proto3#json](https://developers.google.com/protocol-buffers/docs/proto3#json)

Using reflection for processing protobuf messages in this way is less performant
than generating and using native code. Therefore when performance is critical it
is recommended that you use Benthos plugins instead for processing protobuf
messages natively, you can find an example of Benthos plugins at
[https://github.com/benthosdev/benthos-plugin-example](https://github.com/benthosdev/benthos-plugin-example)

## Operators

### `to_json`

Converts protobuf messages into a generic JSON structure. This makes it easier
to manipulate the contents of the document within Benthos.

### `from_json`

Attempts to create a target protobuf message from a generic JSON structure.

## Fields

### `operator`

The [operator](#operators) to execute


Type: `string`  
Default: `"to_json"`  
Options: `to_json`, `from_json`.

### `message`

The fully qualified name of the protobuf message to convert to/from.


Type: `string`  
Default: `""`  

### `import_paths`

A list of directories containing .proto files, including all definitions required for parsing the target message. If left empty the current directory is used. Each directory listed will be walked with all found .proto files imported.


Type: `array`  
Default: `[]`  

### `parts`

An optional array of message indexes of a batch that the processor should apply to.
If left empty all messages are processed. This field is only applicable when
batching messages [at the input level](/docs/configuration/batching).

Indexes can be negative, and if so the part will be selected from the end
counting backwards starting from -1.


Type: `array`  
Default: `[]`  

## Examples

<Tabs defaultValue="JSON to Protobuf" values={[
{ label: 'JSON to Protobuf', value: 'JSON to Protobuf', },
{ label: 'Protobuf to JSON', value: 'Protobuf to JSON', },
]}>

<TabItem value="JSON to Protobuf">


If we have the following protobuf definition within a directory called `testing/schema`:

```protobuf
syntax = "proto3";
package testing;

import "google/protobuf/timestamp.proto";

message Person {
  string first_name = 1;
  string last_name = 2;
  string full_name = 3;
  int32 age = 4;
  int32 id = 5; // Unique ID number for this person.
  string email = 6;

  google.protobuf.Timestamp last_updated = 7;
}
```

And a stream of JSON documents of the form:

```json
{
	"firstName": "caleb",
	"lastName": "quaye",
	"email": "caleb@myspace.com"
}
```

We can convert the documents into protobuf messages with the following config:

```yaml
pipeline:
  processors:
    - protobuf:
        operator: from_json
        message: testing.Person
        import_paths: [ testing/schema ]
```

</TabItem>
<TabItem value="Protobuf to JSON">


If we have the following protobuf definition within a directory called `testing/schema`:

```protobuf
syntax = "proto3";
package testing;

import "google/protobuf/timestamp.proto";

message Person {
  string first_name = 1;
  string last_name = 2;
  string full_name = 3;
  int32 age = 4;
  int32 id = 5; // Unique ID number for this person.
  string email = 6;

  google.protobuf.Timestamp last_updated = 7;
}
```

And a stream of protobuf messages of the type `Person`, we could convert them into JSON documents of the format:

```json
{
	"firstName": "caleb",
	"lastName": "quaye",
	"email": "caleb@myspace.com"
}
```

With the following config:

```yaml
pipeline:
  processors:
    - protobuf:
        operator: to_json
        message: testing.Person
        import_paths: [ testing/schema ]
```

</TabItem>
</Tabs>