github.com/Jeffail/benthos/v3@v3.65.0/website/docs/guides/performance_tuning.md (about)

     1  ---
     2  title: Performance Tuning
     3  ---
     4  
     5  ## Maximising IO Throughput
     6  
     7  This section outlines a few common throughput issues and ways in which they can be solved within Benthos.
     8  
     9  It is assumed here that your Benthos instance is performing only minor processing steps, and therefore has minimal reliance on your CPU resource. If this is not the case the following still applies to an extent, but you should also refer to [the next section regarding CPU utilisation](#maximising-cpu-utilisation).
    10  
    11  Firstly, before venturing into Benthos configurations, you should take an in-depth look at your sources and sinks. Benthos is generally much simpler architecturally than the inputs and outputs it supports. Spend some time understanding how to squeeze the most out of these services and it will make it easier (or unnecessary) to tune your Benthos configuration.
    12  
    13  ### Benthos Reads Too Slowly
    14  
    15  If Benthos isn't reading fast enough from your source it might not necessarily be due to a slow consumer. If the sink is slow this can cause back pressure that throttles the amount Benthos can read. Try consuming a test feed with the output replaced with `drop`. If you notice that the input consumption suddenly speeds up then the issue is likely with the output, in which case [try the next section](#benthos-writes-too-slowly).
    16  
    17  If the `drop` output pipe didn't help then take a quick look at the basic configuration fields for the input source type. Sometimes there are fields for setting a number of background prefetches or similar concepts that can increase your throughput. For example, increasing the value of `prefetch_count` for an AMQP consumer can greatly increase the rate at which it is consumed.
    18  
    19  Next, if your source supports multiple parallel consumers then you can try doing that within Benthos by using a [broker][broker-input]. For example, if you started with:
    20  
    21  ```yaml
    22  input:
    23    http_client:
    24      url: http://localhost:4195/get
    25      verb: GET
    26  ```
    27  
    28  You could change to:
    29  
    30  ```yaml
    31  input:
    32    broker:
    33      copies: 4
    34      inputs:
    35        - http_client:
    36            url: http://localhost:4195/get
    37            verb: GET
    38  ```
    39  
    40  Which would create the exact same consumer as before with four connections in total. Try increasing the number of copies to see how that affects the throughput. If your multiple consumers would require different configurations then set copies to `1` and write each consumer as a separate object in the `inputs` array.
    41  
    42  Read the [broker documentation][broker-input] for more tips on simplifying broker configs.
    43  
    44  If your source doesn't support multiple parallel consumers then unfortunately your options are more limited. A logical next step might be to look at your network/disk configuration to see if that's a potential cause of contention.
    45  
    46  ### Benthos Writes Too Slowly
    47  
    48  If you have an output sink that regularly places back pressure on your source there are a few solutions depending on the details of the issue.
    49  
    50  Firstly, you should check the config parameters of your output sink. There are often fields specifically for controlling the level of acknowledgement to expect before moving onto the next message, if these levels of guarantee are overkill you can disable them for greater throughput. For example, setting the `ack_replicas` field to `false` in the Kafka sink can have a high impact on throughput.
    51  
    52  If the config parameters for an output sink aren't enough then you can try the following:
    53  
    54  #### Increase in flight messages
    55  
    56  Most outputs have a field `max_in_flight` that allows you to specify how many messages can be in flight at the same time. Increasing this value can improve throughput significantly.
    57  
    58  #### Send messages in batches
    59  
    60  Most outputs will send data quicker when messages are batched, this is often done automatically in the background. However, for a few outputs your batches need to be configured. Read the [batching documentation][batching] for more guidance on how to tune message batches within Benthos.
    61  
    62  #### Increase the number of parallel output sinks
    63  
    64  If your output sink supports multiple parallel writers then it can greatly increase your throughput to have multiple connections configured.
    65  
    66  Increasing the number of parallel output sinks is similar to doing the same for input sources and is done using a [broker][broker-output]. The output broker type supports a few different routing patterns depending on your intention. In this case we want to maximize throughput so our best choice is a `greedy` pattern. For example, if you started with:
    67  
    68  ```yaml
    69  output:
    70    kafka:
    71      addresses:
    72        - localhost:9092
    73      topic: benthos_stream
    74  ```
    75  
    76  You could change to:
    77  
    78  ```yaml
    79  output:
    80    broker:
    81      pattern: greedy
    82      copies: 4
    83      outputs:
    84        - kafka:
    85            addresses:
    86              - localhost:9092
    87            topic: benthos_stream
    88  ```
    89  
    90  Which would create the exact same output writer as before with four copies in total. Try increasing the number of copies to see how that affects the throughput. If your multiple output writers would require different configurations (client ids, for example) then set copies to `1` and write each consumer as a separate object in the `outputs` array.
    91  
    92  Read the [broker documentation][broker-output] for more tips on simplifying broker configs.
    93  
    94  #### Level out input spikes with a buffer
    95  
    96  There are many reasons why an input source might have spikes or inconsistent throughput rates. It is possible that your output is capable of keeping up with
    97  the long term average flow of data, but fails to keep up when an intermittent spike occurs.
    98  
    99  In situations like these it is sometimes a better use of your hardware and resources to level out the flow of data rather than try and match the peak throughput. This would depend on the frequency and duration of the spikes as well as your latency requirements, and is therefore a matter of judgement.
   100  
   101  Leveling out the flow of data can be done within Benthos using a [buffer][buffers]. Buffers allow an input source to store a bounded amount of data temporarily, which a consumer can work through at its own pace. Buffers always have a fixed capacity, which when full will proceed to block the input just like a busy output would.
   102  
   103  Therefore, it's still important to have an output that can keep up with the flow of data, the difference that a buffer makes is that the output only needs to keep up with the _average_ flow of data versus the instantaneous flow of data.
   104  
   105  For example, if your input usually produces 10 msgs/s, but occasionally spikes to 100 msgs/s, and your output can handle up to 50 msgs/s, it might be possible to configure a buffer large enough to store spikes in their entirety. As long as the average flow of messages from the input remains below 50 msgs/s then your service should be able to continue indefinitely without ever blocking the input source.
   106  
   107  ## Maximising CPU Utilisation
   108  
   109  Some [processors][processors] within Benthos are relatively heavy on your CPU, and can potentially become the bottleneck of a service. In these circumstances it is worth configuring Benthos so that your processors are running on each available core of your machine without contention.
   110  
   111  An array of processors in any section of a Benthos config becomes a single logical pipeline of steps running on a single logical thread. The easiest way to create parallel processor threads is to configure them inside the [pipeline][pipeline] configuration block, where we can explicitly set any number of parallel processor threads independent of how many inputs or outputs we want to use.
   112  
   113  Please refer [to the documentation regarding pipelines][pipeline] for some examples.
   114  
   115  [pipeline]: /docs/configuration/processing_pipelines
   116  [batching]: /docs/configuration/batching
   117  [processors]: /docs/components/processors/about
   118  [buffers]: /docs/components/buffers/about
   119  [broker-input]: /docs/components/inputs/broker
   120  [broker-output]: /docs/components/outputs/broker