github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/rate_limits/about.md (about)

     1  ---
     2  title: Rate Limits
     3  sidebar_label: About
     4  ---
     5  
     6  A rate limit is a strategy for limiting the usage of a shared resource across parallel components in a Benthos instance, or potentially across multiple instances. They are configured as a resource:
     7  
     8  ```yaml
     9  rate_limit_resources:
    10    - label: foobar
    11      local:
    12        count: 500
    13        interval: 1s
    14  ```
    15  
    16  And most components that hit external services have a field `rate_limit` for specifying a rate limit resource to use, identified by the `label` field. For example, if we wanted to use our `foobar` rate limit with an [`http_client`][input.http_client] input it would look like this:
    17  
    18  ```yaml
    19  input:
    20    http_client:
    21      url: TODO
    22      verb: GET
    23      rate_limit: foobar
    24  ```
    25  
    26  By using a rate limit in this way we can guarantee that our input will only poll our HTTP source at the rate of 500 requests per second.
    27  
    28  Some components don't have a `rate_limit` field but we might still wish to throttle them by a rate limit, in which case we can use the [`rate_limit` processor][processor.rate_limit] that applies back pressure to a processing pipeline when the limit is reached. For example, if we wished to limit the consumption of lines of a [`csv` file input][inputs.csv] to a specified rate limit we can do that with the following:
    29  
    30  ```yaml
    31  input:
    32    csv:
    33      paths:
    34        - ./foo.csv
    35    processors:
    36      - rate_limit:
    37          resource: foobar
    38  ```
    39  
    40  You can find out more about resources [in this document.][config.resources]
    41  
    42  import ComponentSelect from '@theme/ComponentSelect';
    43  
    44  <ComponentSelect type="rate_limits" singular="rate limit"></ComponentSelect>
    45  
    46  [processor.rate_limit]: /docs/components/processors/rate_limit
    47  [input.csv]: /docs/components/input/csv
    48  [input.http_client]: /docs/components/inputs/http_client
    49  [config.resources]: /docs/configuration/resources