code.vegaprotocol.io/vega@v0.79.0/datanode/ratelimit/README.md (about)

     1  # Rate Limiting
     2  
     3  To prevent abuse of the APIs provided by datanode, we provide optional tooling to limit the rate
     4  of API requests.
     5  
     6  Currently the rate limiting is simply applied on a per-remote-ip address basis, though we may implement
     7  more sophisticated methods in the future.
     8  
     9  The rate limiting mechanism is based on a [token bucket](https://en.wikipedia.org/wiki/Token_bucket) algorithm.
    10  
    11  The idea is that:
    12  
    13  - Each IP address which connects to datanode is assigned a `bucket` of `tokens`
    14  - That bucket has a maximum capacity and is initially full of `tokens`
    15  - Each API request costs one token, which is removed from the bucket when the call is made
    16  - Datanode adds some number of tokens each seconds (the `rate` of the limiter) to the bucket _up to it's maximum capacity_
    17  
    18  On average over time, this enforces the average rate of API requests to not exceed `rate` requests/second. However it allows temporary periods of more intensive use; the maximum rate being to use the entire capacity of the bucket within one second. For this reason the capacity of the bucket is called the `burst`.
    19  
    20  There is an [IETF RFC](https://datatracker.ietf.org/doc/html/draft-ietf-httpapi-ratelimit-headers) for clients to get information about the rate limiting status. We implement this by sending the following headers in each API response (success or failure)
    21  
    22  - `RateLimit-Limit` The maximum request limit within the time window (1s).
    23  - `RateLimit-Reset` The rate-limiter time window duration in seconds (always 1s).
    24  - `RateLimit-Remaining` The remaining tokens.
    25  
    26  Upon rejection, the following HTTP response headers are available to users:
    27  
    28  - `X-Rate-Limit-Limit` The maximum request limit.
    29  - `X-Rate-Limit-Duration` The rate-limiter duration.
    30  - `X-Rate-Limit-Request-Forwarded-For` The rejected request X-Forwarded-For.
    31  - `X-Rate-Limit-Request-Remote-Addr` The rejected request RemoteAddr.
    32  
    33  If a client continues to make requests despite having no tokens available, the response will be
    34   - HTTP `429` `StatusTooManyRequests` for HTTP APIs
    35   - gRPC `14` Unavailable for the gRPC API.
    36  
    37  **Each unsuccessful response will deduct a token from a separate bucket** with the same refill rate and capacity as the before.
    38  
    39  **Exhausting the supply of tokens in this second bucket will result in the client's IP address being banned for a period of time.**
    40  
    41  If the user is banned the reponse will be 
    42   - HTTP 403 Forbidden for HTTP APIs
    43   - gRPC 14 Unavailable for the gRPC API.
    44  
    45  ## GraphQL, gRPC, and REST
    46  
    47  You can currently configure `GraphQL` and `GRPC` rate limiting separately. `REST` inherits and shares the limits of the GRPC API.
    48  
    49  There is a mechanism in places so that the `GRPC` `API` calls generated inside a `GraphQL` call are not rate limited a second time.
    50  
    51  ## Configuration
    52  
    53  For example in datanode's `config.toml` the GRPC API rate limiting is configured
    54  ```
    55  [API]
    56    [API.RateLimit]
    57      Enabled = true                 # Set to false to disable rate limiting
    58      TrustedProxies = ["127.0.0.1"] # List of trusted proxies used when handling X-Forwarded-For headers
    59      Rate = 10.0                    # Refill rate of token bucket per second i.e. limit of average request rate
    60      Burst = 50                     # Size of token bucket; maximum number of requests in short time window
    61      TTL = "1h0m0s"                 # Time after which inactive token buckets are reset
    62      BanFor = "10m0s"               # If IP continues to make requests after passing rate limit threshold,
    63                                     # ban for this duration. Setting to 0 seconds prevents banning.
    64  ```
    65  
    66  That configuration will apply to gRPC and REST. GraphQL is configured separately in
    67  ```
    68  [Gateway]
    69    [Gateway.RateLimits]
    70      Enabled = true
    71      TrustedProxies = ["127.0.0.1"]
    72      Rate = 10.0
    73      Burst = 50
    74      TTL = "1h0m0s"
    75      BanFor = "10m0s"
    76  ```
    77  
    78  ## WebSocket streams
    79  
    80  WebSocket connections use a different rate limiting mechanism. They are rate limited by a maximum allowed number of subscriptions per IP address. The default maximum is set to 250 connections. This can be changed in the following section of the data node configuration:
    81  
    82  ```
    83  [API]
    84    Level = "Info"
    85    ...
    86    MaxSubscriptionPerClient = 250
    87  
    88  ```
    89  
    90  ## Trusted Proxies
    91  
    92  When rate limiting is enabled, it's recommended to use trusted proxies. This ensures the IP used by the rate limiter has been verified by the trusted proxy. If no proxies (trusted or otherwise) are found in the `XFF` header, the peer IP is used for rate-limiting.