github.com/ulule/limiter/v3@v3.11.3-0.20230613131926-4cb9c1da4633/README.md (about)

     1  # Limiter
     2  
     3  [![Documentation][godoc-img]][godoc-url]
     4  ![License][license-img]
     5  [![Build Status][circle-img]][circle-url]
     6  [![Go Report Card][goreport-img]][goreport-url]
     7  
     8  _Dead simple rate limit middleware for Go._
     9  
    10  - Simple API
    11  - "Store" approach for backend
    12  - Redis support (but not tied too)
    13  - Middlewares: HTTP, [FastHTTP][6] and [Gin][4]
    14  
    15  ## Installation
    16  
    17  Using [Go Modules](https://github.com/golang/go/wiki/Modules)
    18  
    19  ```bash
    20  $ go get github.com/ulule/limiter/v3@v3.11.2
    21  ```
    22  
    23  ## Usage
    24  
    25  In five steps:
    26  
    27  - Create a `limiter.Rate` instance _(the number of requests per period)_
    28  - Create a `limiter.Store` instance _(see [Redis](https://github.com/ulule/limiter/blob/master/drivers/store/redis/store.go) or [In-Memory](https://github.com/ulule/limiter/blob/master/drivers/store/memory/store.go))_
    29  - Create a `limiter.Limiter` instance that takes store and rate instances as arguments
    30  - Create a middleware instance using the middleware of your choice
    31  - Give the limiter instance to your middleware initializer
    32  
    33  **Example:**
    34  
    35  ```go
    36  // Create a rate with the given limit (number of requests) for the given
    37  // period (a time.Duration of your choice).
    38  import "github.com/ulule/limiter/v3"
    39  
    40  rate := limiter.Rate{
    41      Period: 1 * time.Hour,
    42      Limit:  1000,
    43  }
    44  
    45  // You can also use the simplified format "<limit>-<period>"", with the given
    46  // periods:
    47  //
    48  // * "S": second
    49  // * "M": minute
    50  // * "H": hour
    51  // * "D": day
    52  //
    53  // Examples:
    54  //
    55  // * 5 reqs/second: "5-S"
    56  // * 10 reqs/minute: "10-M"
    57  // * 1000 reqs/hour: "1000-H"
    58  // * 2000 reqs/day: "2000-D"
    59  //
    60  rate, err := limiter.NewRateFromFormatted("1000-H")
    61  if err != nil {
    62      panic(err)
    63  }
    64  
    65  // Then, create a store. Here, we use the bundled Redis store. Any store
    66  // compliant to limiter.Store interface will do the job. The defaults are
    67  // "limiter" as Redis key prefix and a maximum of 3 retries for the key under
    68  // race condition.
    69  import "github.com/ulule/limiter/v3/drivers/store/redis"
    70  
    71  store, err := redis.NewStore(client)
    72  if err != nil {
    73      panic(err)
    74  }
    75  
    76  // Alternatively, you can pass options to the store with the "WithOptions"
    77  // function. For example, for Redis store:
    78  import "github.com/ulule/limiter/v3/drivers/store/redis"
    79  
    80  store, err := redis.NewStoreWithOptions(pool, limiter.StoreOptions{
    81      Prefix:   "your_own_prefix",
    82  })
    83  if err != nil {
    84      panic(err)
    85  }
    86  
    87  // Or use a in-memory store with a goroutine which clears expired keys.
    88  import "github.com/ulule/limiter/v3/drivers/store/memory"
    89  
    90  store := memory.NewStore()
    91  
    92  // Then, create the limiter instance which takes the store and the rate as arguments.
    93  // Now, you can give this instance to any supported middleware.
    94  instance := limiter.New(store, rate)
    95  
    96  // Alternatively, you can pass options to the limiter instance with several options.
    97  instance := limiter.New(store, rate, limiter.WithClientIPHeader("True-Client-IP"), limiter.WithIPv6Mask(mask))
    98  
    99  // Finally, give the limiter instance to your middleware initializer.
   100  import "github.com/ulule/limiter/v3/drivers/middleware/stdlib"
   101  
   102  middleware := stdlib.NewMiddleware(instance)
   103  ```
   104  
   105  See middleware examples:
   106  
   107  - [HTTP](https://github.com/ulule/limiter-examples/tree/master/http/main.go)
   108  - [Gin](https://github.com/ulule/limiter-examples/tree/master/gin/main.go)
   109  - [Beego](https://github.com/ulule/limiter-examples/blob/master//beego/main.go)
   110  - [Chi](https://github.com/ulule/limiter-examples/tree/master/chi/main.go)
   111  - [Echo](https://github.com/ulule/limiter-examples/tree/master/echo/main.go)
   112  - [Fasthttp](https://github.com/ulule/limiter-examples/tree/master/fasthttp/main.go)
   113  
   114  ## How it works
   115  
   116  The ip address of the request is used as a key in the store.
   117  
   118  If the key does not exist in the store we set a default
   119  value with an expiration period.
   120  
   121  You will find two stores:
   122  
   123  - Redis: rely on [TTL](http://redis.io/commands/ttl) and incrementing the rate limit on each request.
   124  - In-Memory: rely on a fork of [go-cache](https://github.com/patrickmn/go-cache) with a goroutine to clear expired keys using a default interval.
   125  
   126  When the limit is reached, a `429` HTTP status code is sent.
   127  
   128  ## Limiter behind a reverse proxy
   129  
   130  ### Introduction
   131  
   132  If your limiter is behind a reverse proxy, it could be difficult to obtain the "real" client IP.
   133  
   134  Some reverse proxies, like AWS ALB, lets all header values through that it doesn't set itself.
   135  Like for example, `True-Client-IP` and `X-Real-IP`.
   136  Similarly, `X-Forwarded-For` is a list of comma-separated IPs that gets appended to by each traversed proxy.
   137  The idea is that the first IP _(added by the first proxy)_ is the true client IP. Each subsequent IP is another proxy along the path.
   138  
   139  An attacker can spoof either of those headers, which could be reported as a client IP.
   140  
   141  By default, limiter doesn't trust any of those headers: you have to explicitly enable them in order to use them.
   142  If you enable them, **you must always be aware** that any header added by any _(reverse)_ proxy not controlled
   143  by you **are completely unreliable.**
   144  
   145  ### X-Forwarded-For
   146  
   147  For example, if you make this request to your load balancer:
   148  ```bash
   149  curl -X POST https://example.com/login -H "X-Forwarded-For: 1.2.3.4, 11.22.33.44"
   150  ```
   151  
   152  And your server behind the load balancer obtain this:
   153  ```
   154  X-Forwarded-For: 1.2.3.4, 11.22.33.44, <actual client IP>
   155  ```
   156  
   157  That's mean you can't use `X-Forwarded-For` header, because it's **unreliable** and **untrustworthy**.
   158  So keep `TrustForwardHeader` disabled in your limiter option.
   159  
   160  However, if you have configured your reverse proxy to always remove/overwrite `X-Forwarded-For` and/or `X-Real-IP` headers
   161  so that if you execute this _(same)_ request:
   162  ```bash
   163  curl -X POST https://example.com/login -H "X-Forwarded-For: 1.2.3.4, 11.22.33.44"
   164  ```
   165  
   166  And your server behind the load balancer obtain this:
   167  ```
   168  X-Forwarded-For: <actual client IP>
   169  ```
   170  
   171  Then, you can enable `TrustForwardHeader` in your limiter option.
   172  
   173  ### Custom header
   174  
   175  Many CDN and Cloud providers add a custom header to define the client IP. Like for example, this non exhaustive list:
   176  
   177  * `Fastly-Client-IP` from Fastly
   178  * `CF-Connecting-IP` from Cloudflare
   179  * `X-Azure-ClientIP` from Azure
   180  
   181  You can use these headers using `ClientIPHeader` in your limiter option.
   182  
   183  ### None of the above
   184  
   185  If none of the above solution are working, please use a custom `KeyGetter` in your middleware.
   186  
   187  You can use this excellent article to help you define the best strategy depending on your network topology and your security need:
   188  https://adam-p.ca/blog/2022/03/x-forwarded-for/
   189  
   190  If you have any idea/suggestions on how we could simplify this steps, don't hesitate to raise an issue.
   191  We would like some feedback on how we could implement this steps in the Limiter API.
   192  
   193  Thank you.
   194  
   195  ## Why Yet Another Package
   196  
   197  You could ask us: why yet another rate limit package?
   198  
   199  Because existing packages did not suit our needs.
   200  
   201  We tried a lot of alternatives:
   202  
   203  1. [Throttled][1]. This package uses the generic cell-rate algorithm. To cite the
   204     documentation: _"The algorithm has been slightly modified from its usual form to
   205     support limiting with an additional quantity parameter, such as for limiting the
   206     number of bytes uploaded"_. It is brillant in term of algorithm but
   207     documentation is quite unclear at the moment, we don't need _burst_ feature for
   208     now, impossible to get a correct `After-Retry` (when limit exceeds, we can still
   209     make a few requests, because of the max burst) and it only supports `http.Handler`
   210     middleware (we use [Gin][4]). Currently, we only need to return `429`
   211     and `X-Ratelimit-*` headers for `n reqs/duration`.
   212  
   213  2. [Speedbump][3]. Good package but maybe too lightweight. No `Reset` support,
   214     only one middleware for [Gin][4] framework and too Redis-coupled. We rather
   215     prefer to use a "store" approach.
   216  
   217  3. [Tollbooth][5]. Good one too but does both too much and too little. It limits by
   218     remote IP, path, methods, custom headers and basic auth usernames... but does not
   219     provide any Redis support (only _in-memory_) and a ready-to-go middleware that sets
   220     `X-Ratelimit-*` headers. `tollbooth.LimitByRequest(limiter, r)` only returns an HTTP
   221     code.
   222  
   223  4. [ratelimit][2]. Probably the closer to our needs but, once again, too
   224     lightweight, no middleware available and not active (last commit was in August
   225     2014). Some parts of code (Redis) comes from this project. It should deserve much
   226     more love.
   227  
   228  There are other many packages on GitHub but most are either too lightweight, too
   229  old (only support old Go versions) or unmaintained. So that's why we decided to
   230  create yet another one.
   231  
   232  ## Contributing
   233  
   234  - Ping us on twitter:
   235    - [@oibafsellig](https://twitter.com/oibafsellig)
   236    - [@thoas](https://twitter.com/thoas)
   237    - [@novln\_](https://twitter.com/novln_)
   238  - Fork the [project](https://github.com/ulule/limiter)
   239  - Fix [bugs](https://github.com/ulule/limiter/issues)
   240  
   241  Don't hesitate ;)
   242  
   243  [1]: https://github.com/throttled/throttled
   244  [2]: https://github.com/r8k/ratelimit
   245  [3]: https://github.com/etcinit/speedbump
   246  [4]: https://github.com/gin-gonic/gin
   247  [5]: https://github.com/didip/tollbooth
   248  [6]: https://github.com/valyala/fasthttp
   249  [godoc-url]: https://pkg.go.dev/github.com/ulule/limiter/v3
   250  [godoc-img]: https://pkg.go.dev/badge/github.com/ulule/limiter/v3
   251  [license-img]: https://img.shields.io/badge/license-MIT-blue.svg
   252  [goreport-url]: https://goreportcard.com/report/github.com/ulule/limiter
   253  [goreport-img]: https://goreportcard.com/badge/github.com/ulule/limiter
   254  [circle-url]: https://circleci.com/gh/ulule/limiter/tree/master
   255  [circle-img]: https://circleci.com/gh/ulule/limiter.svg?style=shield&circle-token=baf62ec320dd871b3a4a7e67fa99530fbc877c99