github.com/wfusion/gofusion@v1.1.14/common/infra/watermill/README.md (about) 1 # Watermill 2 <img align="right" width="200" src="https://threedots.tech/watermill-io/watermill-logo.png"> 3 4 [](https://github.com/ThreeDotsLabs/watermill/actions/workflows/master.yml) 5 [](https://pkg.go.dev/github.com/ThreeDotsLabs/watermill) 6 [](https://goreportcard.com/report/github.com/ThreeDotsLabs/watermill) 7 [](https://codecov.io/gh/ThreeDotsLabs/watermill) 8 9 Watermill is a Go library for working efficiently with message streams. It is intended 10 for building event driven applications, enabling event sourcing, RPC over messages, 11 sagas and basically whatever else comes to your mind. You can use conventional pub/sub 12 implementations like Kafka or RabbitMQ, but also HTTP or MySQL binlog if that fits your use case. 13 14 ## Goals 15 16 * **Easy** to understand. 17 * **Universal** - event-driven architecture, messaging, stream processing, CQRS - use it for whatever you need. 18 * **Fast** (see [Benchmarks](#benchmarks)). 19 * **Flexible** with middlewares, plugins and Pub/Sub configurations. 20 * **Resilient** - using proven technologies and passing stress tests (see [Stability](#stability)). 21 22 ## Getting Started 23 24 Pick what you like the best or see in order: 25 26 1. Follow the [Getting Started guide](https://watermill.io/docs/getting-started/). 27 2. See examples below. 28 3. Read the full documentation: https://watermill.io/ 29 30 ## Our online hands-on training 31 32 <a href="https://threedots.tech/event-driven/?utm_source=watermill-readme"><img align="center" width="400" src="https://threedots.tech/event-driven-banner.png"></a> 33 34 ## Examples 35 36 * Basic 37 * [Your first app](_examples/basic/1-your-first-app) - **start here!** 38 * [Realtime feed](_examples/basic/2-realtime-feed) 39 * [Router](_examples/basic/3-router) 40 * [Metrics](_examples/basic/4-metrics) 41 * [CQRS with protobuf](_examples/basic/5-cqrs-protobuf) 42 * [Pub/Subs usage](_examples/pubsubs) 43 * These examples are part of the [Getting started guide](https://watermill.io/docs/getting-started/) and show usage of a single Pub/Sub at a time. 44 * Real-world examples 45 * [Exactly-once delivery counter](_examples/real-world-examples/exactly-once-delivery-counter) 46 * [Receiving webhooks](_examples/real-world-examples/receiving-webhooks) 47 * [Sending webhooks](_examples/real-world-examples/sending-webhooks) 48 * [Synchronizing Databases](_examples/real-world-examples/synchronizing-databases) 49 * [Persistent Event Log](_examples/real-world-examples/persistent-event-log) 50 * [Transactional Events](_examples/real-world-examples/transactional-events) 51 * [Real-time HTTP updates with Server-Sent Events](_examples/real-world-examples/server-sent-events) 52 * Complete projects 53 * [NATS example with live code reloading](https://github.com/ThreeDotsLabs/nats-example) 54 * [RabbitMQ, webhooks and Kafka integration](https://github.com/ThreeDotsLabs/event-driven-example) 55 56 ## Background 57 58 Building distributed and scalable services is rarely as easy as some may suggest. There is a 59 lot of hidden knowledge that comes with writing such systems. Just like you don't need to know the 60 whole TCP stack to create a HTTP REST server, you shouldn't need to study all of this knowledge to 61 start with building message-driven applications. 62 63 Watermill's goal is to make communication with messages as easy to use as HTTP routers. It provides 64 the tools needed to begin working with event-driven architecture and allows you to learn the details 65 on the go. 66 67 At the heart of Watermill there is one simple interface: 68 ```go 69 func(*Message) ([]*Message, error) 70 ``` 71 72 Your handler receives a message and decides whether to publish new message(s) or return 73 an error. What happens next is up to the middlewares you've chosen. 74 75 You can find more about our motivations in our [*Introducing Watermill* blog post](https://threedots.tech/post/introducing-watermill/). 76 77 ## Pub/Subs 78 79 All publishers and subscribers have to implement an interface: 80 81 ```go 82 type Publisher interface { 83 Publish(topic string, messages ...*Message) error 84 Close() error 85 } 86 87 type Subscriber interface { 88 Subscribe(ctx context.Context, topic string) (<-chan *Message, error) 89 Close() error 90 } 91 ``` 92 93 Supported Pub/Subs: 94 95 - AMQP Pub/Sub [(`github.com/ThreeDotsLabs/watermill-amqp/v2`)](https://github.com/ThreeDotsLabs/watermill-amqp/) 96 - Bolt Pub/Sub [(`github.com/ThreeDotsLabs/watermill-bolt`)](https://github.com/ThreeDotsLabs/watermill-bolt/) 97 - Firestore Pub/Sub [(`github.com/ThreeDotsLabs/watermill-firestore`)](https://github.com/ThreeDotsLabs/watermill-firestore/) 98 - Google Cloud Pub/Sub [(`github.com/ThreeDotsLabs/watermill-googlecloud`)](https://github.com/ThreeDotsLabs/watermill-googlecloud/) 99 - HTTP Pub/Sub [(`github.com/ThreeDotsLabs/watermill-http`)](https://github.com/ThreeDotsLabs/watermill-http/) 100 - io.Reader/io.Writer Pub/Sub [(`github.com/ThreeDotsLabs/watermill-io`)](https://github.com/ThreeDotsLabs/watermill-io/) 101 - Kafka Pub/Sub [(`github.com/ThreeDotsLabs/watermill-kafka/v2`)](https://github.com/ThreeDotsLabs/watermill-kafka/) 102 - NATS Pub/Sub [(`github.com/ThreeDotsLabs/watermill-nats`)](https://github.com/ThreeDotsLabs/watermill-nats/) 103 - Redis Stream Pub/Sub [(`github.com/ThreeDotsLabs/watermill-redisstream`)](https://github.com/ThreeDotsLabs/watermill-redisstream/) 104 - SQL Pub/Sub [(`github.com/ThreeDotsLabs/watermill-sql/v2`)](https://github.com/ThreeDotsLabs/watermill-sql/) 105 106 107 All Pub/Subs implementation documentation can be found in the [documentation](https://watermill.io/pubsubs/). 108 109 ## Unofficial libraries 110 111 Can't find your favorite Pub/Sub or library integration? Check [Awesome Watermill](https://watermill.io/docs/awesome/). 112 113 If you know another library or are an author of one, please [add it to the list](https://github.com/ThreeDotsLabs/watermill/edit/master/docs/content/docs/awesome.md). 114 115 ## Contributing 116 117 Please check our [contributing guide](CONTRIBUTING.md). 118 119 ## Stability 120 121 Watermill v1.0.0 has been released and is production-ready. The public API is stable and will not change without changing the major version. 122 123 To ensure that all Pub/Subs are stable and safe to use in production, we created a [set of tests](https://github.com/ThreeDotsLabs/watermill/blob/master/pubsub/tests/test_pubsub.go#L34) that need to pass for each of the implementations before merging to master. 124 All tests are also executed in [*stress*](https://github.com/ThreeDotsLabs/watermill/blob/master/pubsub/tests/test_pubsub.go#L171) mode - that means that we are running all the tests **20x** in parallel. 125 126 All tests are run with the race condition detector enabled (`-race` flag in tests). 127 128 For more information about debugging tests, you should check [tests troubleshooting guide](http://watermill.io/docs/troubleshooting/#debugging-pubsub-tests). 129 130 ## Benchmarks 131 132 Initial tools for benchmarking Pub/Subs can be found in [watermill-benchmark](https://github.com/ThreeDotsLabs/watermill-benchmark). 133 134 All benchmarks are being done on a single 16 CPU VM instance, running one binary and dependencies in Docker Compose. 135 136 These numbers are meant to serve as a rough estimate of how fast messages can be processed by different Pub/Subs. 137 Keep in mind that the results can be vastly different, depending on the setup and configuration (both much lower and higher). 138 139 Here's the short version for message size of 16 bytes. 140 141 | Pub/Sub | Publish (messages / s) | Subscribe (messages / s) | 142 |---------------------------------|------------------------|--------------------------| 143 | GoChannel | 331,882 | 118,943 | 144 | Redis Streams | 61,642 | 11,213 | 145 | NATS Jetstream (16 Subscribers) | 49,255 | 33,009 | 146 | Kafka (one node) | 44,090 | 108,285 | 147 | SQL (MySQL) | 5,599 | 167 | 148 | SQL (PostgreSQL, batch size=1) | 3,834 | 455 | 149 | Google Cloud Pub/Sub | 3,689 | 30,229 | 150 | AMQP | 2,702 | 13,192 | 151 152 ## Support 153 154 If you didn't find the answer to your question in [the documentation](https://watermill.io/), feel free to ask us directly! 155 156 Please join us on the `#watermill` channel on the [Three Dots Labs Discord](https://discord.gg/QV6VFg4YQE). 157 158 Every bit of feedback is very welcome and appreciated. Please submit it using [the survey](https://www.surveymonkey.com/r/WZXD392). 159 160 ## Why the name? 161 162 It processes streams! 163 164 ## License 165 166 [MIT License](./LICENSE)