github.com/thiagoyeds/go-cloud@v0.26.0/internal/docs/pubsub/design.md (about) 1 # Go CDK `pubsub` Design 2 3 ## Summary 4 5 This document proposes a new `pubsub` package for the Go CDK. 6 7 ## Motivation 8 9 A developer designing a new system with cross-cloud portability in mind could 10 choose a messaging system supporting pubsub, such as ZeroMQ, Kafka or RabbitMQ. 11 These pubsub systems run on AWS, Azure, GCP and others, so they pose no obstacle 12 to portability between clouds. They can also be run on-prem. Users wanting 13 managed pubsub could go with Confluent Cloud for Kafka (AWS, GCP), or CloudAMQP 14 for RabbitMQ (AWS, Azure) without losing much in the way of portability. 15 16 So what’s missing? The solution described above means being locked into a 17 particular implementation of pubsub. There is also a potential for lock-in when 18 building systems in terms of the cloud-specific services such as AWS SNS+SQS, 19 GCP PubSub or Azure Service Bus. 20 21 Developers may wish to compare different pubsub systems in terms of their 22 performance, reliability, cost or other factors, and they may want the option to 23 move between these systems without too much friction. A `pubsub` package in the 24 Go CDK could lower the cost of such experiments and migrations. 25 26 ## Goals 27 28 * Publish messages to an existing topic. 29 * Receive messages from an existing subscription. 30 * Perform not much worse than 90% compared to directly using the APIs of 31 various pubsub systems. 32 * Work well with managed pubsub services on AWS, Azure, GCP and the most used 33 open source pubsub systems. 34 35 ## Non-goals 36 37 * Create new topics in the cloud. The Go CDK focuses on developer concerns, 38 but topic creation is an 39 [operator concern](https://github.com/google/go-cloud/blob/master/internal/docs/design.md#developers-and-operators). 40 41 * Create new subscriptions in the cloud. The subscribers are assumed to 42 correspond to components of a distributed system rather than to users of 43 that system. 44 45 ## Background 46 47 [Pubsub](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern) is a 48 frequently requested feature for the Go CDK project 49 \[[github issue](https://github.com/google/go-cloud/issues/312)]. A key use case 50 motivating these requests is to support 51 [event driven architectures](https://en.wikipedia.org/wiki/Event-driven_architecture). 52 53 There are several pubsub systems available that could be made to work with the 54 Go CDK by writing drivers for them. Here is a 55 [table](https://docs.google.com/a/google.com/spreadsheets/d/e/2PACX-1vQ2CML8muCrqhinxOeKTcWtwAeGk-RFFFMjB3O2u5DbbBt9R3YnUQcgRjRp6TySXe1CzSOtPVCsKACY/pubhtml) 56 comparing some of them. 57 58 ## Design overview 59 60 ### Developer’s perspective 61 62 Given a topic that has already been created on the pubsub server, messages can 63 be sent to that topic by calling `acmepubsub.OpenTopic` and calling the `Send` 64 method of the returned `Topic`, like this (assuming a fictional pubsub service 65 called "acme"): 66 67 ```go 68 package main 69 70 import ( 71 "context" 72 "log" 73 "net/http" 74 75 rawacmepubsub "github.com/acme/pubsub" 76 "github.com/google/go-cloud/pubsub" 77 "github.com/google/go-cloud/pubsub/acmepubsub" 78 ) 79 80 func main() { 81 log.Fatal(serve()) 82 } 83 84 func serve() error { 85 ctx := context.Background() 86 client, err := rawacmepubsub.NewClient(ctx, "unicornvideohub") 87 if err != nil { 88 return err 89 } 90 t, err := acmepubsub.OpenTopic(ctx, client, "user-signup", nil) 91 if err != nil { 92 return err 93 } 94 defer t.Close() 95 http.HandleFunc("/signup", func(w http.ResponseWriter, r *http.Request) { 96 err := t.Send(r.Context(), pubsub.Message{Body: []byte("Someone signed up")}) 97 if err != nil { 98 log.Println(err) 99 } 100 }) 101 return http.ListenAndServe(":8080", nil) 102 } 103 ``` 104 105 The call to `Send` will only return after the message has been sent to the 106 server or its sending has failed. 107 108 Messages can be received from an existing subscription to a topic by calling the 109 `Receive` method on a `Subscription` object returned from 110 `acmepubsub.OpenSubscription`, like this: 111 112 ```go 113 package main 114 115 import ( 116 "context" 117 "fmt" 118 "log" 119 120 rawacmepubsub "github.com/acme/pubsub" 121 "github.com/google/go-cloud/pubsub" 122 "github.com/google/go-cloud/pubsub/acmepubsub" 123 ) 124 125 func main() { 126 if err := receive(); err != nil { 127 log.Fatal(err) 128 } 129 } 130 131 func receive() error { 132 ctx := context.Background() 133 client, err := rawacmepubsub.NewClient(ctx, "unicornvideohub") 134 if err != nil { 135 return err 136 } 137 s, err := acmepubsub.OpenSubscription(ctx, client, "user-signup-minder", nil) 138 if err != nil { 139 return err 140 } 141 defer s.Close() 142 msg, err := s.Receive(ctx) 143 if err != nil { 144 return err 145 } 146 // Do something with msg. 147 fmt.Printf("Got message: %s\n", msg.Body) 148 // Acknowledge that we handled the message. 149 msg.Ack() 150 } 151 ``` 152 153 A more realistic subscriber client would process messages in a loop, like this: 154 155 ```go 156 package main 157 158 import ( 159 "context" 160 "log" 161 "os" 162 "os/signal" 163 164 "github.com/google/go-cloud/pubsub" 165 "github.com/google/go-cloud/pubsub/acmepubsub" 166 ) 167 168 func main() { 169 if err := receive(); err != nil { 170 log.Fatal(err) 171 } 172 } 173 174 func receive() error { 175 ctx := context.Background() 176 client, err := rawacmepubsub.NewClient(ctx, "unicornvideohub") 177 if err != nil { 178 return err 179 } 180 s, err := acmepubsub.OpenSubscription(ctx, client, "signup-minder", nil) 181 if err != nil { 182 return err 183 } 184 defer s.Close() 185 186 // Process messages. 187 for { 188 msg, err := s.Receive(ctx) 189 if err { 190 return err 191 } 192 log.Printf("Got message: %s\n", msg.Body) 193 msg.Ack() 194 } 195 } 196 ``` 197 198 The messages can be processed concurrently with an 199 [inverted worker pool](https://www.youtube.com/watch?v=5zXAHh5tJqQ&t=26m58s), 200 like this: 201 202 ```go 203 package main 204 205 import ( 206 "context" 207 "log" 208 "os" 209 "os/signal" 210 211 "github.com/google/go-cloud/pubsub" 212 "github.com/google/go-cloud/pubsub/acmepubsub" 213 ) 214 215 func main() { 216 if err := receive(); err != nil { 217 log.Fatal(err) 218 } 219 } 220 221 func receive() error { 222 ctx := context.Background() 223 client, err := rawacmepubsub.NewClient(ctx, "unicornvideohub") 224 if err != nil { 225 return err 226 } 227 s, err := acmepubsub.OpenSubscription(ctx, client, "user-signup-minder", nil) 228 if err != nil { 229 return err 230 } 231 defer s.Close() 232 233 // Process messages. 234 const poolSize = 10 235 // Use a buffered channel as a semaphore. 236 sem := make(chan struct{}, poolSize) 237 for { 238 msg, err := s.Receive(ctx) 239 if err { 240 return err 241 } 242 sem <- struct{}{} 243 go func() { 244 log.Printf("Got message: %s", msg.Body) 245 msg.Ack() 246 <-sem 247 }() 248 } 249 for n := poolSize; n > 0; n-- { 250 sem <- struct{}{} 251 } 252 } 253 ``` 254 255 ### Driver implementer’s perspective 256 257 Adding support for a new pubsub system involves the following steps, continuing 258 with the "acme" example: 259 260 1. Add a new package called `acmepubsub`. 261 2. Add private `topic` and `subscription` types to `acmepubsub` implementing 262 the corresponding interfaces in the `github.com/go-cloud/pubsub/driver` 263 package. 264 3. Add `func OpenTopic(...)` that creates an `acmepubsub.topic` and returns a 265 concrete `pubsub.Topic` object made from it. 266 4. Add `func OpenSubscription(...)` that creates an `acmepubsub.subscription` 267 and returns a `pubsub.Subscription` object made from it. 268 269 Here is a sketch of what the `acmepubsub` package could look like: 270 271 ```go 272 package acmepubsub 273 274 import ( 275 "context" 276 277 rawacmepubsub "github.com/acme/pubsub" 278 "github.com/google/go-cloud/pubsub" 279 "github.com/google/go-cloud/pubsub/driver" 280 ) 281 282 // OpenTopic opens an existing topic on the pubsub server and returns a Topic 283 // that can be used to send messages to that topic. 284 func OpenTopic(ctx context.Context, client *rawacmepubsub.Client, topicName string) (*pubsub.Topic, error) { 285 rt, err := client.Topic(ctx, topicName) 286 if err != nil { 287 return nil, err 288 } 289 rt, err := client.Topic(ctx, topicName) 290 if err != nil { 291 return err 292 } 293 t := &topic{ rawTopic: rt } 294 return pubsub.NewTopic(t) 295 } 296 297 // OpenSubscription opens an existing subscription on the server and returns a 298 // Subscription that can be used to receive messages. 299 func OpenSubscription(ctx context.Context, client *rawacmepubsub.Client, subscriptionName string) (*pubsub.Subscription, error) { 300 rs, err := client.Subscription(ctx, subscriptionName) 301 if err != nil { 302 return err 303 } 304 s := &subscription{ rawSub: rs } 305 return pubsub.NewSubscription(s) 306 } 307 308 type topic struct { 309 rawTopic *rawacmepubsub.Topic 310 } 311 312 func (t *topic) SendBatch(ctx context.Context, []*pubsub.Message) error { 313 // ... 314 } 315 316 func (t *topic) Close() error { 317 // ... 318 } 319 320 type subscription struct { 321 rawSub *rawacmepubsub.Subscription 322 } 323 324 func (s *subscription) ReceiveBatch(ctx context.Context) ([]*pubsub.Message, error) { 325 // ... 326 } 327 328 func (s *subscription) SendAcks(ctx context.Context, []pubsub.AckID) error { 329 // ... 330 } 331 332 func (s *subscription) Close() error { 333 // ... 334 } 335 ``` 336 337 The driver interfaces are batch-oriented because some pubsub systems can more 338 efficiently deal with batches of messages than with one at a time. Streaming was 339 considered but it does not appear to provide enough of a performance gain to be 340 worth the additional complexity of supporting it across different pubsub systems 341 \[[benchmarks](https://github.com/ijt/pubsub/tree/master/benchmarks)]. 342 343 The driver interfaces will be located in the 344 `github.com/google/go-cloud/pubsub/driver` package and will look something like 345 this: 346 347 ```go 348 package driver 349 350 type AckID interface{} 351 352 type Message struct { 353 // Body contains the content of the message. 354 Body []byte 355 356 // Attributes has key/value metadata for the message. 357 Attributes map[string]string 358 359 // AckID identifies the message on the server. 360 // It can be used to ack the message after it has been received. 361 AckID AckID 362 } 363 364 // Topic publishes messages. 365 type Topic interface { 366 // SendBatch publishes all the messages in ms. 367 SendBatch(ctx context.Context, ms []*Message) error 368 369 // Close disconnects the Topic. 370 Close() error 371 } 372 373 // Subscription receives published messages. 374 type Subscription interface { 375 // ReceiveBatch should return a batch of messages that have queued up 376 // for the subscription on the server. 377 // 378 // If there is a transient failure, this method should not retry but 379 // should return a nil slice and an error. The concrete API will take 380 // care of retry logic. 381 // 382 // If the service returns no messages for some other reason, this 383 // method should return the empty slice of messages and not attempt to 384 // retry. 385 // 386 // ReceiveBatch is only called sequentially for individual 387 // Subscriptions. 388 ReceiveBatch(ctx context.Context) ([]*Message, error) 389 390 // SendAcks acknowledges the messages with the given ackIDs on the 391 // server so that they 392 // will not be received again for this subscription. This method 393 // returns only after all the ackIDs are sent. 394 SendAcks(ctx context.Context, ackIDs []interface{}) error 395 396 // Close disconnects the Subscription. 397 Close() error 398 } 399 ``` 400 401 ## Detailed design 402 403 The developer experience of using Go CDK's pubsub involves sending, receiving 404 and acknowledging one message at a time, all in terms of synchronous calls. 405 Behind the scenes, the driver implementations deal with batches of messages and 406 acks. The concrete API, to be written by the Go CDK team, takes care of creating 407 the batches in the case of Send or Ack, and dealing out messages one at a time 408 in the case of Receive. 409 410 The concrete API will be located at `github.com/google/go-cloud/pubsub` and will 411 look something like this: 412 413 ```go 414 package pubsub 415 416 import ( 417 "context" 418 "github.com/google/go-cloud/pubsub/driver" 419 ) 420 421 // Message contains data to be published. 422 type Message struct { 423 // Body contains the content of the message. 424 Body []byte 425 426 // Attributes contains key/value pairs with metadata about the message. 427 Attributes map[string]string 428 429 // ackID is an ID for the message on the server, used for acking. 430 ackID AckID 431 432 // sub is the Subscription this message was received from. 433 sub *Subscription 434 435 // isAcked is true if Ack has been called on this message. 436 isAcked bool 437 } 438 439 type AckID interface{} 440 441 // Ack acknowledges the message, telling the server that it does not need to 442 // be sent again to the associated Subscription. This method returns 443 // immediately. If Ack has already been called on the message, Ack panics. 444 func (m *Message) Ack() { 445 // Send the ack ID back to the subscriber for batching. 446 // The ack is sent to the server in a separate goroutine 447 // managed by the Subscription from which this message was 448 // received. 449 // ... 450 } 451 452 // Topic publishes messages to all its subscribers. 453 type Topic struct { 454 driver driver.Topic 455 mcChan chan msgCtx 456 doneChan chan struct{} 457 } 458 459 // msgCtx pairs a Message with the Context of its Send call. 460 type msgCtx struct { 461 msg *Message 462 ctx context.Context 463 } 464 465 // Send publishes a message. It only returns after the message has been 466 // sent, or failed to be sent. The call will fail if ctx is canceled. 467 // Send can be called from multiple goroutines at once. 468 func (t *Topic) Send(ctx context.Context, m *Message) error { 469 // Send this message over t.mcChan and then wait for the batch including 470 // this message to be sent to the server. 471 // ... 472 } 473 474 // Close disconnects the Topic. 475 func (t *Topic) Close() error { 476 close(t.doneChan) 477 return t.driver.Close() 478 } 479 480 // NewTopic makes a pubsub.Topic from a driver.Topic. 481 func NewTopic(d driver.Topic) *Topic { 482 t := &Topic{ 483 driver: d, 484 mcChan: make(chan msgCtx), 485 doneChan: make(chan struct{}), 486 } 487 go func() { 488 // Pull messages from t.mcChan and put them in batches. Send the current 489 // batch whenever it is large enough or enough time has elapsed since 490 // the last send. 491 // ... 492 }() 493 return t 494 } 495 496 // Subscription receives published messages. 497 type Subscription struct { 498 driver driver.Subscription 499 500 // ackChan conveys ackIDs from Message.Ack to the ack batcher goroutine. 501 ackChan chan AckID 502 503 // ackErrChan reports errors back to Message.Ack. 504 ackErrChan chan error 505 506 // doneChan tells the goroutine from startAckBatcher to finish. 507 doneChan chan struct{} 508 509 // q is the local queue of messages downloaded from the server. 510 q []*Message 511 } 512 513 // Receive receives and returns the next message from the Subscription's queue, 514 // blocking if none are available. This method can be called concurrently from 515 // multiple goroutines. On systems that support acks, the Ack() method of the 516 // returned Message has to be called once the message has been processed, to 517 // prevent it from being received again. 518 func (s *Subscription) Receive(ctx context.Context) (*Message, error) { 519 if len(s.q) == 0 { 520 // Get the next batch of messages from the server. 521 // ... 522 } 523 m := s.q[0] 524 s.q = s.q[1:] 525 return m, nil 526 } 527 528 // Close disconnects the Subscription. 529 func (s *Subscription) Close() error { 530 close(s.doneChan) 531 return s.driver.Close() 532 } 533 534 // NewSubscription creates a Subscription from a driver.Subscription and opts to 535 // tune sending and receiving of acks and messages. Behind the scenes, 536 // NewSubscription spins up a goroutine to gather acks into batches and 537 // periodically send them to the server. 538 func NewSubscription(s driver.Subscription) *Subscription { 539 // Details similar to the body of NewTopic should go here. 540 } 541 ``` 542 543 Topics will gather messages into batches for sending. The batch size will be 544 dynamically tuned according to how many messages are being sent concurrently. 545 546 Subscriptions will gather message acks into batches the same way, also 547 dynamically tuning the batch size. If sending acks back to the server fails 548 transiently then it will be retried, most likely within a loop in the concrete 549 API. If an unrecoverable error occurs while sending acks then a flag will be set 550 on the `pubsub.Subscription` saying that the whole `Subscription` is no longer 551 usable. Calls to `Receive` will fail from then on. 552 553 ## Alternative designs considered 554 555 ### Batch oriented concrete API 556 557 In this alternative, the application code sends, receives and acknowledges 558 messages in batches. Here is an example of how it would look from the 559 developer's perspective, in a situation where not too many signups are happening 560 per second. 561 562 ```go 563 package main 564 565 import ( 566 "context" 567 "log" 568 "net/http" 569 570 rawacmepubsub "github.com/acme/pubsub" 571 "github.com/google/go-cloud/pubsub" 572 "github.com/google/go-cloud/pubsub/acmepubsub" 573 ) 574 575 func main() { 576 log.Fatal(serve()) 577 } 578 579 func serve() error { 580 ctx := context.Background() 581 client, err := rawacmepubsub.NewClient(ctx, "unicornvideohub") 582 if err != nil { 583 return err 584 } 585 t, err := acmepubsub.OpenTopic(ctx, client, "user-signup", nil) 586 if err != nil { 587 return err 588 } 589 defer t.Close() 590 http.HandleFunc("/signup", func(w http.ResponseWriter, r *http.Request) { 591 err := t.Send(r.Context(), []pubsub.Message{{Body: []byte("Someone signed up")}}) 592 if err != nil { 593 log.Println(err) 594 } 595 }) 596 return http.ListenAndServe(":8080", nil) 597 } 598 ``` 599 600 For a company experiencing explosive growth or enthusiastic spammers creating 601 more signups than this simple-minded implementation can handle, the app would 602 have to be adapted to create non-singleton batches, like this: 603 604 ```go 605 package main 606 607 import ( 608 "context" 609 "log" 610 "net/http" 611 612 rawacmepubsub "github.com/acme/pubsub" 613 "github.com/google/go-cloud/pubsub" 614 "github.com/google/go-cloud/pubsub/acmepubsub" 615 ) 616 617 const batchSize = 1000 618 619 func main() { 620 log.Fatal(serve()) 621 } 622 623 func serve() error { 624 ctx := context.Background() 625 client, err := rawacmepubsub.NewClient(ctx, "unicornvideohub") 626 if err != nil { 627 return err 628 } 629 t, err := acmepubsub.OpenTopic(ctx, client, "user-signup", nil) 630 if err != nil { 631 return err 632 } 633 defer t.Close() 634 c := make(chan *pubsub.Message) 635 go sendBatches(ctx, t, c) 636 http.HandleFunc("/signup", func(w http.ResponseWriter, r *http.Request) { 637 c <- &pubsub.Message{Body: []byte("Someone signed up")} 638 }) 639 return http.ListenAndServe(":8080", nil) 640 } 641 642 func sendBatches(ctx context.Context, t *pubsub.Topic, c chan *pubsub.Message) { 643 batch := make([]*pubsub.Message, batchSize) 644 for { 645 for i := 0; i < batchSize; i++ { 646 batch[i] = <-c 647 } 648 if err := t.Send(ctx, batch); err != nil { 649 log.Println(err) 650 } 651 } 652 } 653 ``` 654 655 This shows how the complexity of batching has been pushed onto the application 656 code. Removing messages from the batch when HTTP/2 requests are canceled would 657 require the application code to be even more complex, adding more risk of bugs. 658 659 In this API, the application code has to either request batches of size 1, 660 meaning more network traffic, or it has to explicitly manage the batches of 661 messages it receives. Here is an example of how this API would be used for 662 serial message processing: 663 664 ```go 665 package main 666 667 import ( 668 "context" 669 "log" 670 "os" 671 "os/signal" 672 673 rawacmepubsub "github.com/acme/pubsub" 674 "github.com/google/go-cloud/pubsub" 675 "github.com/google/go-cloud/pubsub/acmepubsub" 676 ) 677 678 const batchSize = 10 679 680 func main() { 681 if err := receive(); err != nil { 682 log.Fatal(err) 683 } 684 } 685 686 func receive() error { 687 ctx := context.Background() 688 client, err := rawacmepubsub.NewClient(ctx, "unicornvideohub") 689 if err != nil { 690 return err 691 } 692 s, err := acmepubsub.OpenSubscription(ctx, client, "signup-minder", nil) 693 if err != nil { 694 return err 695 } 696 defer s.Close() 697 698 // Process messages. 699 for { 700 msgs, err := s.Receive(ctx, batchSize) 701 if err { 702 return err 703 } 704 acks := make([]pubsub.AckID, 0, batchSize) 705 for _, msg := range msgs { 706 // Do something with msg. 707 fmt.Printf("Got message: %q\n", msg.Body) 708 acks = append(acks, msg.AckID) 709 } 710 err := s.SendAcks(ctx, acks) 711 if err != nil { 712 return err 713 } 714 } 715 } 716 ``` 717 718 Here’s what it might look like to use this batch-only API with the inverted 719 worker pool pattern: 720 721 ```go 722 package main 723 724 import ( 725 "context" 726 "log" 727 "os" 728 "os/signal" 729 730 rawacmepubsub "github.com/acme/pubsub" 731 "github.com/google/go-cloud/pubsub" 732 "github.com/google/go-cloud/pubsub/acmepubsub" 733 ) 734 735 const batchSize = 100 736 const poolSize = 10 737 738 func main() { 739 if err := receive(); err != nil { 740 log.Fatal(err) 741 } 742 } 743 744 func receive() error { 745 ctx := context.Background() 746 client, err := rawacmepubsub.NewClient(ctx, "unicornvideohub") 747 if err != nil { 748 return err 749 } 750 s, err := acmepubsub.OpenSubscription(ctx, client, "user-signup-minder", nil) 751 if err != nil { 752 return err 753 } 754 defer s.Close() 755 756 // Receive the messages and forward them to a chan. 757 msgsChan := make(chan *pubsub.Message) 758 go func() { 759 for { 760 msgs, err := s.Receive(ctx, batchSize) 761 if err { 762 log.Fatal(err) 763 } 764 for _, m := range msgs { 765 msgsChan <- m 766 } 767 } 768 } 769 770 // Get the acks from a chan and send them back to the 771 // server in batches. 772 acksChan := make(chan pubsub.AckID) 773 go func() { 774 for { 775 batch := make([]pubsub.AckID, batchSize) 776 for i := 0; i < len(batch); i++ { 777 batch[i] = <-acksChan 778 } 779 if err := s.SendAcks(ctx, batch); err != nil { 780 /* handle err */ 781 } 782 } 783 } 784 785 // Use a buffered channel as a semaphore. 786 sem := make(chan struct{}, poolSize) 787 for msg := range msgsChan { 788 sem <- struct{}{} 789 go func(msg *pubsub.Message) { 790 log.Printf("Got message: %s", msg.Body) 791 acksChan <- msg.AckID 792 <-sem 793 }(msg) 794 } 795 for n := poolSize; n > 0; n-- { 796 sem <- struct{}{} 797 } 798 } 799 ``` 800 801 Here are some trade-offs of this design: 802 803 Pro: 804 805 * The semantics are simple, making it 806 * straightforward to implement the concrete API and the drivers for most 807 pubsub services 808 * easy for developers to reason about how it will behave 809 * less risky that bugs will be present in the concrete API 810 * Fairly efficient sending and receiving of messages is possible by tuning 811 batch size and the number of goroutines sending or receiving messages. 812 813 Con: 814 815 * This style of API makes the inverted worker pool pattern verbose. 816 * Apps needing to send or receive a large volume of messages must have their 817 own logic to create batches of size greater than 1. 818 819 ### go-micro 820 821 Here is an example of what application code could look like for a pubsub API 822 inspired by [`go-micro`](https://github.com/micro/go-micro)'s `broker` package: 823 824 ```go 825 b := somepubsub.NewBroker(...) 826 if err := b.Connect(); err != nil { 827 /* handle err */ 828 } 829 topic := "user-signups" 830 subID := "user-signups-subscription-1" 831 s, err := b.Subscription(ctx, topic, subID, func(pub broker.Publication) error { 832 fmt.Printf("%s\n", pub.Message.Body) 833 return nil 834 }) 835 if err := b.Publish(ctx, topic, &broker.Message{ Body: []byte("alice signed up") }); err != nil { 836 /* handle err */ 837 } 838 // Sometime later: 839 if err := s.Unsubscribe(ctx); err != nil { 840 /* handle err */ 841 } 842 ``` 843 844 Pro: 845 846 * The callback to the subscription returning an error to decide whether to 847 acknowledge the message means the developer cannot forget to ack. 848 849 Con: 850 851 * Go micro has code to auto-create 852 [topics](https://github.com/micro/go-plugins/blob/f3fcfcdf77392b4e053c8d5b361abfabc0c623d3/broker/googlepubsub/googlepubsub.go#L152) 853 and 854 [subscriptions](https://github.com/micro/go-plugins/blob/f3fcfcdf77392b4e053c8d5b361abfabc0c623d3/broker/googlepubsub/googlepubsub.go#L185) 855 as needed, but this is not consistent with the Go CDK’s design principle to 856 not get involved in operations. 857 * The subscription callback idea does not appear to be compatible with 858 inverted worker pools. 859 860 ## Acknowledgements 861 862 In pubsub systems with acknowledgement, messages are kept in a queue associated 863 with the subscription on the server. When a client receives one of these 864 messages, its counterpart on the server is marked as being processed. Once the 865 client finishes processing the message, it sends an acknowledgement (or "ack") 866 to the server and the server removes the message from the subscription queue. 867 There may be a deadline for the acknowledgement, past which the server unmarks 868 the message so that it can be received again for another try at processing. 869 870 Redis Pub/Sub and ZeroMQ don’t support acking, but many others do including GCP 871 PubSub, Azure Service Bus, RabbitMQ, and 872 [Redis Streams](https://redis.io/topics/streams-intro). Given the wide support 873 and usefulness, it makes sense to support message acking in the Go CDK. 874 875 As of this writing, it is an open question as to what should be done about 876 pubsub systems that do not support acks. Some possibilities have been discussed, 877 but no clear best option has emerged yet: 878 879 1. simulating acknowledgement by constructing queues on the server. Con: the 880 magically created queues would probably be a less than pleasant surprise for 881 some users. 882 2. making ack a no-op for systems that don't support it. With this, do we 883 return a sentinel error from `Ack`, and if so then doesn't that unduly 884 complicate the code for apps that never use non-acking systems? This option 885 is also potentially misleading for developers who would naturally assume 886 that un-acked messages would be redelivered. 887 888 ### Rejected acknowledgement API: `Receive` method returns an `ack` func 889 890 In this alternative, the application code would look something like this: 891 892 ```go 893 msg, ack, err := s.Receive(ctx) 894 log.Printf("Received message: %q", msg.Body) 895 ack(msg) 896 ``` 897 898 Pro: 899 900 * The compiler will complain if the returned `ack` function is not used. 901 902 Con: 903 904 * Receive has one more return value. 905 * Passing `ack` around along with `msg` is inconvenient. 906 907 ## Tests 908 909 ### Unit tests for the concrete API (`github.com/go-cloud/pubsub`) 910 911 We can test that the batched sending, receiving and acking work as intended by 912 making mock implementations of the driver interfaces. 913 914 At least the following things should be tested: 915 916 * Calling `pubsub.Message.Ack` causes `driver.Subscription.SendAcks` to be 917 called. 918 * Calling `pubsub.Topic.Send` causes `driver.Topic.SendBatch` to be called. 919 * Calling `pubsub.Subscription.Receive` causes 920 `driver.Subscription.ReceiveBatch` to be called. 921 922 ### Conformance tests for specific implementations (*e.g.*, `github.com/go-cloud/pubsub/acmepubsub`) 923 924 * Sent messages with random contents are received with the same contents. 925 * Sent messages with random attributes are received with the same attributes. 926 * Error occurs when making a local topic with an ID that doesn’t exist on the 927 server. 928 * Error occurs when making a subscription with an ID that doesn’t exist on the 929 server. 930 * Message gets sent again after ack deadline if a message is never 931 acknowledged. 932 * ~~Acked messages don't get received again after waiting twice the ack 933 deadline.~~ :point_left: This test would probably be too flakey. 934 935 ## Benchmarks 936 937 What is the throughput and latency of the Go CDK's `pubsub` package, relative to 938 directly using the APIs for various services? 939 940 * send, for 1, 10, 100 topics, and for 1, 10, 100 goroutines sending messages 941 to those topics 942 * receive, for 1, 10, 100 subscriptions, and for 1, 10, 100 goroutines 943 receiving from each subscription 944 945 ## References 946 947 * https://github.com/google/go-cloud/issues/312 948 * http://queues.io/