github.com/cozy/cozy-stack@v0.0.0-20240603063001-31110fa4cae1/docs/realtime-internals.md (about)

     1  [Table of contents](README.md#table-of-contents)
     2  
     3  # Realtime internals
     4  
     5  The stack has a realtime package, which can be seen as a pub/sub of events.
     6  Most events are created by the CouchDB package when a document is created,
     7  updated or deleted, but there are also some synthetic events like the end of
     8  the initial synchronization of a Cozy to Cozy sharing. Those events can be
     9  sent via the realtime websockets to clients and can be used by the scheduler
    10  to fire triggers.
    11  
    12  There are two versions of the Hub implementation: an in-memory one, and a redis
    13  one. The in-memory one is used by the developers and the self-hosted instances.
    14  The redis one is used when there are several stacks, to send the events from
    15  one stack to the others, and then use the in-memory hub internally.
    16  
    17  ## Model
    18  
    19  ![Model of realtime package](diagrams/realtime-model.png)
    20  
    21  Let's start by the most basic unit. The EventsChan is go channel for sending
    22  events.
    23  
    24  Then, we have the topic. There is one topic per doctype of an instance if
    25  there is at least one subscriber for it. It receives events via its
    26  `broadcast` channel.
    27  
    28  The mem hub is where the stack keeps the list of topics. There is a mutex to
    29  protect against race conditions (a sync.Map is not enough, as we also need to
    30  initialize topics).
    31  
    32  The redis hub is here to ensure that each stack can publish the events in its
    33  own mem hub. It also has the concept of "firehose": it is a special topic where
    34  every events created on this stack are published. It is used by the scheduler
    35  for the events trigger. The mem hub is not used for that, as we don't want to
    36  fire the same triggers n times (one on each stack), but only once.
    37  
    38  And then, we have the subscribers. A subscriber is used to receive events. It
    39  is tied to an instance (via the prefixer) and can subscribes to doctypes. With
    40  the instance + doctype, the hub can find the right topic, and adds the
    41  subscriber to the `subs` map of the topic. We have the notion of filter because
    42  the subscriber may want to receive the events for the whole doctype (subscribe)
    43  or just for some identifiers (watch).
    44  
    45  ## Workflow of a realtime event (redis hub)
    46  
    47  ![Workflow of a realtime event](diagrams/realtime-workflow.png)
    48  
    49  1. A client connects to the realtime websocket and subscribe to some doctypes
    50  2. The stack creates a Subscriber for it and connects it to the mem hub.
    51  3. Another client sends an HTTP request, which is makes a CouchDB write
    52  4. This create an event to publish on the redis hub
    53  5. The event is sent to redis (PUB command)
    54  6. Each stack listens to redis and receive this event
    55  7. The redis hub send the event to the mem hub
    56  8. The mem hub finds the topic for this event, sees that there is subscriber for it, and forward it the event
    57  9. The subscriber sends the event to the client
    58  10. The redis hub also send the event to the firehose topic (only on the stack where the event was created)
    59  11. The redis scheduler is listening to the broadcast and receive the events
    60  12. With this event, the redis scheduler can look at the matching triggers, and fire them