github.com/voedger/voedger@v0.0.0-20240520144910-273e84102129/design/consistency/README-v1.md (about)

     1  ## Abstract
     2  
     3  Consistency handling design
     4  
     5  
     6  
     7  ## Motivation
     8  One of the main challenges in CQRS systems is eventual consistency of the Read Model.
     9  [Microsoft Azure Documentation](https://learn.microsoft.com/en-us/azure/architecture/patterns/cqrs):
    10  
    11  > Eventual consistency. If you separate the read and write databases, the read data may be stale. The read model store must be updated to reflect changes to the write model store, and it can be difficult to detect when a user has issued a request based on stale read data.
    12  
    13  Scenarios:
    14  
    15  - Client wants to read ASAP, consistency doesn't matter
    16    - Examples:
    17      - Read dashboard figures
    18      - Read journal (WLog) for building reports
    19  - Client wants to read the data which reflects the last operation made by Client
    20    - New operations from other clients can be seen
    21    - Examples:
    22      - Read transaction history after making an order or payment (not used atm)
    23  - Client wants the data snapshot
    24    - Examples:
    25      - Read the BO state
    26      - Read the TablesOverview
    27  - Read by [Projectors](./inv-projector-reads.md)
    28  - Read by [Command Functions](./inv-cmdfunction-reads.md) and Validators
    29  
    30  ### Literature review
    31  
    32  Based on [inv-articles-consistency.md](inv-articles-consistency.md)
    33  
    34  ## Terms
    35  - Consistency (Согласованность)
    36      - Explains when the Model reflects changes made by command
    37      - Объясняет в какой момент модель отобразит изменения в результате выполнения команды
    38  - [Write model](../README.md#event-sourcing--cqrs) in Heeus
    39      - Represented by PLogPartition
    40      - Strongly consistent (строго согласована): you are guaranteed that the data is up-to-date imediately after [command is handled](https://10consulting.com/2017/10/06/dealing-with-eventual-consistency/)
    41  - [Read Model](../README.md#event-sourcing--cqrs) in Heeus
    42      - Represented by the set of [Internal Projections](../projectors/README.md#terms): Views, Records, WLog
    43      - Eventually consistent (Согласована в конечном итоге): you are guaranteed that the Read Model will be (sometime) updated by Projectors with the events from PLogPartition
    44  - Event Offset, Offset (Смещение события)
    45      - Uniquiely identifies the event
    46  - Projection version, Version
    47      - The offset of the last event, applied to the projection
    48  - Last offset
    49      - Offset of the last event, handled by Command Processor
    50  
    51  ## Principles
    52  - Read operations support `Isolation Level` as the tool for working with the eventual consistency of the Read Model
    53  - The following isolation levels are supported by Heeus for reading operations:
    54  
    55  |       Isolation Level     |  Projection Versions  |  Same Projection Versions* | Same Rows Versions |
    56  | ------------------------- | --------------------- | -------------------------- | ------------------ |
    57  |   Read Uncommitted        |         Any           |             No             |         No         |
    58  |   Read Committed          |     >= Offset**       |             No             |         Yes        |
    59  |   Snapshot                |     >= Offset**       |             Yes            |         Yes        |
    60  
    61  *When the result combines data from more than one projection
    62  
    63  **Offset - specified by read operation (or the last offset if not specified)
    64  
    65  - Isolation levels in components:
    66      - QueryProcessor: isolation level defined by request, one of:
    67          - Read Uncommitted (default)
    68          - Read Committed
    69          - Snapshot
    70      - Actualizer
    71          - Always reads from own Projection with "Read Uncommited"
    72          - Always reads from other Projections with "Read Committed" with the offset = `current event offset`.
    73      - Command Processor
    74          - Read from Projections with "Read Committed" with the offset = `previous event offset`.
    75  
    76  ## Concepts
    77  
    78  - AppPartition has `WsProjectionsState` component which keeps the current statuses and versions of workspace projections.
    79      - Status of the projection (idle, raw, active) is needed to provide [Lazy Projections](../projectors/lazy-projections.md).
    80  AppPartition
    81  - Intents for projections are applied in batches, One batch per workspace. The projection versions are increased in the same batch. Version in `WsProjectionsState` is updated when the intents are applied for this workspace.
    82  
    83  ### WsProjectionsState: Architecture
    84  
    85  ```mermaid
    86  erDiagram
    87  VVM ||--|{ AppPartition: handles
    88  AppPartition ||--|{ QueryProcessor: has
    89  AppPartition ||--|| CommandProcessor: has
    90  AppPartition ||--|{ Actualizer: has
    91  Projector }|--|{ State: "read from"
    92  CommandProcessor ||--|{ State: "read from"
    93  QueryProcessor ||--|{ State: "read from"
    94  State }|--|{ ViewRecordsStorage: "views read by"
    95  State }|--|{ RecordsStorage: "tables read by"
    96  Actualizer ||--|| Projector: feeds
    97  Projector }|--|{ Projection: "updates"
    98  Projection ||--|| View: "can be"
    99  Projection ||--|| Table: "can be"
   100  Projection ||--|| WLog: "can be"
   101  Actualizer }|--|| WsProjectionsState: "updates"
   102  AppPartition ||--|| WsProjectionsState: has
   103  ViewRecordsStorage ||--|{ ProjectorUpdatedViews: "finds projector"
   104  ViewRecordsStorage ||--|| WsProjectionsState: "gets projection status & offset from"
   105  RecordsStorage ||--|| WsProjectionsState: "get offset from"
   106  ```
   107  
   108  ### Common
   109  - WsProjectionsState is used by State to:
   110      - Provide the requested Isolation Level for Read operations;
   111      - Start initializing [Lazy Projections](../projectors/lazy-projections.md);
   112  
   113  ### Query Processor
   114  - Isolation Level specified by header `Isolation-Level`. Possible values:
   115      - `read-uncommitted`
   116      - `read-committed[;offset]`
   117      - `snapshot[;offset]`
   118  - Configures State to use the Isolation Level, as specified in request
   119  - Error 503 when reading from inconsistent projection
   120  
   121  ### Projectors & Actualizers
   122  
   123  - All actualizers are asynchronous
   124  - Projector's attempt to read from inconsistent projection throws Error 503;
   125  - Actualizer updates WsProjectionsState(Status[IDLE, RAW, ACTIVE], WlogOffset) when the intents and projections offsets are flushed
   126  - WsProjectionsState is used by Actualizer to:
   127      - Handle idempotency out of the box (do not feed event if it has been already fed);
   128  - Projector declares:
   129      - Which VIEWS it updates. This is needed to find out projection state in WsProjectionsState by the View QName, State is going to read from (`ProjectorUpdatedViews`).
   130      - Which VIEWS it reads from. This is needed to avoid concurrent reads. Example:
   131          - Projector1 reads from V2, updates V1
   132          - Projector2 reads from V1, updates V2
   133          - This causes deadlock, because reading from V2 requires V2 to be updated by the same event, which cannot be done until reading from V1 is done.
   134  
   135  ### Command Processor
   136  - Command Processor always reads with Isolation Level "Read Commited", and offset = last WLog Offset, means that the projection must be updated with previous event for this workspace.
   137  - If a projection is not consistent, 503 is thrown immediately.
   138  - Commands Processor do not inserts records. It only saves validated `CUDs` in the event (a special CUD storage?). The records are inserted/updated by system-defined projector.
   139  
   140  ## All Actualizers are Asynchronous
   141  A new notification mechanism between CP and AA, ref. https://github.com/heeus/inv-wasm/tree/master/20220828-sync-projectors
   142  
   143  
   144  ## See Also
   145  - [Isolation levels in SQL Server](https://www.sqlservercentral.com/articles/isolation-levels-in-sql-server)
   146  - [Understanding the isolation levels](https://learn.microsoft.com/en-us/sql/connect/jdbc/understanding-isolation-levels?view=sql-server-ver16)
   147  - [Query Processor](../queryprocessor/)