github.com/m3db/m3@v1.5.1-0.20231129193456-75a402aa583b/src/dbnode/storage/README.md (about)

     1  # storage
     2  
     3  Storage related documentation.
     4  
     5  ## Flush consistency model
     6  
     7  Flush occurs in the following steps:
     8    - data warm flush
     9    - rotate commit log
    10    - data cold flush
    11      - rotate cold mutable index segments
    12      - flush cold tsdb data and write most files to disk (except checkpoint files)
    13      - flush cold index data to disk and reload
    14      - evict rotated cold mutable index segments
    15      - write tsdb checkpoint files (completes the tsdb cold flush lifecycle)
    16    - data snapshot
    17      - drops rotated commit log when we are done
    18    - index flush
    19  
    20  Since we rotate the commit log before we perform a data cold flush and only drop the rotate commit log after data snapshotting is done we guarantee that no writes will be lost if the node crashes. After data cold flush completes, any new cold writes will exist in the active commit log (and not be dropped) when data snapshotting finishes. This is why data snapshotting only needs to snapshot warm data blocks (that need to be flushed).