github.com/m3db/m3@v1.5.0/src/dbnode/storage/series/README.md (about)

     1  # series
     2  
     3  Series related documentation.
     4  
     5  ## Series flush lifecycle
     6  
     7  Warm/cold writes end up in versioned buckets based on write type (`ColdWrite` or `WarmWrite`). When flushes occur, we fetch in-mem data from all write type specific buckets to persist.
     8  
     9  For warm flushes, we write all warm written buckets to disk and mark the state of the block as `WarmRetrievable`.
    10  
    11  For cold flushes, we merge this data w/ data that's already on disk in `fs/merger.go` and write to disk. Once finished, we then update the `ColdVersionRetrievable` to the cold version we just wrote to disk.
    12  
    13  Data is only evicted from mem during a `Tick()`. This evicts either cold buckets up until flush state `ColdVersionRetrievable` or warm buckets that are marked as `WarmRetrievable` (or warm blocks that we have already warm flushed to disk).
    14  
    15  ## Snapshotting/Bootstrap
    16  
    17  Snapshots work by merging all buckets for a series buffer regardless of write type into streams and persisting to disk. Snapshots are in the commitlog bootstrapper and snapshotted series data are loaded into `BufferBucket.loadedBlocks`. Attempts to call `series.LoadBlock()` for `WarmWrite` blocks will return an error if it already exists on disk.
    18  
    19  Series snapshots persist writes in both warm & cold buckets. During a flush, we persist snapshot files w/ a commit log ID. This ID is later used during the async cleanup process to deleted rotated commit logs.
    20  
    21  ## Repair
    22  
    23  Shard repairs load data as cold writes into series buffer buckets.