code.vegaprotocol.io/vega@v0.79.0/core/snapshot/doc.go (about)

     1  // Copyright (C) 2023 Gobalsky Labs Limited
     2  //
     3  // This program is free software: you can redistribute it and/or modify
     4  // it under the terms of the GNU Affero General Public License as
     5  // published by the Free Software Foundation, either version 3 of the
     6  // License, or (at your option) any later version.
     7  //
     8  // This program is distributed in the hope that it will be useful,
     9  // but WITHOUT ANY WARRANTY; without even the implied warranty of
    10  // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    11  // GNU Affero General Public License for more details.
    12  //
    13  // You should have received a copy of the GNU Affero General Public License
    14  // along with this program.  If not, see <http://www.gnu.org/licenses/>.
    15  
    16  package snapshot
    17  
    18  // Package snapshot implements the snapshot engine responsible of taking a
    19  // snapshot of the node state, that will allow a node to restore the state later
    20  // on without replaying the entire chain.
    21  //
    22  // # Taking a snapshot
    23  //
    24  // The engine gathers the state from the state providers that subscribed to it.
    25  // A state provider can be any component that needs to have its state snapshotted
    26  // and restored to work.
    27  //
    28  // A snapshot is taken at defined interval counted in "block". For example,
    29  // every 10 blocks. It happens when Tendermint calls `Commit()` on the node.
    30  //
    31  // The snapshot is taken asynchronously on the providers.
    32  //
    33  // # Restoration modes
    34  //
    35  // There are 2 ways to load a snapshot:
    36  //   - Loading from a local snapshot, stored on disk,
    37  //   - Loading from the network via Tendermint state-sync.
    38  //
    39  // Once a snapshot is loaded by it engine, the engine distributes the deserialized
    40  // state to the state providers. Once the state is restored, the engine will
    41  // reject any new attempt to restore another state. The node will have to be
    42  // shutdown and reset to apply another state.
    43  //
    44  // ## Restore from local snapshots
    45  //
    46  // Local snapshots are stored locally in a LevelDB database. Snapshots metadata
    47  // are stored in a separate database to go easy on machine resources when looking
    48  // for high-level information, that avoid loading the entire snapshot in memory.
    49  //
    50  // If there is any snapshot stored locally, the snapshot engine will load from
    51  // the last one (or the one matching the configured block height), regardless of
    52  // whether Tendermint's state-sync is enabled or not.
    53  //
    54  // To prevent this loading from this mode, the local snapshots must be entirely
    55  // removed. This can be achieved using the command-line `vega unsafe-reset-all`.
    56  //
    57  // ## Restore from state-sync
    58  //
    59  // Restoring from this mode requires that no snapshot is stored locally.
    60  //
    61  // If there is no local snapshot, the snapshot engine will wait for Tendermint to
    62  // offer a snapshot, through the method (*Engine).ReceiveSnapshot(). The snapshot
    63  // will have to match the expectation of the engine, and if it does, it listens
    64  // for incoming snapshot chunks distributed by network peers, through the
    65  // method (*Engine).ReceiveSnapshotChunk().
    66  //
    67  // If Tendermint fails to retrieve all the chunks, it will automatically abort the
    68  // on-going state-sync, and offer another snapshot to the engine, and go all over
    69  // the chunk distribution process, again. The engine reacts to that change accordingly
    70  // by resetting the process.
    71  //
    72  // When the snapshot is completed, the engine restores the state, and save the
    73  // received snapshot locally.
    74  //
    75  // # Sharing snapshots with peers
    76  //
    77  // To share snapshots with peers, the node must have a snapshot saved locally.
    78  // Tendermint asks which snapshots the node currently have via the method
    79  // (*Engine).ListSnapshots(). If the snapshot Tendermint is looking for is present,
    80  // it then asks for chunks of that snapshot. The node have to load the snapshot in
    81  // memory, and look for the asked chunk.