github.com/lazyledger/lazyledger-core@v0.35.0-dev.0.20210613111200-4c651f053571/docs/lazy-adr/adr-001-block-propagation.md (about)

     1  # ADR 001: Erasure Coding Block Propagation
     2  
     3  ## Changelog
     4  
     5  - 16-2-2021: Created
     6  
     7  ## Context
     8  
     9  Block propagation is currently done by splitting the block into arbitrary chunks and gossiping them to validators via a gossip routine. While this does not have downsides it does not meet the needs of the lazy-ledger chain. The lazyledger chain requires blocks to be encoded in a different way and for the proposer to not propagate the chunks to peers.
    10  
    11  Lazyledger wants validators to pull the block from a IPFS network. What does this mean? As I touched on earlier the proposer pushes the block to the network, this in turn means that each validator downloads and reconstructs the block each time to verify it. Instead Lazyledger will encode and split up the block via erasure codes, stored locally in the nodes IPFS daemon. After the proposer has sent the block to IPFS and received the CIDs it will include them into the proposal. This proposal will be gossiped to other validators, once a validator receives the proposal it will begin requesting the CIDs included in the proposal.
    12  
    13  There are two forms of a validator, one that downloads the block and one that samples it. What does sampling mean? Sampling is the act of checking that a portion or entire block is available for download.
    14  
    15  ## Detailed Design
    16  
    17  The proposed design is as follows.
    18  
    19  ### Types
    20  
    21  The proposal and vote types have a BlockID, this will be replaced with a header hash. The proposal will contain add fields.
    22  
    23  The current proposal will be updated to include required fields. The entirety of the message will be reworked at a later date. To see the extent of the needed changes you can visit the [spec repo](https://github.com/lazyledger/lazyledger-specs/blob/master/specs/proto/consensus.proto#L19)
    24  
    25  ```proto
    26  message Proposal {
    27    SignedMsgType             type      = 1;
    28    int64                     height    = 2;
    29    int32                     round     = 3;
    30    int32                     pol_round = 4;
    31  
    32    +++
    33      // 32-byte hash
    34    bytes last_header_hash = 5;
    35    // 32-byte hash
    36    bytes last_commit_hash = 6;
    37      // 32-byte hash
    38    bytes consensus_root = 7;
    39    FeeHeader fee_header = 8;
    40    // 32-byte hash
    41    bytes state_commitment = 9;
    42    uint64 available_data_original_shares_used = 10;
    43    AvailableDataHeader available_data_header = 11;
    44    +++
    45  
    46    google.protobuf.Timestamp timestamp = 12
    47        [(gogoproto.nullable) = false, (gogoproto.stdtime) = true];
    48    bytes signature = 12;
    49  }
    50  ```
    51  
    52  ```proto
    53  // Vote represents a prevote, precommit, or commit vote from validators for
    54  // consensus.
    55  message Vote {
    56    SignedMsgType type     = 1;
    57    int64         height   = 2;
    58    int32         round    = 3;
    59    +++
    60    bytes header_hash      = 4;
    61    +++
    62    google.protobuf.Timestamp timestamp = 5
    63        [(gogoproto.nullable) = false, (gogoproto.stdtime) = true];
    64    bytes validator_address = 6;
    65    int32 validator_index   = 7;
    66    bytes signature         = 8;
    67  }
    68  ```
    69  
    70  See [specs](https://github.com/lazyledger/lazyledger-specs/blob/master/specs/data_structures.md#vote) for more details on the vote.
    71  
    72  ### Disk Storage
    73  
    74  Currently Lazyledger-core stores all blocks in its store. Going forward only the headers of the blocks within the unbonding period will be stored. This will drastically reduce the amount of storage required by a lazyledger-core node. After the unbonding period all headers will have the option of being pruned.
    75  
    76  Proposed amendment to `BlockStore` interface
    77  
    78  ```go
    79  type BlockStore interface {
    80   Base() int64
    81   Height() int64
    82   Size() int64
    83  
    84   LoadBlockMeta(height int64) *types.BlockMeta
    85   LoadHeader(height int64) *types.Header
    86   LoadDAHeader(height int64) *types.DataAvailabilityHeader
    87  
    88   SaveHeaders(header *types.Header, daHeader *types.DataAvailabilityHeader, seenCommit *types.Commit)
    89  
    90   PruneHeaders(height int64) (uint64, error)
    91  
    92   LoadBlockCommit(height int64) *types.Commit
    93   LoadSeenCommit(height int64) *types.Commit
    94  }
    95  ```
    96  
    97  Along side these changes the rpc layer will need to change. Instead of querying the LL-core store, the node will redirect the query through IPFS.
    98  
    99  Example:
   100  
   101  When a user requests a block from the LL node, the request will be set to the IPLD plugin. If the IPLD does not have the requested block, it will make a request to the lazyledger IPFS network for the required CIDs. If the full node does not have the DAheader they will not be able to request the block data.
   102  
   103  ![user request flow](./assets/user-request.png)
   104  
   105  The goal is to not change the public interface for RPC's. It is yet to be seen if this possible. This means that CIDs will need to be set and loaded from the store in order to get all the related block information an user requires.
   106  
   107  ## Status
   108  
   109  Proposed
   110  
   111  
   112  ### Positive
   113  
   114  - Minimal breakage to public interface
   115  - Only store the block in a single place (IPFS)
   116  - Reduce the public interface of the storage within LazyLedger.
   117  
   118  ### Negative
   119  
   120  - User requests may take more time to process
   121  
   122  ### Neutral
   123  
   124  ## References