github.com/ledgerwatch/erigon-lib@v1.0.0/kv/temporal/historyv2/readme.md (about)

     1  #Changesets encoding
     2  ## Storage changeset encoding 
     3  Storage encoding contains several blocks: Address hashes, Incarnations, Length of values, Values. AccountChangeSet is serialized in the following manner in order to facilitate binary search
     4  ### Address hashes
     5  There are a lot of address hashes duplication in storage changeset when we have multiple changes in one contract. To avoid it we can store only unique address hashes.
     6  First 4 bytes contains number or unique contract hashes in one changeset.
     7  Then we store address hashes with sum of key hashes from first element.
     8  
     9  For example: for `addrHash1Inc1Key1, addrHash1Inc1Key2, addrHash2Inc1Key1, addrHash2Inc1Key3` it stores
    10  `2,addrHash1,2,addrHash2,4`  
    11  ### Incarnations
    12  Currently, there are a few not default incarnations(!=1) in current state. That was the reason why we store incarnation only if it's not equal to fffffffe(inverted 1).
    13  First part is 4 byte that contains number of not default incarnations
    14  Then we store array of id of address hash(4 byte) plus incarnation(8 byte)
    15  For example: for `addrHash1fffffffe..., addrHash1fffffffd...` it stores
    16  `1,1,fffffffd`
    17  
    18  ### Values lengths
    19  The default value length is 32(common.Hash), but if we remove leading 0 then average size became ~7. Because a length of value could be from 0 to 32 we need this section to be able to find quite fast value by key.
    20  It is contiguous array of accumulating value indexes like `len(val0), len(val0)+len(val1), ..., len(val0)+len(val1)+...+len(val_{N-1})`
    21  To reduce cost of it we have three numbers: numOfUint8, numOfUint16, numOfUint32. They can answer to the question: How many lengths of values we can put to uint8, uint16, uint32.
    22  This number could be huge if one of the contracts was suicided during block execution. Then we could have thousands of empty values, and we are able to store them in uint8(but it depends).
    23  For example for values: "ffa","","faa" it stores `3,0,0,3,3,6`
    24  
    25  ### Values
    26  Contiguous array of values.
    27  
    28  ### Finally
    29  Value | Type | Comment
    30  ------------ | ------------- | -------------
    31  numOfUniqueElements | uint32 |
    32  Address hashes | [numOfUniqueElements]{[32]byte+[4]byte}  | [numOfUniqueElements](common.Hash + uint32) 
    33  numOfNotDefaultIncarnations | uint32 | mostly - 0
    34  Incarnations |  [numOfNotDefaultIncarnations]{[4]byte + [8]byte}  | []{idOfAddrHash(uint32) + incarnation(uint64)}
    35  Keys | [][32]byte | []common.Hash
    36  numOfUint8 | uint32 | 
    37  numOfUint16 | uint32 |  
    38  numOfUint32 | uint32 | 
    39  Values lengths in uint8 | [numOfUint8]uint8 | 
    40  Values lengths in uint16 | [numOfUint16]uint16 | 
    41  Values lengths in uint32 | [numOfUint32]uint32 | 
    42  Values | [][]byte | 
    43  
    44  
    45  
    46  
    47  ## Account changeset encoding
    48  AccountChangeSet is serialized in the following manner in order to facilitate binary search. Account changeset encoding contains several blocks: Keys, Length of values, Values. Key is address hash of account. Value is CBOR encoded account without storage root and code hash.
    49  
    50  ### Keys
    51  The number of keys N (uint32, 4 bytes)
    52  Contiguous array of keys (N*32 bytes)
    53  ### Values lengthes
    54   Contiguous array of accumulating value indexes:
    55  len(val0), len(val0)+len(val1), ..., len(val0)+len(val1)+...+len(val_{N-1})
    56  (4*N bytes since the lengths are treated as uint32).
    57  ### Values
    58  Contiguous array of values.
    59  ### Finally
    60  Value | Type | Comment
    61  ------------ | ------------- | -------------
    62  num of keys | uint32 |
    63  address hashes | [num of keys][32]byte | [num of keys]common.Hash
    64  values lengthes | [num of keys]uint32
    65  values | [num of keys][]byte