github.com/alexeirbv/blockbook@v0.3.2/docs/rocksdb.md (about)

     1  # Data storage in RocksDB
     2  
     3  **Blockbook** stores data the key-value store [RocksDB](https://github.com/facebook/rocksdb/wiki). As there are multiple indexes, Blockbook uses RocksDB **column families** feature to store indexes separately.
     4  
     5  >The database structure is described in golang pseudo types in the form *(name type)*. 
     6  >
     7  >Operators used in the description: 
     8  >- *->* mapping from key to value. 
     9  >- *\+* concatenation, 
    10  >- *[]* array
    11  >
    12  >Types used in the description:
    13  >- *[]byte* - variable length array of bytes
    14  >- *[32]byte* - fixed length array of bytes (32 bytes long in this case)
    15  >- *uint32* - unsigned integer, stored as array of 4 bytes in big endian*
    16  >- *vint*, *vuint* - variable length signed/unsigned int
    17  >- *addrDesc* - address descriptor, abstraction of an address.
    18  For Bitcoin type coins it is the transaction output script, stored as variable length array of bytes. 
    19  For Ethereum type coins it is fixed size array of 20 bytes.
    20  >- *bigInt* - unsigned big integer, stored as length of the array (1 byte) followed by array of bytes of big int, i.e. *(int_len byte)+(int_value []byte)*. Zero is stored as one byte of value 0.
    21  
    22  **Database structure:**
    23  
    24  The database structure described here is of Blockbook version **0.3.2** (internal data format version 5). 
    25  
    26  The database structure for **Bitcoin type** and **Ethereum type** coins is slightly different. Column families used for both types:
    27  - default, height, addresses, transactions, blockTxs
    28  
    29  Column families used only by **Bitcoin type** coins:
    30  - addressBalance, txAddresses
    31  
    32  Column families used only by **Ethereum type** coins:
    33  - addressContracts
    34  
    35  **Column families description:**
    36  
    37  - **default**
    38  
    39    Stores internal state in json format, under the key *internalState*. 
    40    
    41    Most important internal state values are:
    42    - coin - which coin is indexed in DB
    43    - data format version - currently 5
    44    - dbState - closed, open, inconsistent
    45      
    46    Blockbook is checking on startup these values and does not allow to run against wrong coin, data format version and in inconsistent state. The database must be recreated if the internal state does not match.
    47  
    48  - **height** 
    49  
    50      Maps *block height* to *block hash* and additional data about block.
    51      ```
    52      (height uint32) -> (hash [32]byte)+(time uint32)+(nr_txs vuint)+(size vuint)
    53      ```
    54  
    55  - **addresses**
    56  
    57      Maps *addrDesc+block height* to *array of transactions with array of input/output indexes*.
    58      
    59      The *block height* in the key is stored as bitwise complement ^ of the height to sort the keys in the order from newest to oldest.
    60      
    61      As there can be multiple inputs/outputs for the same address in one transaction, each txid is followed by variable length array of input/output indexes.
    62      The index values in the array are multiplied by two, the last element of the array has the lowest bit set to 1.
    63      Input or output is distinguished by the sign of the index, output is positive, input is negative (by operation bitwise complement ^ performed on the number).   
    64      ```
    65      (addrDesc []byte)+(^height uint32) -> []((txid [32]byte)+[](index vint))
    66      ```
    67  
    68  - **addressBalance** (used only by Bitcoin type coins)
    69  
    70      Maps *addrDesc* to *number of transactions*, *sent amount*, *total balance* and a list of *unspent transactions outputs (UTXOs)*, ordered from oldest to newest
    71      ```
    72      (addrDesc []byte) -> (nr_txs vuint)+(sent_amount bigInt)+(balance bigInt)+
    73                           []((txid [32]byte)+(vout vuint)+(block_height vuint)+(amount bigInt))
    74      ```
    75  
    76  - **txAddresses** (used only by Bitcoin type coins)
    77  
    78      Maps *txid* to *block height* and array of *input addrDesc* with *amounts* and array of *output addrDesc* with *amounts*, with flag if output is spent. In case of spent output, *addrDesc_len* is negative (negative sign is achieved by bitwise complement ^).
    79      ```
    80      (txid []byte) -> (height vuint)+
    81                       (nr_inputs vuint)+[]((addrDesc_len vuint)+(addrDesc []byte)+(amount bigInt))+
    82                       (nr_outputs vuint)+[]((addrDesc_len vint)+(addrDesc []byte)+(amount bigInt))
    83      ```
    84  
    85  - **addressContracts** (used only by Ethereum type coins)
    86  
    87      Maps *addrDesc* to *total number of transactions*, *number of non contract transactions* and array of *contracts* with *number of transfers* of given address.
    88      ```
    89      (addrDesc []byte) -> (total_txs vuint)+(non-contract_txs vuint)+[]((contractAddrDesc []byte)+(nr_transfers vuint))
    90      ```
    91  
    92  - **blockTxs**
    93  
    94      Maps *block height* to data necessary for blockchain rollback. Only last 300 (by default) blocks are kept. 
    95      The content of value data differs for Bitcoin and Ethereum types.
    96  
    97      - Bitcoin type
    98  
    99      The value is an array of *txids* and *input points* in the block.
   100      ```
   101      (height uint32) -> []((txid [32]byte)+(nr_inputs vuint)+[]((txid [32]byte)+(index vint)))
   102      ```
   103  
   104      - Ethereum type
   105      
   106      The value is an array of transaction data. For each transaction is stored *txid*,
   107       *from* and *to* address descriptors and array of *contract address descriptors* with *transfer address descriptors*.
   108      ```
   109      (height uint32) -> []((txid [32]byte)+(from addrDesc)+(to addrDesc)+(nr_contracts vuint)+[]((contract addrDesc)+(addr addrDesc)))
   110      ```
   111  
   112  - **transactions**
   113  
   114      Transaction cache, *txdata* is generated by coin specific parser function PackTx.
   115      ```
   116      (txid []byte) -> (txdata []byte)
   117      ```
   118  
   119  - **fiatRates**
   120  
   121      Stores fiat rates in json format.
   122      ```
   123      (timestamp YYYYMMDDhhmmss) -> (rates json)
   124      ```
   125  
   126  
   127  The `txid` field as specified in this documentation is a byte array of fixed size with length 32 bytes (*[32]byte*), however some coins may define other fixed size lengths.