github.com/ethereum-optimism/optimism@v1.7.2/op-node/cmd/batch_decoder/README.md (about)

     1  # Batch Decoding Tool
     2  
     3  The batch decoding tool is a utility to aid in debugging the batch submitter & the op-node
     4  by looking at what batches were submitted on L1.
     5  
     6  ## Design Philosophy
     7  
     8  The `batch_decoder` tool is designed to be simple & flexible. It offloads as much data analysis
     9  as possible to other tools. It is built around manipulating JSON on disk. The first stage is to
    10  fetch all transactions which are sent to a batch inbox address. Those transactions are decoded into
    11  frames in that step & information about them is recorded. After transactions are fetched the frames
    12  are re-assembled into channels in a second step that does not touch the network.
    13  
    14  
    15  ## Commands
    16  
    17  ### Fetch
    18  
    19  `batch_decoder fetch` pulls all L1 transactions sent to the batch inbox address in a given L1 block
    20  range and then stores them on disk to a specified path as JSON files where the name of the file is
    21  the transaction hash.
    22  
    23  ### Reassemble
    24  
    25  `batch_decoder reassemble` goes through all of the found frames in the cache & then turns them
    26  into channels. It then stores the channels with metadata on disk where the file name is the Channel ID.
    27  Each channel can contain multiple batches.
    28  
    29  If the batch is span batch, `batch_decoder` derives span batch using `L2BlockTime`, `L2GenesisTime`, and `L2ChainID`.
    30  These arguments can be provided to the binary using flags.
    31  
    32  If the batch is a singular batch, `batch_decoder` does not derive and stores the batch as is.
    33  
    34  ### Force Close
    35  
    36  `batch_decoder force-close` will create a transaction data that can be sent from the batcher address to
    37  the batch inbox address which will force close the given channels. This will allow future channels to
    38  be read without waiting for the channel timeout. It uses the results from `batch_decoder fetch` to
    39  create the close transaction because the transaction it creates for a specific channel requires information
    40  about if the channel has been closed or not. If it has been closed already but is missing specific frames
    41  those frames need to be generated differently than simply closing the channel.
    42  
    43  
    44  ## JQ Cheat Sheet
    45  
    46  `jq` is a really useful utility for manipulating JSON files.
    47  
    48  ```
    49  # Pretty print a JSON file
    50  jq . $JSON_FILE
    51  
    52  # Print the number of valid & invalid transactions
    53  jq .valid_data $TX_DIR/* | sort | uniq -c
    54  
    55  # Select all transactions that have invalid data & then print the transaction hash
    56  jq "select(.valid_data == false)|.tx.hash" $TX_DIR
    57  
    58  # Select all channels that are not ready and then get the id and inclusion block & tx hash of the first frame.
    59  jq "select(.is_ready == false)|[.id, .frames[0].inclusion_block, .frames[0].transaction_hash]"  $CHANNEL_DIR
    60  
    61  # Show all of the frames in a channel without seeing the batches or frame data
    62  jq 'del(.batches)|del(.frames[]|.frame.data)' $CHANNEL_FILE
    63  
    64  # Show all batches (without timestamps) in a channel
    65  jq '.batches|del(.[]|.Transactions)' $CHANNEL_FILE
    66  ```
    67  
    68  
    69  ## Roadmap
    70  
    71  - Pull the batches out of channels & store that information inside the ChannelWithMetadata (CLI-3565)
    72  	- Transaction Bytes used
    73  	- Total uncompressed (different from tx bytes) + compressed bytes
    74  - Invert ChannelWithMetadata so block numbers/hashes are mapped to channels they are submitted in (CLI-3560)