github.com/ari-anchor/sei-tendermint@v0.0.0-20230519144642-dc826b7b56bb/docs/tools/debugging/README.md (about)

     1  ---
     2  order: 1
     3  parent:
     4    title: Debugging
     5    order: 1
     6  ---
     7  
     8  # Debugging
     9  
    10  ## Tendermint debug kill
    11  
    12  Tendermint comes with a `debug` sub-command that allows you to kill a live
    13  Tendermint process while collecting useful information in a compressed archive.
    14  The information includes the configuration used, consensus state, network
    15  state, the node' status, the WAL, and even the stack trace of the process
    16  before exit. These files can be useful to examine when debugging a faulty
    17  Tendermint process.
    18  
    19  ```bash
    20  tendermint debug kill <pid> </path/to/out.zip> --home=</path/to/app.d>
    21  ```
    22  
    23  will write debug info into a compressed archive. The archive will contain the
    24  following:
    25  
    26  ```sh
    27  ├── config.toml
    28  ├── consensus_state.json
    29  ├── net_info.json
    30  ├── stacktrace.out
    31  ├── status.json
    32  └── wal
    33  ```
    34  
    35  Under the hood, `debug kill` fetches info from `/status`, `/net_info`, and
    36  `/dump_consensus_state` HTTP endpoints, and kills the process with `-6`, which
    37  catches the go-routine dump.
    38  
    39  ## Tendermint debug dump
    40  
    41  Also, the `debug dump` sub-command allows you to dump debugging data into
    42  compressed archives at a regular interval. These archives contain the goroutine
    43  and heap profiles in addition to the consensus state, network info, node
    44  status, and even the WAL.
    45  
    46  ```bash
    47  tendermint debug dump </path/to/out> --home=</path/to/app.d>
    48  ```
    49  
    50  will perform similarly to `kill` except it only polls the node and
    51  dumps debugging data every frequency seconds to a compressed archive under a
    52  given destination directory. Each archive will contain:
    53  
    54  ```sh
    55  ├── consensus_state.json
    56  ├── goroutine.out
    57  ├── heap.out
    58  ├── net_info.json
    59  ├── status.json
    60  └── wal
    61  ```
    62  
    63  Note: goroutine.out and heap.out will only be written if a profile address is
    64  provided and is operational. This command is blocking and will log any error.
    65  
    66  ## Tendermint Inspect
    67  
    68  Tendermint includes an `inspect` command for querying Tendermint's state store and block
    69  store over Tendermint RPC.
    70  
    71  When the Tendermint consensus engine detects inconsistent state, it will crash the
    72  entire Tendermint process. 
    73  While in this inconsistent state, a node running Tendermint's consensus engine will not start up. 
    74  The `inspect` command runs only a subset of Tendermint's RPC endpoints for querying the block store
    75  and state store. 
    76  `inspect` allows operators to query a read-only view of the stage.
    77  `inspect` does not run the consensus engine at all and can therefore be used to debug
    78  processes that have crashed due to inconsistent state. 
    79  
    80  
    81  To start the `inspect` process, run
    82  ```bash
    83  tendermint inspect
    84  ```
    85  
    86  ### RPC endpoints
    87  The list of available RPC endpoints can be found by making a request to the RPC port.
    88  For an `inspect` process running on `127.0.0.1:26657`, navigate your browser to 
    89  `http://127.0.0.1:26657/` to retrieve the list of enabled RPC endpoints.
    90  
    91  Additional information on the Tendermint RPC endpoints can be found in the [rpc documentation](https://docs.tendermint.com/master/rpc).