github.com/ari-anchor/sei-tendermint@v0.0.0-20230519144642-dc826b7b56bb/docs/tools/debugging/README.md (about) 1 --- 2 order: 1 3 parent: 4 title: Debugging 5 order: 1 6 --- 7 8 # Debugging 9 10 ## Tendermint debug kill 11 12 Tendermint comes with a `debug` sub-command that allows you to kill a live 13 Tendermint process while collecting useful information in a compressed archive. 14 The information includes the configuration used, consensus state, network 15 state, the node' status, the WAL, and even the stack trace of the process 16 before exit. These files can be useful to examine when debugging a faulty 17 Tendermint process. 18 19 ```bash 20 tendermint debug kill <pid> </path/to/out.zip> --home=</path/to/app.d> 21 ``` 22 23 will write debug info into a compressed archive. The archive will contain the 24 following: 25 26 ```sh 27 ├── config.toml 28 ├── consensus_state.json 29 ├── net_info.json 30 ├── stacktrace.out 31 ├── status.json 32 └── wal 33 ``` 34 35 Under the hood, `debug kill` fetches info from `/status`, `/net_info`, and 36 `/dump_consensus_state` HTTP endpoints, and kills the process with `-6`, which 37 catches the go-routine dump. 38 39 ## Tendermint debug dump 40 41 Also, the `debug dump` sub-command allows you to dump debugging data into 42 compressed archives at a regular interval. These archives contain the goroutine 43 and heap profiles in addition to the consensus state, network info, node 44 status, and even the WAL. 45 46 ```bash 47 tendermint debug dump </path/to/out> --home=</path/to/app.d> 48 ``` 49 50 will perform similarly to `kill` except it only polls the node and 51 dumps debugging data every frequency seconds to a compressed archive under a 52 given destination directory. Each archive will contain: 53 54 ```sh 55 ├── consensus_state.json 56 ├── goroutine.out 57 ├── heap.out 58 ├── net_info.json 59 ├── status.json 60 └── wal 61 ``` 62 63 Note: goroutine.out and heap.out will only be written if a profile address is 64 provided and is operational. This command is blocking and will log any error. 65 66 ## Tendermint Inspect 67 68 Tendermint includes an `inspect` command for querying Tendermint's state store and block 69 store over Tendermint RPC. 70 71 When the Tendermint consensus engine detects inconsistent state, it will crash the 72 entire Tendermint process. 73 While in this inconsistent state, a node running Tendermint's consensus engine will not start up. 74 The `inspect` command runs only a subset of Tendermint's RPC endpoints for querying the block store 75 and state store. 76 `inspect` allows operators to query a read-only view of the stage. 77 `inspect` does not run the consensus engine at all and can therefore be used to debug 78 processes that have crashed due to inconsistent state. 79 80 81 To start the `inspect` process, run 82 ```bash 83 tendermint inspect 84 ``` 85 86 ### RPC endpoints 87 The list of available RPC endpoints can be found by making a request to the RPC port. 88 For an `inspect` process running on `127.0.0.1:26657`, navigate your browser to 89 `http://127.0.0.1:26657/` to retrieve the list of enabled RPC endpoints. 90 91 Additional information on the Tendermint RPC endpoints can be found in the [rpc documentation](https://docs.tendermint.com/master/rpc).