github.com/KYVENetwork/cometbft/v38@v38.0.3/spec/abci/abci++_app_requirements.md (about)

     1  ---
     2  order: 3
     3  title: Requirements for the Application
     4  ---
     5  
     6  # Requirements for the Application
     7  
     8  - [Requirements for the Application](#requirements-for-the-application)
     9      - [Formal Requirements](#formal-requirements)
    10          - [Consensus Connection Requirements](#consensus-connection-requirements)
    11          - [Mempool Connection Requirements](#mempool-connection-requirements)
    12      - [Managing the Application state and related topics](#managing-the-application-state-and-related-topics)
    13          - [Connection State](#connection-state)
    14              - [Concurrency](#concurrency)
    15              - [FinalizeBlock](#finalizeblock)
    16              - [Commit](#commit)
    17              - [Candidate States](#candidate-states)
    18          - [States and ABCI++ Connections](#states-and-abci-connections)
    19              - [Consensus Connection](#consensus-connection)
    20              - [Mempool Connection](#mempool-connection)
    21                  - [Replay Protection](#replay-protection)
    22              - [Info/Query Connection](#infoquery-connection)
    23              - [Snapshot Connection](#snapshot-connection)
    24          - [Transaction Results](#transaction-results)
    25              - [Gas](#gas)
    26              - [Specifics of `ResponseCheckTx`](#specifics-of-responsechecktx)
    27              - [Specifics of `ExecTxResult`](#specifics-of-exectxresult)
    28          - [Updating the Validator Set](#updating-the-validator-set)
    29          - [Consensus Parameters](#consensus-parameters)
    30              - [List of Parameters](#list-of-parameters)
    31                  - [ABCIParams.VoteExtensionsEnableHeight](#abciparamsvoteextensionsenableheight)
    32                  - [BlockParams.MaxBytes](#blockparamsmaxbytes)
    33                  - [BlockParams.MaxGas](#blockparamsmaxgas)
    34                  - [EvidenceParams.MaxAgeDuration](#evidenceparamsmaxageduration)
    35                  - [EvidenceParams.MaxAgeNumBlocks](#evidenceparamsmaxagenumblocks)
    36                  - [EvidenceParams.MaxBytes](#evidenceparamsmaxbytes)
    37                  - [ValidatorParams.PubKeyTypes](#validatorparamspubkeytypes)
    38                  - [VersionParams.App](#versionparamsapp)
    39              - [Updating Consensus Parameters](#updating-consensus-parameters)
    40                  - [`InitChain`](#initchain)
    41                  - [`FinalizeBlock`, `PrepareProposal`/`ProcessProposal`](#finalizeblock-prepareproposalprocessproposal)
    42          - [`Query`](#query)
    43              - [Query Proofs](#query-proofs)
    44              - [Peer Filtering](#peer-filtering)
    45              - [Paths](#paths)
    46          - [Crash Recovery](#crash-recovery)
    47          - [State Sync](#state-sync)
    48              - [Taking Snapshots](#taking-snapshots)
    49              - [Bootstrapping a Node](#bootstrapping-a-node)
    50                  - [Snapshot Discovery](#snapshot-discovery)
    51                  - [Snapshot Restoration](#snapshot-restoration)
    52                  - [Snapshot Verification](#snapshot-verification)
    53                  - [Transition to Consensus](#transition-to-consensus)
    54      - [Application configuration required to switch to ABCI 2.0](#application-configuration-required-to-switch-to-abci-20)
    55  
    56  
    57  ## Formal Requirements
    58  
    59  ### Consensus Connection Requirements
    60  
    61  This section specifies what CometBFT expects from the Application. It is structured as a set
    62  of formal requirements that can be used for testing and verification of the Application's logic.
    63  
    64  Let *p* and *q* be two correct processes.
    65  Let *r<sub>p</sub>* (resp. *r<sub>q</sub>*) be a round of height *h* where *p* (resp. *q*) is the
    66  proposer.
    67  Let *s<sub>p,h-1</sub>* be *p*'s Application's state committed for height *h-1*.
    68  Let *v<sub>p</sub>* (resp. *v<sub>q</sub>*) be the block that *p*'s (resp. *q*'s) CometBFT passes
    69  on to the Application
    70  via `RequestPrepareProposal` as proposer of round *r<sub>p</sub>* (resp *r<sub>q</sub>*), height *h*,
    71  also known as the raw proposal.
    72  Let *u<sub>p</sub>* (resp. *u<sub>q</sub>*) the possibly modified block *p*'s (resp. *q*'s) Application
    73  returns via `ResponsePrepareProposal` to CometBFT, also known as the prepared proposal.
    74  
    75  Process *p*'s prepared proposal can differ in two different rounds where *p* is the proposer.
    76  
    77  - Requirement 1 [`PrepareProposal`, timeliness]: If *p*'s Application fully executes prepared blocks in
    78    `PrepareProposal` and the network is in a synchronous period while processes *p* and *q* are in *r<sub>p</sub>*,
    79    then the value of *TimeoutPropose* at *q* must be such that *q*'s propose timer does not time out
    80    (which would result in *q* prevoting `nil` in *r<sub>p</sub>*).
    81  
    82  Full execution of blocks at `PrepareProposal` time stands on CometBFT's critical path. Thus,
    83  Requirement 1 ensures the Application or operator will set a value for `TimeoutPropose` such that the time it takes
    84  to fully execute blocks in `PrepareProposal` does not interfere with CometBFT's propose timer.
    85  Note that the violation of Requirement 1 may lead to further rounds, but will not
    86  compromise liveness because even though `TimeoutPropose` is used as the initial
    87  value for proposal timeouts, CometBFT will be dynamically adjust these timeouts
    88  such that they will eventually be enough for completing `PrepareProposal`.
    89  
    90  - Requirement 2 [`PrepareProposal`, tx-size]: When *p*'s Application calls `ResponsePrepareProposal`, the
    91    total size in bytes of the transactions returned does not exceed `RequestPrepareProposal.max_tx_bytes`.
    92  
    93  Busy blockchains might seek to gain full visibility into transactions in CometBFT's mempool,
    94  rather than having visibility only on *a* subset of those transactions that fit in a block.
    95  The application can do so by setting `ConsensusParams.Block.MaxBytes` to -1.
    96  This instructs CometBFT (a) to enforce the maximum possible value for `MaxBytes` (100 MB) at CometBFT level,
    97  and (b) to provide *all* transactions in the mempool when calling `RequestPrepareProposal`.
    98  Under these settings, the aggregated size of all transactions may exceed `RequestPrepareProposal.max_tx_bytes`.
    99  Hence, Requirement 2 ensures that the size in bytes of the transaction list returned by the application will never
   100  cause the resulting block to go beyond its byte size limit.
   101  
   102  - Requirement 3 [`PrepareProposal`, `ProcessProposal`, coherence]: For any two correct processes *p* and *q*,
   103    if *q*'s CometBFT calls `RequestProcessProposal` on *u<sub>p</sub>*,
   104    *q*'s Application returns Accept in `ResponseProcessProposal`.
   105  
   106  Requirement 3 makes sure that blocks proposed by correct processes *always* pass the correct receiving process's
   107  `ProcessProposal` check.
   108  On the other hand, if there is a deterministic bug in `PrepareProposal` or `ProcessProposal` (or in both),
   109  strictly speaking, this makes all processes that hit the bug byzantine. This is a problem in practice,
   110  as very often validators are running the Application from the same codebase, so potentially *all* would
   111  likely hit the bug at the same time. This would result in most (or all) processes prevoting `nil`, with the
   112  serious consequences on CometBFT's liveness that this entails. Due to its criticality, Requirement 3 is a
   113  target for extensive testing and automated verification.
   114  
   115  - Requirement 4 [`ProcessProposal`, determinism-1]: `ProcessProposal` is a (deterministic) function of the current
   116    state and the block that is about to be applied. In other words, for any correct process *p*, and any arbitrary block *u*,
   117    if *p*'s CometBFT calls `RequestProcessProposal` on *u* at height *h*,
   118    then *p*'s Application's acceptance or rejection **exclusively** depends on *u* and *s<sub>p,h-1</sub>*.
   119  
   120  - Requirement 5 [`ProcessProposal`, determinism-2]: For any two correct processes *p* and *q*, and any arbitrary
   121    block *u*,
   122    if *p*'s (resp. *q*'s) CometBFT calls `RequestProcessProposal` on *u* at height *h*,
   123    then *p*'s Application accepts *u* if and only if *q*'s Application accepts *u*.
   124    Note that this requirement follows from Requirement 4 and the Agreement property of consensus.
   125  
   126  Requirements 4 and 5 ensure that all correct processes will react in the same way to a proposed block, even
   127  if the proposer is Byzantine. However, `ProcessProposal` may contain a bug that renders the
   128  acceptance or rejection of the block non-deterministic, and therefore prevents processes hitting
   129  the bug from fulfilling Requirements 4 or 5 (effectively making those processes Byzantine).
   130  In such a scenario, CometBFT's liveness cannot be guaranteed.
   131  Again, this is a problem in practice if most validators are running the same software, as they are likely
   132  to hit the bug at the same point. There is currently no clear solution to help with this situation, so
   133  the Application designers/implementors must proceed very carefully with the logic/implementation
   134  of `ProcessProposal`. As a general rule `ProcessProposal` SHOULD always accept the block.
   135  
   136  According to the Tendermint consensus algorithm, currently adopted in CometBFT,
   137  a correct process can broadcast at most one precommit
   138  message in round *r*, height *h*.
   139  Since, as stated in the [Methods](./abci++_methods.md#extendvote) section, `ResponseExtendVote`
   140  is only called when the consensus algorithm
   141  is about to broadcast a non-`nil` precommit message, a correct process can only produce one vote extension
   142  in round *r*, height *h*.
   143  Let *e<sup>r</sup><sub>p</sub>* be the vote extension that the Application of a correct process *p* returns via
   144  `ResponseExtendVote` in round *r*, height *h*.
   145  Let *w<sup>r</sup><sub>p</sub>* be the proposed block that *p*'s CometBFT passes to the Application via `RequestExtendVote`
   146  in round *r*, height *h*.
   147  
   148  - Requirement 6 [`ExtendVote`, `VerifyVoteExtension`, coherence]: For any two different correct
   149    processes *p* and *q*, if *q* receives *e<sup>r</sup><sub>p</sub>* from *p* in height *h*, *q*'s
   150    Application returns Accept in `ResponseVerifyVoteExtension`.
   151  
   152  Requirement 6 constrains the creation and handling of vote extensions in a similar way as Requirement 3
   153  constrains the creation and handling of proposed blocks.
   154  Requirement 6 ensures that extensions created by correct processes *always* pass the `VerifyVoteExtension`
   155  checks performed by correct processes receiving those extensions.
   156  However, if there is a (deterministic) bug in `ExtendVote` or `VerifyVoteExtension` (or in both),
   157  we will face the same liveness issues as described for Requirement 5, as Precommit messages with invalid vote
   158  extensions will be discarded.
   159  
   160  - Requirement 7 [`VerifyVoteExtension`, determinism-1]: `VerifyVoteExtension` is a (deterministic) function of
   161    the current state, the vote extension received, and the prepared proposal that the extension refers to.
   162    In other words, for any correct process *p*, and any arbitrary vote extension *e*, and any arbitrary
   163    block *w*, if *p*'s (resp. *q*'s) CometBFT calls `RequestVerifyVoteExtension` on *e* and *w* at height *h*,
   164    then *p*'s Application's acceptance or rejection **exclusively** depends on *e*, *w* and *s<sub>p,h-1</sub>*.
   165  
   166  - Requirement 8 [`VerifyVoteExtension`, determinism-2]: For any two correct processes *p* and *q*,
   167    and any arbitrary vote extension *e*, and any arbitrary block *w*,
   168    if *p*'s (resp. *q*'s) CometBFT calls `RequestVerifyVoteExtension` on *e* and *w* at height *h*,
   169    then *p*'s Application accepts *e* if and only if *q*'s Application accepts *e*.
   170    Note that this requirement follows from Requirement 7 and the Agreement property of consensus.
   171  
   172  Requirements 7 and 8 ensure that the validation of vote extensions will be deterministic at all
   173  correct processes.
   174  Requirements 7 and 8 protect against arbitrary vote extension data from Byzantine processes,
   175  in a similar way as Requirements 4 and 5 protect against arbitrary proposed blocks.
   176  Requirements 7 and 8 can be violated by a bug inducing non-determinism in
   177  `VerifyVoteExtension`. In this case liveness can be compromised.
   178  Extra care should be put in the implementation of `ExtendVote` and `VerifyVoteExtension`.
   179  As a general rule, `VerifyVoteExtension` SHOULD always accept the vote extension.
   180  
   181  - Requirement 9 [*all*, no-side-effects]: *p*'s calls to `RequestPrepareProposal`,
   182    `RequestProcessProposal`, `RequestExtendVote`, and `RequestVerifyVoteExtension` at height *h* do
   183    not modify *s<sub>p,h-1</sub>*.
   184  
   185  
   186  - Requirement 10 [`ExtendVote`, `FinalizeBlock`, non-dependency]: for any correct process *p*,
   187  and any vote extension *e* that *p* received at height *h*, the computation of
   188  *s<sub>p,h</sub>* does not depend on *e*.
   189  
   190  The call to correct process *p*'s `RequestFinalizeBlock` at height *h*, with block *v<sub>p,h</sub>*
   191  passed as parameter, creates state *s<sub>p,h</sub>*.
   192  Additionally, *p*'s `FinalizeBlock` creates a set of transaction results *T<sub>p,h</sub>*.
   193  
   194  - Requirement 11 [`FinalizeBlock`, determinism-1]: For any correct process *p*,
   195    *s<sub>p,h</sub>* exclusively depends on *s<sub>p,h-1</sub>* and *v<sub>p,h</sub>*.
   196  
   197  - Requirement 12 [`FinalizeBlock`, determinism-2]: For any correct process *p*,
   198    the contents of *T<sub>p,h</sub>* exclusively depend on *s<sub>p,h-1</sub>* and *v<sub>p,h</sub>*.
   199  
   200  Note that Requirements 11 and 12, combined with the Agreement property of consensus ensure
   201  state machine replication, i.e., the Application state evolves consistently at all correct processes.
   202  
   203  Also, notice that neither `PrepareProposal` nor `ExtendVote` have determinism-related
   204  requirements associated.
   205  Indeed, `PrepareProposal` is not required to be deterministic:
   206  
   207  - *u<sub>p</sub>* may depend on *v<sub>p</sub>* and *s<sub>p,h-1</sub>*, but may also depend on other values or operations.
   208  - *v<sub>p</sub> = v<sub>q</sub> &#8655; u<sub>p</sub> = u<sub>q</sub>*.
   209  
   210  Likewise, `ExtendVote` can also be non-deterministic:
   211  
   212  - *e<sup>r</sup><sub>p</sub>* may depend on *w<sup>r</sup><sub>p</sub>* and *s<sub>p,h-1</sub>*,
   213    but may also depend on other values or operations.
   214  - *w<sup>r</sup><sub>p</sub> = w<sup>r</sup><sub>q</sub> &#8655;
   215    e<sup>r</sup><sub>p</sub> = e<sup>r</sup><sub>q</sub>*
   216  
   217  ### Mempool Connection Requirements
   218  
   219  Let *CheckTxCodes<sub>tx,p,h</sub>* denote the set of result codes returned by *p*'s Application,
   220  via `ResponseCheckTx`,
   221  to successive calls to `RequestCheckTx` occurring while the Application is at height *h*
   222  and having transaction *tx* as parameter.
   223  *CheckTxCodes<sub>tx,p,h</sub>* is a set since *p*'s Application may
   224  return different result codes during height *h*.
   225  If *CheckTxCodes<sub>tx,p,h</sub>* is a singleton set, i.e. the Application always returned
   226  the same result code in `ResponseCheckTx` while at height *h*,
   227  we define *CheckTxCode<sub>tx,p,h</sub>* as the singleton value of *CheckTxCodes<sub>tx,p,h</sub>*.
   228  If *CheckTxCodes<sub>tx,p,h</sub>* is not a singleton set, *CheckTxCode<sub>tx,p,h</sub>* is undefined.
   229  Let predicate *OK(CheckTxCode<sub>tx,p,h</sub>)* denote whether *CheckTxCode<sub>tx,p,h</sub>* is `SUCCESS`.
   230  
   231  - Requirement 13 [`CheckTx`, eventual non-oscillation]: For any transaction *tx*,
   232    there exists a boolean value *b*,
   233    and a height *h<sub>stable</sub>* such that,
   234    for any correct process *p*,
   235    *CheckTxCode<sub>tx,p,h</sub>* is defined, and
   236    *OK(CheckTxCode<sub>tx,p,h</sub>) = b*
   237    for any height *h &#8805; h<sub>stable</sub>*.
   238  
   239  Requirement 13 ensures that
   240  a transaction will eventually stop oscillating between `CheckTx` success and failure
   241  if it stays in *p's* mempool for long enough.
   242  This condition on the Application's behavior allows the mempool to ensure that
   243  a transaction will leave the mempool of all full nodes,
   244  either because it is expunged everywhere due to failing `CheckTx` calls,
   245  or because it stays valid long enough to be gossipped, proposed and decided.
   246  Although Requirement 13 defines a global *h<sub>stable</sub>*, application developers
   247  can consider such stabilization height as local to process *p* (*h<sub>p,stable</sub>*),
   248  without loss for generality.
   249  In contrast, the value of *b* MUST be the same across all processes.
   250  
   251  ## Managing the Application state and related topics
   252  
   253  ### Connection State
   254  
   255  CometBFT maintains four concurrent ABCI++ connections, namely
   256  [Consensus Connection](#consensus-connection),
   257  [Mempool Connection](#mempool-connection),
   258  [Info/Query Connection](#infoquery-connection), and
   259  [Snapshot Connection](#snapshot-connection).
   260  It is common for an application to maintain a distinct copy of
   261  the state for each connection, which are synchronized upon `Commit` calls.
   262  
   263  #### Concurrency
   264  
   265  In principle, each of the four ABCI++ connections operates concurrently with one
   266  another. This means applications need to ensure access to state is
   267  thread safe. Both the
   268  [default in-process ABCI client](https://github.com/KYVENetwork/cometbft/v38/blob/v0.38.x/abci/client/local_client.go#L13)
   269  and the
   270  [default Go ABCI server](https://github.com/KYVENetwork/cometbft/v38/blob/v0.38.x/abci/server/socket_server.go#L20)
   271  use a global lock to guard the handling of events across all connections, so they are not
   272  concurrent at all. This means whether your app is compiled in-process with
   273  CometBFT using the `NewLocalClient`, or run out-of-process using the `SocketServer`,
   274  ABCI messages from all connections are received in sequence, one at a
   275  time.
   276  
   277  The existence of this global mutex means Go application developers can get thread safety for application state by routing all reads and writes through the ABCI system. Thus it may be unsafe to expose application state directly to an RPC interface, and unless explicit measures are taken, all queries should be routed through the ABCI Query method.
   278  
   279  #### FinalizeBlock
   280  
   281  When the consensus algorithm decides on a block, CometBFT uses `FinalizeBlock` to send the
   282  decided block's data to the Application, which uses it to transition its state, but MUST NOT persist it;
   283  persisting MUST be done during `Commit`.
   284  
   285  The Application must remember the latest height from which it
   286  has run a successful `Commit` so that it can tell CometBFT where to
   287  pick up from when it recovers from a crash. See information on the Handshake
   288  [here](#crash-recovery).
   289  
   290  #### Commit
   291  
   292  The Application should persist its state during `Commit`, before returning from it.
   293  
   294  Before invoking `Commit`, CometBFT locks the mempool and flushes the mempool connection. This ensures that
   295  no new messages
   296  will be received on the mempool connection during this processing step, providing an opportunity to safely
   297  update all four
   298  connection states to the latest committed state at the same time.
   299  
   300  When `Commit` returns, CometBFT unlocks the mempool.
   301  
   302  WARNING: if the ABCI app logic processing the `Commit` message sends a
   303  `/broadcast_tx_sync` or `/broadcast_tx` and waits for the response
   304  before proceeding, it will deadlock. Executing `broadcast_tx` calls
   305  involves acquiring the mempool lock that CometBFT holds during the `Commit` call.
   306  Synchronous mempool-related calls must be avoided as part of the sequential logic of the
   307  `Commit` function.
   308  
   309  #### Candidate States
   310  
   311  CometBFT calls `PrepareProposal` when it is about to send a proposed block to the network.
   312  Likewise, CometBFT calls `ProcessProposal` upon reception of a proposed block from the
   313  network. The proposed block's data
   314  that is disclosed to the Application by these two methods is the following:
   315  
   316  - the transaction list
   317  - the `LastCommit` referring to the previous block
   318  - the block header's hash (except in `PrepareProposal`, where it is not known yet)
   319  - list of validators that misbehaved
   320  - the block's timestamp
   321  - `NextValidatorsHash`
   322  - Proposer address
   323  
   324  The Application may decide to *immediately* execute the given block (i.e., upon `PrepareProposal`
   325  or `ProcessProposal`). There are two main reasons why the Application may want to do this:
   326  
   327  - *Avoiding invalid transactions in blocks*.
   328    In order to be sure that the block does not contain *any* invalid transaction, there may be
   329    no way other than fully executing the transactions in the block as though it was the *decided*
   330    block.
   331  - *Quick `FinalizeBlock` execution*.
   332    Upon reception of the decided block via `FinalizeBlock`, if that same block was executed
   333    upon `PrepareProposal` or `ProcessProposal` and the resulting state was kept in memory, the
   334    Application can simply apply that state (faster) to the main state, rather than reexecuting
   335    the decided block (slower).
   336  
   337  `PrepareProposal`/`ProcessProposal` can be called many times for a given height. Moreover,
   338  it is not possible to accurately predict which of the blocks proposed in a height will be decided,
   339  being delivered to the Application in that height's `FinalizeBlock`.
   340  Therefore, the state resulting from executing a proposed block, denoted a *candidate state*, should
   341  be kept in memory as a possible final state for that height. When `FinalizeBlock` is called, the Application should
   342  check if the decided block corresponds to one of its candidate states; if so, it will apply it as
   343  its *ExecuteTxState* (see [Consensus Connection](#consensus-connection) below),
   344  which will be persisted during the upcoming `Commit` call.
   345  
   346  Under adverse conditions (e.g., network instability), the consensus algorithm might take many rounds.
   347  In this case, potentially many proposed blocks will be disclosed to the Application for a given height.
   348  By the nature of Tendermint consensus algorithm, currently adopted in CometBFT, the number of proposed blocks received by the Application
   349  for a particular height cannot be bound, so Application developers must act with care and use mechanisms
   350  to bound memory usage. As a general rule, the Application should be ready to discard candidate states
   351  before `FinalizeBlock`, even if one of them might end up corresponding to the
   352  decided block and thus have to be reexecuted upon `FinalizeBlock`.
   353  
   354  ### [States and ABCI++ Connections](#states-and-abci-connections)
   355  
   356  #### Consensus Connection
   357  
   358  The Consensus Connection should maintain an *ExecuteTxState* &mdash; the working state
   359  for block execution. It should be updated by the call to `FinalizeBlock`
   360  during block execution and committed to disk as the "latest
   361  committed state" during `Commit`. Execution of a proposed block (via `PrepareProposal`/`ProcessProposal`)
   362  **must not** update the *ExecuteTxState*, but rather be kept as a separate candidate state until `FinalizeBlock`
   363  confirms which of the candidate states (if any) can be used to update *ExecuteTxState*.
   364  
   365  #### Mempool Connection
   366  
   367  The mempool Connection maintains *CheckTxState*. CometBFT sequentially processes an incoming
   368  transaction (via RPC from client or P2P from the gossip layer) against *CheckTxState*.
   369  If the processing does not return any error, the transaction is accepted into the mempool
   370  and CometBFT starts gossipping it.
   371  *CheckTxState* should be reset to the latest committed state
   372  at the end of every `Commit`.
   373  
   374  During the execution of a consensus instance, the *CheckTxState* may be updated concurrently with the
   375  *ExecuteTxState*, as messages may be sent concurrently on the Consensus and Mempool connections.
   376  At the end of the consensus instance, as described above, CometBFT locks the mempool and flushes
   377  the mempool connection before calling `Commit`. This ensures that all pending `CheckTx` calls are
   378  responded to and no new ones can begin.
   379  
   380  After the `Commit` call returns, while still holding the mempool lock, `CheckTx` is run again on all
   381  transactions that remain in the node's local mempool after filtering those included in the block.
   382  Parameter `Type` in `RequestCheckTx`
   383  indicates whether an incoming transaction is new (`CheckTxType_New`), or a
   384  recheck (`CheckTxType_Recheck`).
   385  
   386  Finally, after re-checking transactions in the mempool, CometBFT will unlock
   387  the mempool connection. New transactions are once again able to be processed through `CheckTx`.
   388  
   389  Note that `CheckTx` is just a weak filter to keep invalid transactions out of the mempool and,
   390  ultimately, ouf of the blockchain.
   391  Since the transaction cannot be guaranteed to be checked against the exact same state as it
   392  will be executed as part of a (potential) decided block, `CheckTx` shouldn't check *everything*
   393  that affects the transaction's validity, in particular those checks whose validity may depend on
   394  transaction ordering. `CheckTx` is weak because a Byzantine node need not care about `CheckTx`;
   395  it can propose a block full of invalid transactions if it wants. The mechanism ABCI++ has
   396  in place for dealing with such behavior is `ProcessProposal`.
   397  
   398  ##### Replay Protection
   399  
   400  It is possible for old transactions to be sent again to the Application. This is typically
   401  undesirable for all transactions, except for a generally small subset of them which are idempotent.
   402  
   403  The mempool has a mechanism to prevent duplicated transactions from being processed.
   404  This mechanism is nevertheless best-effort (currently based on the indexer)
   405  and does not provide any guarantee of non duplication.
   406  It is thus up to the Application to implement an application-specific
   407  replay protection mechanism with strong guarantees as part of the logic in `CheckTx`.
   408  
   409  #### Info/Query Connection
   410  
   411  The Info (or Query) Connection should maintain a `QueryState`. This connection has two
   412  purposes: 1) having the application answer the queries CometBFT receives from users
   413  (see section [Query](#query)),
   414  and 2) synchronizing CometBFT and the Application at start up time (see
   415  [Crash Recovery](#crash-recovery))
   416  or after state sync (see [State Sync](#state-sync)).
   417  
   418  `QueryState` is a read-only copy of *ExecuteTxState* as it was after the last
   419  `Commit`, i.e.
   420  after the full block has been processed and the state committed to disk.
   421  
   422  #### Snapshot Connection
   423  
   424  The Snapshot Connection is used to serve state sync snapshots for other nodes
   425  and/or restore state sync snapshots to a local node being bootstrapped.
   426  Snapshot management is optional: an Application may choose not to implement it.
   427  
   428  For more information, see Section [State Sync](#state-sync).
   429  
   430  ### Transaction Results
   431  
   432  The Application is expected to return a list of
   433  [`ExecTxResult`](./abci%2B%2B_methods.md#exectxresult) in
   434  [`ResponseFinalizeBlock`](./abci%2B%2B_methods.md#finalizeblock). The list of transaction
   435  results MUST respect the same order as the list of transactions delivered via
   436  [`RequestFinalizeBlock`](./abci%2B%2B_methods.md#finalizeblock).
   437  This section discusses the fields inside this structure, along with the fields in
   438  [`ResponseCheckTx`](./abci%2B%2B_methods.md#checktx),
   439  whose semantics are similar.
   440  
   441  The `Info` and `Log` fields are
   442  non-deterministic values for debugging/convenience purposes. CometBFT logs them but they
   443  are otherwise ignored.
   444  
   445  #### Gas
   446  
   447  Ethereum introduced the notion of *gas* as an abstract representation of the
   448  cost of the resources consumed by nodes when processing a transaction. Every operation in the
   449  Ethereum Virtual Machine uses some amount of gas.
   450  Gas has a market-variable price based on which miners can accept or reject to execute a
   451  particular operation.
   452  
   453  Users propose a maximum amount of gas for their transaction; if the transaction uses less, they get
   454  the difference credited back. CometBFT adopts a similar abstraction,
   455  though uses it only optionally and weakly, allowing applications to define
   456  their own sense of the cost of execution.
   457  
   458  In CometBFT, the [ConsensusParams.Block.MaxGas](#consensus-parameters) limits the amount of
   459  total gas that can be used by all transactions in a block.
   460  The default value is `-1`, which means the block gas limit is not enforced, or that the concept of
   461  gas is meaningless.
   462  
   463  Responses contain a `GasWanted` and `GasUsed` field. The former is the maximum
   464  amount of gas the sender of a transaction is willing to use, and the latter is how much it actually
   465  used. Applications should enforce that `GasUsed <= GasWanted` &mdash; i.e. transaction execution
   466  or validation should fail before it can use more resources than it requested.
   467  
   468  When `MaxGas > -1`, CometBFT enforces the following rules:
   469  
   470  - `GasWanted <= MaxGas` for every transaction in the mempool
   471  - `(sum of GasWanted in a block) <= MaxGas` when proposing a block
   472  
   473  If `MaxGas == -1`, no rules about gas are enforced.
   474  
   475  In v0.34.x and earlier versions, CometBFT does not enforce anything about Gas in consensus,
   476  only in the mempool.
   477  This means it does not guarantee that committed blocks satisfy these rules.
   478  It is the application's responsibility to return non-zero response codes when gas limits are exceeded
   479  when executing the transactions of a block.
   480  Since the introduction of `PrepareProposal` and `ProcessProposal` in v.0.37.x, it is now possible
   481  for the Application to enforce that all blocks proposed (and voted for) in consensus &mdash; and thus all
   482  blocks decided &mdash; respect the `MaxGas` limits described above.
   483  
   484  Since the Application should enforce that `GasUsed <= GasWanted` when executing a transaction, and
   485  it can use `PrepareProposal` and `ProcessProposal` to enforce that `(sum of GasWanted in a block) <= MaxGas`
   486  in all proposed or prevoted blocks,
   487  we have:
   488  
   489  - `(sum of GasUsed in a block) <= MaxGas` for every block
   490  
   491  The `GasUsed` field is ignored by CometBFT.
   492  
   493  #### Specifics of `ResponseCheckTx`
   494  
   495  If `Code != 0`, it will be rejected from the mempool and hence
   496  not broadcasted to other peers and not included in a proposal block.
   497  
   498  `Data` contains the result of the `CheckTx` transaction execution, if any. It does not need to be
   499  deterministic since, given a transaction, nodes' Applications
   500  might have a different *CheckTxState* values when they receive it and check their validity
   501  via `CheckTx`.
   502  CometBFT ignores this value in `ResponseCheckTx`.
   503  
   504  From v0.34.x on, there is a `Priority` field in `ResponseCheckTx` that can be
   505  used to explicitly prioritize transactions in the mempool for inclusion in a block
   506  proposal.
   507  
   508  #### Specifics of `ExecTxResult`
   509  
   510  `FinalizeBlock` is the workhorse of the blockchain. CometBFT delivers the decided block,
   511  including the list of all its transactions synchronously to the Application.
   512  The block delivered (and thus the transaction order) is the same at all correct nodes as guaranteed
   513  by the Agreement property of consensus.
   514  
   515  The `Data` field in `ExecTxResult` contains an array of bytes with the transaction result.
   516  It must be deterministic (i.e., the same value must be returned at all nodes), but it can contain arbitrary
   517  data. Likewise, the value of `Code` must be deterministic.
   518  If `Code != 0`, the transaction will be marked invalid,
   519  though it is still included in the block. Invalid transactions are not indexed, as they are
   520  considered analogous to those that failed `CheckTx`.
   521  
   522  Both the `Code` and `Data` are included in a structure that is hashed into the
   523  `LastResultsHash` of the block header in the next height.
   524  
   525  `Events` include any events for the execution, which CometBFT will use to index
   526  the transaction by. This allows transactions to be queried according to what
   527  events took place during their execution.
   528  
   529  ### Updating the Validator Set
   530  
   531  The application may set the validator set during
   532  [`InitChain`](./abci%2B%2B_methods.md#initchain), and may update it during
   533  [`FinalizeBlock`](./abci%2B%2B_methods.md#finalizeblock). In both cases, a structure of type
   534  [`ValidatorUpdate`](./abci%2B%2B_methods.md#validatorupdate) is returned.
   535  
   536  The `InitChain` method, used to initialize the Application, can return a list of validators.
   537  If the list is empty, CometBFT will use the validators loaded from the genesis
   538  file.
   539  If the list returned by `InitChain` is not empty, CometBFT will use its contents as the validator set.
   540  This way the application can set the initial validator set for the
   541  blockchain.
   542  
   543  Applications must ensure that a single set of validator updates does not contain duplicates, i.e.
   544  a given public key can only appear once within a given update. If an update includes
   545  duplicates, the block execution will fail irrecoverably.
   546  
   547  Structure `ValidatorUpdate` contains a public key, which is used to identify the validator:
   548  The public key currently supports three types:
   549  
   550  - `ed25519`
   551  - `secp256k1`
   552  - `sr25519`
   553  
   554  Structure `ValidatorUpdate` also contains an `ìnt64` field denoting the validator's new power.
   555  Applications must ensure that
   556  `ValidatorUpdate` structures abide by the following rules:
   557  
   558  - power must be non-negative
   559  - if power is set to 0, the validator must be in the validator set; it will be removed from the set
   560  - if power is greater than 0:
   561      - if the validator is not in the validator set, it will be added to the
   562        set with the given power
   563      - if the validator is in the validator set, its power will be adjusted to the given power
   564  - the total power of the new validator set must not exceed `MaxTotalVotingPower`, where
   565    `MaxTotalVotingPower = MaxInt64 / 8`
   566  
   567  Note the updates returned after processing the block at height `H` will only take effect
   568  at block `H+2` (see Section [Methods](./abci%2B%2B_methods.md)).
   569  
   570  ### Consensus Parameters
   571  
   572  `ConsensusParams` are global parameters that apply to all validators in a blockchain.
   573  They enforce certain limits in the blockchain, like the maximum size
   574  of blocks, amount of gas used in a block, and the maximum acceptable age of
   575  evidence. They can be set in
   576  [`InitChain`](./abci%2B%2B_methods.md#initchain), and updated in
   577  [`FinalizeBlock`](./abci%2B%2B_methods.md#finalizeblock).
   578  These parameters are deterministically set and/or updated by the Application, so
   579  all full nodes have the same value at a given height.
   580  
   581  #### List of Parameters
   582  
   583  These are the current consensus parameters (as of v0.38.x):
   584  
   585  1. [ABCIParams.VoteExtensionsEnableHeight](#abciparamsvoteextensionsenableheight)
   586  2. [BlockParams.MaxBytes](#blockparamsmaxbytes)
   587  3. [BlockParams.MaxGas](#blockparamsmaxgas)
   588  4. [EvidenceParams.MaxAgeDuration](#evidenceparamsmaxageduration)
   589  5. [EvidenceParams.MaxAgeNumBlocks](#evidenceparamsmaxagenumblocks)
   590  6. [EvidenceParams.MaxBytes](#evidenceparamsmaxbytes)
   591  7. [ValidatorParams.PubKeyTypes](#validatorparamspubkeytypes)
   592  8. [VersionParams.App](#versionparamsapp)
   593  
   594  ##### ABCIParams.VoteExtensionsEnableHeight
   595  
   596  This parameter is either 0 or a positive height at which vote extensions
   597  become mandatory. If the value is zero (which is the default), vote
   598  extensions are not expected. Otherwise, at all heights greater than the
   599  configured height `H` vote extensions must be present (even if empty).
   600  When the configured height `H` is reached, `PrepareProposal` will not
   601  include vote extensions yet, but `ExtendVote` and `VerifyVoteExtension` will
   602  be called. Then, when reaching height `H+1`, `PrepareProposal` will
   603  include the vote extensions from height `H`. For all heights after `H`
   604  
   605  - vote extensions cannot be disabled,
   606  - they are mandatory: all precommit messages sent MUST have an extension
   607    attached. Nevertheless, the application MAY provide 0-length
   608    extensions.
   609  
   610  Must always be set to a future height, 0, or the same height that was previously set.
   611  Once the chain's height reaches the value set, it cannot be changed to a different value.
   612  
   613  ##### BlockParams.MaxBytes
   614  
   615  The maximum size of a complete Protobuf encoded block.
   616  This is enforced by the consensus algorithm.
   617  
   618  This implies a maximum transaction size that is `MaxBytes`, less the expected size of
   619  the header, the validator set, and any included evidence in the block.
   620  
   621  The Application should be aware that honest validators *may* produce and
   622  broadcast blocks with up to the configured `MaxBytes` size.
   623  As a result, the consensus
   624  [timeout parameters](../../docs/core/configuration.md#consensus-timeouts-explained)
   625  adopted by nodes should be configured so as to account for the worst-case
   626  latency for the delivery of a full block with `MaxBytes` size to all validators.
   627  
   628  If the Application wants full control over the size of blocks,
   629  it can do so by enforcing a byte limit set up at the Application level.
   630  This Application-internal limit is used by `PrepareProposal` to bound the total size
   631  of transactions it returns, and by `ProcessProposal` to reject any received block
   632  whose total transaction size is bigger than the enforced limit.
   633  In such case, the Application MAY set `MaxBytes` to -1.
   634  
   635  If the Application sets value -1, consensus will:
   636  
   637  - consider that the actual value to enforce is 100 MB
   638  - will provide *all* transactions in the mempool in calls to `PrepareProposal`
   639  
   640  Must have `MaxBytes == -1` OR `0 < MaxBytes <= 100 MB`.
   641  
   642  > Bear in mind that the default value for the `BlockParams.MaxBytes` consensus
   643  > parameter accepts as valid blocks with size up to 21 MB.
   644  > If the Application's use case does not need blocks of that size,
   645  > or if the impact (specially on bandwidth consumption and block latency)
   646  > of propagating blocks of that size was not evaluated,
   647  > it is strongly recommended to wind down this default value.
   648  
   649  ##### BlockParams.MaxGas
   650  
   651  The maximum of the sum of `GasWanted` that will be allowed in a proposed block.
   652  This is *not* enforced by the consensus algorithm.
   653  It is left to the Application to enforce (ie. if transactions are included past the
   654  limit, they should return non-zero codes). It is used by CometBFT to limit the
   655  transactions included in a proposed block.
   656  
   657  Must have `MaxGas >= -1`.
   658  If `MaxGas == -1`, no limit is enforced.
   659  
   660  ##### EvidenceParams.MaxAgeDuration
   661  
   662  This is the maximum age of evidence in time units.
   663  This is enforced by the consensus algorithm.
   664  
   665  If a block includes evidence older than this (AND the evidence was created more
   666  than `MaxAgeNumBlocks` ago), the block will be rejected (validators won't vote
   667  for it).
   668  
   669  Must have `MaxAgeDuration > 0`.
   670  
   671  ##### EvidenceParams.MaxAgeNumBlocks
   672  
   673  This is the maximum age of evidence in blocks.
   674  This is enforced by the consensus algorithm.
   675  
   676  If a block includes evidence older than this (AND the evidence was created more
   677  than `MaxAgeDuration` ago), the block will be rejected (validators won't vote
   678  for it).
   679  
   680  Must have `MaxAgeNumBlocks > 0`.
   681  
   682  ##### EvidenceParams.MaxBytes
   683  
   684  This is the maximum size of total evidence in bytes that can be committed to a
   685  single block. It should fall comfortably under the max block bytes.
   686  
   687  Its value must not exceed the size of
   688  a block minus its overhead ( ~ `BlockParams.MaxBytes`).
   689  
   690  Must have `MaxBytes > 0`.
   691  
   692  ##### ValidatorParams.PubKeyTypes
   693  
   694  The parameter restricts the type of keys validators can use. The parameter uses ABCI pubkey naming, not Amino names.
   695  
   696  ##### VersionParams.App
   697  
   698  This is the version of the ABCI application.
   699  
   700  #### Updating Consensus Parameters
   701  
   702  The application may set the `ConsensusParams` during
   703  [`InitChain`](./abci%2B%2B_methods.md#initchain),
   704  and update them during
   705  [`FinalizeBlock`](./abci%2B%2B_methods.md#finalizeblock).
   706  If the `ConsensusParams` is empty, it will be ignored. Each field
   707  that is not empty will be applied in full. For instance, if updating the
   708  `Block.MaxBytes`, applications must also set the other `Block` fields (like
   709  `Block.MaxGas`), even if they are unchanged, as they will otherwise cause the
   710  value to be updated to the default.
   711  
   712  ##### `InitChain`
   713  
   714  `ResponseInitChain` includes a `ConsensusParams` parameter.
   715  If `ConsensusParams` is `nil`, CometBFT will use the params loaded in the genesis
   716  file. If `ConsensusParams` is not `nil`, CometBFT will use it.
   717  This way the application can determine the initial consensus parameters for the
   718  blockchain.
   719  
   720  ##### `FinalizeBlock`, `PrepareProposal`/`ProcessProposal`
   721  
   722  `ResponseFinalizeBlock` accepts a `ConsensusParams` parameter.
   723  If `ConsensusParams` is `nil`, CometBFT will do nothing.
   724  If `ConsensusParams` is not `nil`, CometBFT will use it.
   725  This way the application can update the consensus parameters over time.
   726  
   727  The updates returned in block `H` will take effect right away for block
   728  `H+1`.
   729  
   730  ### `Query`
   731  
   732  `Query` is a generic method with lots of flexibility to enable diverse sets
   733  of queries on application state. CometBFT makes use of `Query` to filter new peers
   734  based on ID and IP, and exposes `Query` to the user over RPC.
   735  
   736  Note that calls to `Query` are not replicated across nodes, but rather query the
   737  local node's state - hence they may return stale reads. For reads that require
   738  consensus, use a transaction.
   739  
   740  The most important use of `Query` is to return Merkle proofs of the application state at some height
   741  that can be used for efficient application-specific light-clients.
   742  
   743  Note CometBFT has technically no requirements from the `Query`
   744  message for normal operation - that is, the ABCI app developer need not implement
   745  Query functionality if they do not wish to.
   746  
   747  #### Query Proofs
   748  
   749  The CometBFT block header includes a number of hashes, each providing an
   750  anchor for some type of proof about the blockchain. The `ValidatorsHash` enables
   751  quick verification of the validator set, the `DataHash` gives quick
   752  verification of the transactions included in the block.
   753  
   754  The `AppHash` is unique in that it is application specific, and allows for
   755  application-specific Merkle proofs about the state of the application.
   756  While some applications keep all relevant state in the transactions themselves
   757  (like Bitcoin and its UTXOs), others maintain a separated state that is
   758  computed deterministically *from* transactions, but is not contained directly in
   759  the transactions themselves (like Ethereum contracts and accounts).
   760  For such applications, the `AppHash` provides a much more efficient way to verify light-client proofs.
   761  
   762  ABCI applications can take advantage of more efficient light-client proofs for
   763  their state as follows:
   764  
   765  - return the Merkle root of the deterministic application state in
   766    `ResponseFinalizeBlock.Data`. This Merkle root will be included as the `AppHash` in the next block.
   767  - return efficient Merkle proofs about that application state in `ResponseQuery.Proof`
   768    that can be verified using the `AppHash` of the corresponding block.
   769  
   770  For instance, this allows an application's light-client to verify proofs of
   771  absence in the application state, something which is much less efficient to do using the block hash.
   772  
   773  Some applications (eg. Ethereum, Cosmos-SDK) have multiple "levels" of Merkle trees,
   774  where the leaves of one tree are the root hashes of others. To support this, and
   775  the general variability in Merkle proofs, the `ResponseQuery.Proof` has some minimal structure:
   776  
   777  ```protobuf
   778  message ProofOps {
   779    repeated ProofOp ops = 1
   780  }
   781  
   782  message ProofOp {
   783    string type = 1;
   784    bytes key   = 2;
   785    bytes data  = 3;
   786  }
   787  ```
   788  
   789  Each `ProofOp` contains a proof for a single key in a single Merkle tree, of the specified `type`.
   790  This allows ABCI to support many different kinds of Merkle trees, encoding
   791  formats, and proofs (eg. of presence and absence) just by varying the `type`.
   792  The `data` contains the actual encoded proof, encoded according to the `type`.
   793  When verifying the full proof, the root hash for one ProofOp is the value being
   794  verified for the next ProofOp in the list. The root hash of the final ProofOp in
   795  the list should match the `AppHash` being verified against.
   796  
   797  #### Peer Filtering
   798  
   799  When CometBFT connects to a peer, it sends two queries to the ABCI application
   800  using the following paths, with no additional data:
   801  
   802  - `/p2p/filter/addr/<IP:PORT>`, where `<IP:PORT>` denote the IP address and
   803    the port of the connection
   804  - `p2p/filter/id/<ID>`, where `<ID>` is the peer node ID (ie. the
   805    pubkey.Address() for the peer's PubKey)
   806  
   807  If either of these queries return a non-zero ABCI code, CometBFT will refuse
   808  to connect to the peer.
   809  
   810  #### Paths
   811  
   812  Queries are directed at paths, and may optionally include additional data.
   813  
   814  The expectation is for there to be some number of high level paths
   815  differentiating concerns, like `/p2p`, `/store`, and `/app`. Currently,
   816  CometBFT only uses `/p2p`, for filtering peers. For more advanced use, see the
   817  implementation of
   818  [Query in the Cosmos-SDK](https://github.com/cosmos/cosmos-sdk/blob/e2037f7696fed4fdd4bc076f9e7053fe8178a881/baseapp/abci.go#L557-L565).
   819  
   820  ### Crash Recovery
   821  
   822  CometBFT and the application are expected to crash together and there should not
   823  exist a scenario where the application has persisted state of a height greater than the
   824  latest height persisted by CometBFT.
   825  
   826  In practice, persisting the state of a height consists of three steps, the last of which
   827  is the call to the application's `Commit` method, the only place where the application is expected to
   828  persist/commit its state.
   829  On startup (upon recovery), CometBFT calls the `Info` method on the Info Connection to get the latest
   830  committed state of the app. The app MUST return information consistent with the
   831  last block for which it successfully completed `Commit`.
   832  
   833  The three steps performed before the state of a height is considered persisted are:
   834  
   835  - The block is stored by CometBFT in the blockstore
   836  - CometBFT has stored the state returned by the application through `FinalizeBlockResponse`
   837  - The application has committed its state within `Commit`.
   838  
   839  The following diagram depicts the order in which these events happen, and the corresponding
   840  ABCI functions that are called and executed by CometBFT and the application:
   841  
   842  
   843  ```
   844  APP:                                              Execute block                         Persist application state
   845                                                   /     return ResultFinalizeBlock            /
   846                                                  /                                           /
   847  Event: ------------- block_stored ------------ / ------------ state_stored --------------- / ----- app_persisted_state
   848                            |                   /                   |                       /        |
   849  CometBFT: Decide --- Persist block -- Call FinalizeBlock - Persist results ---------- Call Commit --
   850              on        in the                                (txResults, validator
   851             Block      block store                              updates...)
   852  
   853  ```
   854  
   855  As these three steps are not atomic, we observe different cases based on which steps have been executed
   856  before the crash occurred
   857  (we assume that at least `block_stored` has been executed, otherwise, there is no state persisted,
   858  and the operations for this height are repeated entirely):
   859  
   860  - `block_stored`: we replay `FinalizeBlock` and the steps afterwards.
   861  - `block_stored` and `state_stored`: As the app did not persist its state within `Commit`, we need to re-execute
   862    `FinalizeBlock` to retrieve the results and compare them to the state stored by CometBFT within `state_stored`.
   863    The expected case is that the states will match, otherwise CometBFT panics.
   864  - `block_stored`, `state_stored`, `app_persisted_state`: we move on to the next height.
   865  
   866  Based on the sequence of these events, CometBFT will panic if any of the steps in the sequence happen out of order,
   867  that is if:
   868  
   869  - The application has persisted a block at a height higher than the blocked saved during `state_stored`.
   870  - The `block_stored` step persisted a block at a height smaller than the `state_stored`
   871  - And the difference between the heights of the blocks persisted by `state_stored` and `block_stored` is more
   872  than 1 (this corresponds to a scenario where we stored two blocks in the block store but never persisted the state of the first
   873  block, which should never happen).
   874  
   875  A special case is when a crash happens before the first block is committed - that is, after calling
   876  `InitChain`. In that case, the application's state should still be at height 0 and thus `InitChain`
   877  will be called again.
   878  
   879  
   880  ### State Sync
   881  
   882  A new node joining the network can simply join consensus at the genesis height and replay all
   883  historical blocks until it is caught up. However, for large chains this can take a significant
   884  amount of time, often on the order of days or weeks.
   885  
   886  State sync is an alternative mechanism for bootstrapping a new node, where it fetches a snapshot
   887  of the state machine at a given height and restores it. Depending on the application, this can
   888  be several orders of magnitude faster than replaying blocks.
   889  
   890  Note that state sync does not currently backfill historical blocks, so the node will have a
   891  truncated block history - users are advised to consider the broader network implications of this in
   892  terms of block availability and auditability. This functionality may be added in the future.
   893  
   894  For details on the specific ABCI calls and types, see the
   895  [methods](abci%2B%2B_methods.md) section.
   896  
   897  #### Taking Snapshots
   898  
   899  Applications that want to support state syncing must take state snapshots at regular intervals. How
   900  this is accomplished is entirely up to the application. A snapshot consists of some metadata and
   901  a set of binary chunks in an arbitrary format:
   902  
   903  - `Height (uint64)`: The height at which the snapshot is taken. It must be taken after the given
   904    height has been committed, and must not contain data from any later heights.
   905  
   906  - `Format (uint32)`: An arbitrary snapshot format identifier. This can be used to version snapshot
   907    formats, e.g. to switch from Protobuf to MessagePack for serialization. The application can use
   908    this when restoring to choose whether to accept or reject a snapshot.
   909  
   910  - `Chunks (uint32)`: The number of chunks in the snapshot. Each chunk contains arbitrary binary
   911    data, and should be less than 16 MB; 10 MB is a good starting point.
   912  
   913  - `Hash ([]byte)`: An arbitrary hash of the snapshot. This is used to check whether a snapshot is
   914    the same across nodes when downloading chunks.
   915  
   916  - `Metadata ([]byte)`: Arbitrary snapshot metadata, e.g. chunk hashes for verification or any other
   917    necessary info.
   918  
   919  For a snapshot to be considered the same across nodes, all of these fields must be identical. When
   920  sent across the network, snapshot metadata messages are limited to 4 MB.
   921  
   922  When a new node is running state sync and discovering snapshots, CometBFT will query an existing
   923  application via the ABCI `ListSnapshots` method to discover available snapshots, and load binary
   924  snapshot chunks via `LoadSnapshotChunk`. The application is free to choose how to implement this
   925  and which formats to use, but must provide the following guarantees:
   926  
   927  - **Consistent:** A snapshot must be taken at a single isolated height, unaffected by
   928    concurrent writes. This can be accomplished by using a data store that supports ACID
   929    transactions with snapshot isolation.
   930  
   931  - **Asynchronous:** Taking a snapshot can be time-consuming, so it must not halt chain progress,
   932    for example by running in a separate thread.
   933  
   934  - **Deterministic:** A snapshot taken at the same height in the same format must be identical
   935    (at the byte level) across nodes, including all metadata. This ensures good availability of
   936    chunks, and that they fit together across nodes.
   937  
   938  A very basic approach might be to use a datastore with MVCC transactions (such as RocksDB),
   939  start a transaction immediately after block commit, and spawn a new thread which is passed the
   940  transaction handle. This thread can then export all data items, serialize them using e.g.
   941  Protobuf, hash the byte stream, split it into chunks, and store the chunks in the file system
   942  along with some metadata - all while the blockchain is applying new blocks in parallel.
   943  
   944  A more advanced approach might include incremental verification of individual chunks against the
   945  chain app hash, parallel or batched exports, compression, and so on.
   946  
   947  Old snapshots should be removed after some time - generally only the last two snapshots are needed
   948  (to prevent the last one from being removed while a node is restoring it).
   949  
   950  #### Bootstrapping a Node
   951  
   952  An empty node can be state synced by setting the configuration option `statesync.enabled =
   953  true`. The node also needs the chain genesis file for basic chain info, and configuration for
   954  light client verification of the restored snapshot: a set of CometBFT RPC servers, and a
   955  trusted header hash and corresponding height from a trusted source, via the `statesync`
   956  configuration section.
   957  
   958  Once started, the node will connect to the P2P network and begin discovering snapshots. These
   959  will be offered to the local application via the `OfferSnapshot` ABCI method. Once a snapshot
   960  is accepted CometBFT will fetch and apply the snapshot chunks. After all chunks have been
   961  successfully applied, CometBFT verifies the app's `AppHash` against the chain using the light
   962  client, then switches the node to normal consensus operation.
   963  
   964  ##### Snapshot Discovery
   965  
   966  When the empty node joins the P2P network, it asks all peers to report snapshots via the
   967  `ListSnapshots` ABCI call (limited to 10 per node). After some time, the node picks the most
   968  suitable snapshot (generally prioritized by height, format, and number of peers), and offers it
   969  to the application via `OfferSnapshot`. The application can choose a number of responses,
   970  including accepting or rejecting it, rejecting the offered format, rejecting the peer who sent
   971  it, and so on. CometBFT will keep discovering and offering snapshots until one is accepted or
   972  the application aborts.
   973  
   974  ##### Snapshot Restoration
   975  
   976  Once a snapshot has been accepted via `OfferSnapshot`, CometBFT begins downloading chunks from
   977  any peers that have the same snapshot (i.e. that have identical metadata fields). Chunks are
   978  spooled in a temporary directory, and then given to the application in sequential order via
   979  `ApplySnapshotChunk` until all chunks have been accepted.
   980  
   981  The method for restoring snapshot chunks is entirely up to the application.
   982  
   983  During restoration, the application can respond to `ApplySnapshotChunk` with instructions for how
   984  to continue. This will typically be to accept the chunk and await the next one, but it can also
   985  ask for chunks to be refetched (either the current one or any number of previous ones), P2P peers
   986  to be banned, snapshots to be rejected or retried, and a number of other responses - see the ABCI
   987  reference for details.
   988  
   989  If CometBFT fails to fetch a chunk after some time, it will reject the snapshot and try a
   990  different one via `OfferSnapshot` - the application can choose whether it wants to support
   991  restarting restoration, or simply abort with an error.
   992  
   993  ##### Snapshot Verification
   994  
   995  Once all chunks have been accepted, CometBFT issues an `Info` ABCI call to retrieve the
   996  `LastBlockAppHash`. This is compared with the trusted app hash from the chain, retrieved and
   997  verified using the light client. CometBFT also checks that `LastBlockHeight` corresponds to the
   998  height of the snapshot.
   999  
  1000  This verification ensures that an application is valid before joining the network. However, the
  1001  snapshot restoration may take a long time to complete, so applications may want to employ additional
  1002  verification during the restore to detect failures early. This might e.g. include incremental
  1003  verification of each chunk against the app hash (using bundled Merkle proofs), checksums to
  1004  protect against data corruption by the disk or network, and so on. However, it is important to
  1005  note that the only trusted information available is the app hash, and all other snapshot metadata
  1006  can be spoofed by adversaries.
  1007  
  1008  Apps may also want to consider state sync denial-of-service vectors, where adversaries provide
  1009  invalid or harmful snapshots to prevent nodes from joining the network. The application can
  1010  counteract this by asking CometBFT to ban peers. As a last resort, node operators can use
  1011  P2P configuration options to whitelist a set of trusted peers that can provide valid snapshots.
  1012  
  1013  ##### Transition to Consensus
  1014  
  1015  Once the snapshots have all been restored, CometBFT gathers additional information necessary for
  1016  bootstrapping the node (e.g. chain ID, consensus parameters, validator sets, and block headers)
  1017  from the genesis file and light client RPC servers. It also calls `Info` to verify the following:
  1018  
  1019  - that the app hash from the snapshot it has delivered to the Application matches the apphash
  1020    stored in the next height's block
  1021  
  1022  - that the version that the Application returns in `ResponseInfo` matches the version in the
  1023    current height's block header
  1024  
  1025  Once the state machine has been restored and CometBFT has gathered this additional
  1026  information, it transitions to consensus. As of ABCI 2.0, CometBFT ensures the necessary conditions
  1027  to switch are met [RFC-100](https://github.com/KYVENetwork/cometbft/v38/blob/v0.38.x/docs/rfc/rfc-100-abci-vote-extension-propag.md#base-implementation-persist-and-propagate-extended-commit-history).
  1028  From the application's point of view, these operations are transparent, unless the application has just upgraded to ABCI 2.0.
  1029  In that case, the application needs to be properly configured and aware of certain constraints in terms of when
  1030  to provide vote extensions. More details can be found in the section below.
  1031  
  1032  Once a node switches to consensus, it operates like any other node, apart from having a truncated block history at the height of the restored snapshot.
  1033  
  1034  ## Application configuration required to switch to ABCI 2.0
  1035  
  1036  Introducing vote extensions requires changes to the configuration of the application.
  1037  
  1038  First of all, switching to a version of CometBFT with vote extensions, requires a coordinated upgrade.
  1039  For a detailed description on the upgrade path, please refer to the corresponding
  1040  [section](https://github.com/KYVENetwork/cometbft/v38/blob/v0.38.x/docs/rfc/rfc-100-abci-vote-extension-propag.md#upgrade-path) in RFC-100.
  1041  
  1042  There is a newly introduced [**consensus parameter**](./abci%2B%2B_app_requirements.md#abciparamsvoteextensionsenableheight): `VoteExtensionsEnableHeight`.
  1043  This parameter represents the height at which vote extensions are
  1044  required for consensus to proceed, with 0 being the default value (no vote extensions).
  1045  A chain can enable vote extensions either:
  1046  
  1047  - at genesis by setting `VoteExtensionsEnableHeight` to be equal, e.g., to the `InitialHeight`
  1048  - or via the application logic by changing the `ConsensusParam` to configure the
  1049  `VoteExtensionsEnableHeight`.
  1050  
  1051  Once the (coordinated) upgrade to ABCI 2.0 has taken place, at height  *h<sub>u</sub>*,
  1052  the value of `VoteExtensionsEnableHeight` MAY be set to some height, *h<sub>e</sub>*,
  1053  which MUST be higher than the current height of the chain. Thus the earliest value for
  1054   *h<sub>e</sub>* is  *h<sub>u</sub>* + 1.
  1055  
  1056  Once a node reaches the configured height,
  1057  for all heights *h ≥ h<sub>e</sub>*, the consensus algorithm will
  1058  reject as invalid any precommit messages that do not have signed vote extension data.
  1059  If the application requires it, a 0-length vote extension is allowed, but it MUST be signed
  1060  and present in the precommit message.
  1061  Likewise, for all heights *h < h<sub>e</sub>*, any precommit messages that *do* have vote extensions
  1062  will also be rejected as malformed.
  1063  Height *h<sub>e</sub>* is somewhat special, as calls to `PrepareProposal` MUST NOT
  1064  have vote extension data, but all precommit votes in that height MUST carry a vote extension,
  1065  even if the extension is `nil`.
  1066  Height *h<sub>e</sub> + 1* is the first height for which `PrepareProposal` MUST have vote
  1067  extension data and all precommit votes in that height MUST have a vote extension.
  1068  
  1069  Corollary, [CometBFT will decide](./abci%2B%2B_comet_expected_behavior.md#handling-upgrades-to-abci-20) which data to store, and require for successful operations, based on the current height
  1070  of the chain.