github.com/DFWallet/tendermint-cosmos@v0.0.2/docs/introduction/what-is-tendermint.md (about) 1 --- 2 order: 4 3 --- 4 5 # What is Tendermint 6 7 Tendermint is software for securely and consistently replicating an 8 application on many machines. By securely, we mean that Tendermint works 9 even if up to 1/3 of machines fail in arbitrary ways. By consistently, 10 we mean that every non-faulty machine sees the same transaction log and 11 computes the same state. Secure and consistent replication is a 12 fundamental problem in distributed systems; it plays a critical role in 13 the fault tolerance of a broad range of applications, from currencies, 14 to elections, to infrastructure orchestration, and beyond. 15 16 The ability to tolerate machines failing in arbitrary ways, including 17 becoming malicious, is known as Byzantine fault tolerance (BFT). The 18 theory of BFT is decades old, but software implementations have only 19 became popular recently, due largely to the success of "blockchain 20 technology" like Bitcoin and Ethereum. Blockchain technology is just a 21 reformalization of BFT in a more modern setting, with emphasis on 22 peer-to-peer networking and cryptographic authentication. The name 23 derives from the way transactions are batched in blocks, where each 24 block contains a cryptographic hash of the previous one, forming a 25 chain. In practice, the blockchain data structure actually optimizes BFT 26 design. 27 28 Tendermint consists of two chief technical components: a blockchain 29 consensus engine and a generic application interface. The consensus 30 engine, called Tendermint Core, ensures that the same transactions are 31 recorded on every machine in the same order. The application interface, 32 called the Application BlockChain Interface (ABCI), enables the 33 transactions to be processed in any programming language. Unlike other 34 blockchain and consensus solutions, which come pre-packaged with built 35 in state machines (like a fancy key-value store, or a quirky scripting 36 language), developers can use Tendermint for BFT state machine 37 replication of applications written in whatever programming language and 38 development environment is right for them. 39 40 Tendermint is designed to be easy-to-use, simple-to-understand, highly 41 performant, and useful for a wide variety of distributed applications. 42 43 ## Tendermint vs. X 44 45 Tendermint is broadly similar to two classes of software. The first 46 class consists of distributed key-value stores, like Zookeeper, etcd, 47 and consul, which use non-BFT consensus. The second class is known as 48 "blockchain technology", and consists of both cryptocurrencies like 49 Bitcoin and Ethereum, and alternative distributed ledger designs like 50 Hyperledger's Burrow. 51 52 ### Zookeeper, etcd, consul 53 54 Zookeeper, etcd, and consul are all implementations of a key-value store 55 atop a classical, non-BFT consensus algorithm. Zookeeper uses a version 56 of Paxos called Zookeeper Atomic Broadcast, while etcd and consul use 57 the Raft consensus algorithm, which is much younger and simpler. A 58 typical cluster contains 3-5 machines, and can tolerate crash failures 59 in up to 1/2 of the machines, but even a single Byzantine fault can 60 destroy the system. 61 62 Each offering provides a slightly different implementation of a 63 featureful key-value store, but all are generally focused around 64 providing basic services to distributed systems, such as dynamic 65 configuration, service discovery, locking, leader-election, and so on. 66 67 Tendermint is in essence similar software, but with two key differences: 68 69 - It is Byzantine Fault Tolerant, meaning it can only tolerate up to a 70 1/3 of failures, but those failures can include arbitrary behaviour - 71 including hacking and malicious attacks. - It does not specify a 72 particular application, like a fancy key-value store. Instead, it 73 focuses on arbitrary state machine replication, so developers can build 74 the application logic that's right for them, from key-value store to 75 cryptocurrency to e-voting platform and beyond. 76 77 ### Bitcoin, Ethereum, etc 78 79 Tendermint emerged in the tradition of cryptocurrencies like Bitcoin, 80 Ethereum, etc. with the goal of providing a more efficient and secure 81 consensus algorithm than Bitcoin's Proof of Work. In the early days, 82 Tendermint had a simple currency built in, and to participate in 83 consensus, users had to "bond" units of the currency into a security 84 deposit which could be revoked if they misbehaved -this is what made 85 Tendermint a Proof-of-Stake algorithm. 86 87 Since then, Tendermint has evolved to be a general purpose blockchain 88 consensus engine that can host arbitrary application states. That means 89 it can be used as a plug-and-play replacement for the consensus engines 90 of other blockchain software. So one can take the current Ethereum code 91 base, whether in Rust, or Go, or Haskell, and run it as a ABCI 92 application using Tendermint consensus. Indeed, [we did that with 93 Ethereum](https://github.com/cosmos/ethermint). And we plan to do 94 the same for Bitcoin, ZCash, and various other deterministic 95 applications as well. 96 97 Another example of a cryptocurrency application built on Tendermint is 98 [the Cosmos network](http://cosmos.network). 99 100 ### Other Blockchain Projects 101 102 [Fabric](https://github.com/hyperledger/fabric) takes a similar approach 103 to Tendermint, but is more opinionated about how the state is managed, 104 and requires that all application behaviour runs in potentially many 105 docker containers, modules it calls "chaincode". It uses an 106 implementation of [PBFT](http://pmg.csail.mit.edu/papers/osdi99.pdf). 107 from a team at IBM that is [augmented to handle potentially 108 non-deterministic 109 chaincode](https://www.zurich.ibm.com/~cca/papers/sieve.pdf) It is 110 possible to implement this docker-based behaviour as a ABCI app in 111 Tendermint, though extending Tendermint to handle non-determinism 112 remains for future work. 113 114 [Burrow](https://github.com/hyperledger/burrow) is an implementation of 115 the Ethereum Virtual Machine and Ethereum transaction mechanics, with 116 additional features for a name-registry, permissions, and native 117 contracts, and an alternative blockchain API. It uses Tendermint as its 118 consensus engine, and provides a particular application state. 119 120 ## ABCI Overview 121 122 The [Application BlockChain Interface 123 (ABCI)](https://github.com/DFWallet/tendermint-cosmos/tree/master/abci) 124 allows for Byzantine Fault Tolerant replication of applications 125 written in any programming language. 126 127 ### Motivation 128 129 Thus far, all blockchains "stacks" (such as 130 [Bitcoin](https://github.com/bitcoin/bitcoin)) have had a monolithic 131 design. That is, each blockchain stack is a single program that handles 132 all the concerns of a decentralized ledger; this includes P2P 133 connectivity, the "mempool" broadcasting of transactions, consensus on 134 the most recent block, account balances, Turing-complete contracts, 135 user-level permissions, etc. 136 137 Using a monolithic architecture is typically bad practice in computer 138 science. It makes it difficult to reuse components of the code, and 139 attempts to do so result in complex maintenance procedures for forks of 140 the codebase. This is especially true when the codebase is not modular 141 in design and suffers from "spaghetti code". 142 143 Another problem with monolithic design is that it limits you to the 144 language of the blockchain stack (or vice versa). In the case of 145 Ethereum which supports a Turing-complete bytecode virtual-machine, it 146 limits you to languages that compile down to that bytecode; today, those 147 are Serpent and Solidity. 148 149 In contrast, our approach is to decouple the consensus engine and P2P 150 layers from the details of the application state of the particular 151 blockchain application. We do this by abstracting away the details of 152 the application to an interface, which is implemented as a socket 153 protocol. 154 155 Thus we have an interface, the Application BlockChain Interface (ABCI), 156 and its primary implementation, the Tendermint Socket Protocol (TSP, or 157 Teaspoon). 158 159 ### Intro to ABCI 160 161 [Tendermint Core](https://github.com/DFWallet/tendermint-cosmos) (the 162 "consensus engine") communicates with the application via a socket 163 protocol that satisfies the ABCI. 164 165 To draw an analogy, lets talk about a well-known cryptocurrency, 166 Bitcoin. Bitcoin is a cryptocurrency blockchain where each node 167 maintains a fully audited Unspent Transaction Output (UTXO) database. If 168 one wanted to create a Bitcoin-like system on top of ABCI, Tendermint 169 Core would be responsible for 170 171 - Sharing blocks and transactions between nodes 172 - Establishing a canonical/immutable order of transactions 173 (the blockchain) 174 175 The application will be responsible for 176 177 - Maintaining the UTXO database 178 - Validating cryptographic signatures of transactions 179 - Preventing transactions from spending non-existent transactions 180 - Allowing clients to query the UTXO database. 181 182 Tendermint is able to decompose the blockchain design by offering a very 183 simple API (ie. the ABCI) between the application process and consensus 184 process. 185 186 The ABCI consists of 3 primary message types that get delivered from the 187 core to the application. The application replies with corresponding 188 response messages. 189 190 The messages are specified here: [ABCI Message 191 Types](https://github.com/DFWallet/tendermint-cosmos/blob/master/abci/README.md#message-types). 192 193 The **DeliverTx** message is the work horse of the application. Each 194 transaction in the blockchain is delivered with this message. The 195 application needs to validate each transaction received with the 196 **DeliverTx** message against the current state, application protocol, 197 and the cryptographic credentials of the transaction. A validated 198 transaction then needs to update the application state — by binding a 199 value into a key values store, or by updating the UTXO database, for 200 instance. 201 202 The **CheckTx** message is similar to **DeliverTx**, but it's only for 203 validating transactions. Tendermint Core's mempool first checks the 204 validity of a transaction with **CheckTx**, and only relays valid 205 transactions to its peers. For instance, an application may check an 206 incrementing sequence number in the transaction and return an error upon 207 **CheckTx** if the sequence number is old. Alternatively, they might use 208 a capabilities based system that requires capabilities to be renewed 209 with every transaction. 210 211 The **Commit** message is used to compute a cryptographic commitment to 212 the current application state, to be placed into the next block header. 213 This has some handy properties. Inconsistencies in updating that state 214 will now appear as blockchain forks which catches a whole class of 215 programming errors. This also simplifies the development of secure 216 lightweight clients, as Merkle-hash proofs can be verified by checking 217 against the block hash, and that the block hash is signed by a quorum. 218 219 There can be multiple ABCI socket connections to an application. 220 Tendermint Core creates three ABCI connections to the application; one 221 for the validation of transactions when broadcasting in the mempool, one 222 for the consensus engine to run block proposals, and one more for 223 querying the application state. 224 225 It's probably evident that applications designers need to very carefully 226 design their message handlers to create a blockchain that does anything 227 useful but this architecture provides a place to start. The diagram 228 below illustrates the flow of messages via ABCI. 229 230 ![abci](../imgs/abci.png) 231 232 ## A Note on Determinism 233 234 The logic for blockchain transaction processing must be deterministic. 235 If the application logic weren't deterministic, consensus would not be 236 reached among the Tendermint Core replica nodes. 237 238 Solidity on Ethereum is a great language of choice for blockchain 239 applications because, among other reasons, it is a completely 240 deterministic programming language. However, it's also possible to 241 create deterministic applications using existing popular languages like 242 Java, C++, Python, or Go. Game programmers and blockchain developers are 243 already familiar with creating deterministic programs by avoiding 244 sources of non-determinism such as: 245 246 - random number generators (without deterministic seeding) 247 - race conditions on threads (or avoiding threads altogether) 248 - the system clock 249 - uninitialized memory (in unsafe programming languages like C 250 or C++) 251 - [floating point 252 arithmetic](http://gafferongames.com/networking-for-game-programmers/floating-point-determinism/) 253 - language features that are random (e.g. map iteration in Go) 254 255 While programmers can avoid non-determinism by being careful, it is also 256 possible to create a special linter or static analyzer for each language 257 to check for determinism. In the future we may work with partners to 258 create such tools. 259 260 ## Consensus Overview 261 262 Tendermint is an easy-to-understand, mostly asynchronous, BFT consensus 263 protocol. The protocol follows a simple state machine that looks like 264 this: 265 266 ![consensus-logic](../imgs/consensus_logic.png) 267 268 Participants in the protocol are called **validators**; they take turns 269 proposing blocks of transactions and voting on them. Blocks are 270 committed in a chain, with one block at each **height**. A block may 271 fail to be committed, in which case the protocol moves to the next 272 **round**, and a new validator gets to propose a block for that height. 273 Two stages of voting are required to successfully commit a block; we 274 call them **pre-vote** and **pre-commit**. A block is committed when 275 more than 2/3 of validators pre-commit for the same block in the same 276 round. 277 278 There is a picture of a couple doing the polka because validators are 279 doing something like a polka dance. When more than two-thirds of the 280 validators pre-vote for the same block, we call that a **polka**. Every 281 pre-commit must be justified by a polka in the same round. 282 283 Validators may fail to commit a block for a number of reasons; the 284 current proposer may be offline, or the network may be slow. Tendermint 285 allows them to establish that a validator should be skipped. Validators 286 wait a small amount of time to receive a complete proposal block from 287 the proposer before voting to move to the next round. This reliance on a 288 timeout is what makes Tendermint a weakly synchronous protocol, rather 289 than an asynchronous one. However, the rest of the protocol is 290 asynchronous, and validators only make progress after hearing from more 291 than two-thirds of the validator set. A simplifying element of 292 Tendermint is that it uses the same mechanism to commit a block as it 293 does to skip to the next round. 294 295 Assuming less than one-third of the validators are Byzantine, Tendermint 296 guarantees that safety will never be violated - that is, validators will 297 never commit conflicting blocks at the same height. To do this it 298 introduces a few **locking** rules which modulate which paths can be 299 followed in the flow diagram. Once a validator precommits a block, it is 300 locked on that block. Then, 301 302 1. it must prevote for the block it is locked on 303 2. it can only unlock, and precommit for a new block, if there is a 304 polka for that block in a later round 305 306 ## Stake 307 308 In many systems, not all validators will have the same "weight" in the 309 consensus protocol. Thus, we are not so much interested in one-third or 310 two-thirds of the validators, but in those proportions of the total 311 voting power, which may not be uniformly distributed across individual 312 validators. 313 314 Since Tendermint can replicate arbitrary applications, it is possible to 315 define a currency, and denominate the voting power in that currency. 316 When voting power is denominated in a native currency, the system is 317 often referred to as Proof-of-Stake. Validators can be forced, by logic 318 in the application, to "bond" their currency holdings in a security 319 deposit that can be destroyed if they're found to misbehave in the 320 consensus protocol. This adds an economic element to the security of the 321 protocol, allowing one to quantify the cost of violating the assumption 322 that less than one-third of voting power is Byzantine. 323 324 The [Cosmos Network](https://cosmos.network) is designed to use this 325 Proof-of-Stake mechanism across an array of cryptocurrencies implemented 326 as ABCI applications. 327 328 The following diagram is Tendermint in a (technical) nutshell. 329 330 ![tx-flow](../imgs/tm-transaction-flow.png)