github.com/franono/tendermint@v0.32.2-0.20200527150959-749313264ce9/docs/guides/go.md (about) 1 <!--- 2 order: 1 3 ---> 4 5 # Creating an application in Go 6 7 ## Guide Assumptions 8 9 This guide is designed for beginners who want to get started with a Tendermint 10 Core application from scratch. It does not assume that you have any prior 11 experience with Tendermint Core. 12 13 Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state 14 transition machine - written in any programming language - and securely 15 replicates it on many machines. 16 17 Although Tendermint Core is written in the Golang programming language, prior 18 knowledge of it is not required for this guide. You can learn it as we go due 19 to it's simplicity. However, you may want to go through [Learn X in Y minutes 20 Where X=Go](https://learnxinyminutes.com/docs/go/) first to familiarize 21 yourself with the syntax. 22 23 By following along with this guide, you'll create a Tendermint Core project 24 called kvstore, a (very) simple distributed BFT key-value store. 25 26 ## Built-in app vs external app 27 28 To get maximum performance it is better to run your application alongside 29 Tendermint Core. [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is written 30 this way. Please refer to [Writing a built-in Tendermint Core application in 31 Go](./go-built-in.md) guide for details. 32 33 Having a separate application might give you better security guarantees as two 34 processes would be communicating via established binary protocol. Tendermint 35 Core will not have access to application's state. 36 37 ## 1.1 Installing Go 38 39 Please refer to [the official guide for installing 40 Go](https://golang.org/doc/install). 41 42 Verify that you have the latest version of Go installed: 43 44 ```sh 45 $ go version 46 go version go1.13.1 darwin/amd64 47 ``` 48 49 Make sure you have `$GOPATH` environment variable set: 50 51 ```sh 52 $ echo $GOPATH 53 /Users/melekes/go 54 ``` 55 56 ## 1.2 Creating a new Go project 57 58 We'll start by creating a new Go project. 59 60 ```sh 61 $ mkdir kvstore 62 $ cd kvstore 63 ``` 64 65 Inside the example directory create a `main.go` file with the following content: 66 67 ```go 68 package main 69 70 import ( 71 "fmt" 72 ) 73 74 func main() { 75 fmt.Println("Hello, Tendermint Core") 76 } 77 ``` 78 79 When run, this should print "Hello, Tendermint Core" to the standard output. 80 81 ```sh 82 $ go run main.go 83 Hello, Tendermint Core 84 ``` 85 86 ## 1.3 Writing a Tendermint Core application 87 88 Tendermint Core communicates with the application through the Application 89 BlockChain Interface (ABCI). All message types are defined in the [protobuf 90 file](https://github.com/franono/tendermint/blob/master/abci/types/types.proto). 91 This allows Tendermint Core to run applications written in any programming 92 language. 93 94 Create a file called `app.go` with the following content: 95 96 ```go 97 package main 98 99 import ( 100 abcitypes "github.com/franono/tendermint/abci/types" 101 ) 102 103 type KVStoreApplication struct {} 104 105 var _ abcitypes.Application = (*KVStoreApplication)(nil) 106 107 func NewKVStoreApplication() *KVStoreApplication { 108 return &KVStoreApplication{} 109 } 110 111 func (KVStoreApplication) Info(req abcitypes.RequestInfo) abcitypes.ResponseInfo { 112 return abcitypes.ResponseInfo{} 113 } 114 115 func (KVStoreApplication) SetOption(req abcitypes.RequestSetOption) abcitypes.ResponseSetOption { 116 return abcitypes.ResponseSetOption{} 117 } 118 119 func (KVStoreApplication) DeliverTx(req abcitypes.RequestDeliverTx) abcitypes.ResponseDeliverTx { 120 return abcitypes.ResponseDeliverTx{Code: 0} 121 } 122 123 func (KVStoreApplication) CheckTx(req abcitypes.RequestCheckTx) abcitypes.ResponseCheckTx { 124 return abcitypes.ResponseCheckTx{Code: 0} 125 } 126 127 func (KVStoreApplication) Commit() abcitypes.ResponseCommit { 128 return abcitypes.ResponseCommit{} 129 } 130 131 func (KVStoreApplication) Query(req abcitypes.RequestQuery) abcitypes.ResponseQuery { 132 return abcitypes.ResponseQuery{Code: 0} 133 } 134 135 func (KVStoreApplication) InitChain(req abcitypes.RequestInitChain) abcitypes.ResponseInitChain { 136 return abcitypes.ResponseInitChain{} 137 } 138 139 func (KVStoreApplication) BeginBlock(req abcitypes.RequestBeginBlock) abcitypes.ResponseBeginBlock { 140 return abcitypes.ResponseBeginBlock{} 141 } 142 143 func (KVStoreApplication) EndBlock(req abcitypes.RequestEndBlock) abcitypes.ResponseEndBlock { 144 return abcitypes.ResponseEndBlock{} 145 } 146 ``` 147 148 Now I will go through each method explaining when it's called and adding 149 required business logic. 150 151 ### 1.3.1 CheckTx 152 153 When a new transaction is added to the Tendermint Core, it will ask the 154 application to check it (validate the format, signatures, etc.). 155 156 ```go 157 import "bytes" 158 159 func (app *KVStoreApplication) isValid(tx []byte) (code uint32) { 160 // check format 161 parts := bytes.Split(tx, []byte("=")) 162 if len(parts) != 2 { 163 return 1 164 } 165 166 key, value := parts[0], parts[1] 167 168 // check if the same key=value already exists 169 err := app.db.View(func(txn *badger.Txn) error { 170 item, err := txn.Get(key) 171 if err != nil && err != badger.ErrKeyNotFound { 172 return err 173 } 174 if err == nil { 175 return item.Value(func(val []byte) error { 176 if bytes.Equal(val, value) { 177 code = 2 178 } 179 return nil 180 }) 181 } 182 return nil 183 }) 184 if err != nil { 185 panic(err) 186 } 187 188 return code 189 } 190 191 func (app *KVStoreApplication) CheckTx(req abcitypes.RequestCheckTx) abcitypes.ResponseCheckTx { 192 code := app.isValid(req.Tx) 193 return abcitypes.ResponseCheckTx{Code: code, GasWanted: 1} 194 } 195 ``` 196 197 Don't worry if this does not compile yet. 198 199 If the transaction does not have a form of `{bytes}={bytes}`, we return `1` 200 code. When the same key=value already exist (same key and value), we return `2` 201 code. For others, we return a zero code indicating that they are valid. 202 203 Note that anything with non-zero code will be considered invalid (`-1`, `100`, 204 etc.) by Tendermint Core. 205 206 Valid transactions will eventually be committed given they are not too big and 207 have enough gas. To learn more about gas, check out ["the 208 specification"](https://docs.tendermint.com/master/spec/abci/apps.html#gas). 209 210 For the underlying key-value store we'll use 211 [badger](https://github.com/dgraph-io/badger), which is an embeddable, 212 persistent and fast key-value (KV) database. 213 214 ```go 215 import "github.com/dgraph-io/badger" 216 217 type KVStoreApplication struct { 218 db *badger.DB 219 currentBatch *badger.Txn 220 } 221 222 func NewKVStoreApplication(db *badger.DB) *KVStoreApplication { 223 return &KVStoreApplication{ 224 db: db, 225 } 226 } 227 ``` 228 229 ### 1.3.2 BeginBlock -> DeliverTx -> EndBlock -> Commit 230 231 When Tendermint Core has decided on the block, it's transferred to the 232 application in 3 parts: `BeginBlock`, one `DeliverTx` per transaction and 233 `EndBlock` in the end. DeliverTx are being transferred asynchronously, but the 234 responses are expected to come in order. 235 236 ``` 237 func (app *KVStoreApplication) BeginBlock(req abcitypes.RequestBeginBlock) abcitypes.ResponseBeginBlock { 238 app.currentBatch = app.db.NewTransaction(true) 239 return abcitypes.ResponseBeginBlock{} 240 } 241 242 ``` 243 244 Here we create a batch, which will store block's transactions. 245 246 ```go 247 func (app *KVStoreApplication) DeliverTx(req abcitypes.RequestDeliverTx) abcitypes.ResponseDeliverTx { 248 code := app.isValid(req.Tx) 249 if code != 0 { 250 return abcitypes.ResponseDeliverTx{Code: code} 251 } 252 253 parts := bytes.Split(req.Tx, []byte("=")) 254 key, value := parts[0], parts[1] 255 256 err := app.currentBatch.Set(key, value) 257 if err != nil { 258 panic(err) 259 } 260 261 return abcitypes.ResponseDeliverTx{Code: 0} 262 } 263 ``` 264 265 If the transaction is badly formatted or the same key=value already exist, we 266 again return the non-zero code. Otherwise, we add it to the current batch. 267 268 In the current design, a block can include incorrect transactions (those who 269 passed CheckTx, but failed DeliverTx or transactions included by the proposer 270 directly). This is done for performance reasons. 271 272 Note we can't commit transactions inside the `DeliverTx` because in such case 273 `Query`, which may be called in parallel, will return inconsistent data (i.e. 274 it will report that some value already exist even when the actual block was not 275 yet committed). 276 277 `Commit` instructs the application to persist the new state. 278 279 ```go 280 func (app *KVStoreApplication) Commit() abcitypes.ResponseCommit { 281 app.currentBatch.Commit() 282 return abcitypes.ResponseCommit{Data: []byte{}} 283 } 284 ``` 285 286 ### 1.3.3 Query 287 288 Now, when the client wants to know whenever a particular key/value exist, it 289 will call Tendermint Core RPC `/abci_query` endpoint, which in turn will call 290 the application's `Query` method. 291 292 Applications are free to provide their own APIs. But by using Tendermint Core 293 as a proxy, clients (including [light client 294 package](https://godoc.org/github.com/franono/tendermint/lite2)) can leverage 295 the unified API across different applications. Plus they won't have to call the 296 otherwise separate Tendermint Core API for additional proofs. 297 298 Note we don't include a proof here. 299 300 ```go 301 func (app *KVStoreApplication) Query(reqQuery abcitypes.RequestQuery) (resQuery abcitypes.ResponseQuery) { 302 resQuery.Key = reqQuery.Data 303 err := app.db.View(func(txn *badger.Txn) error { 304 item, err := txn.Get(reqQuery.Data) 305 if err != nil && err != badger.ErrKeyNotFound { 306 return err 307 } 308 if err == badger.ErrKeyNotFound { 309 resQuery.Log = "does not exist" 310 } else { 311 return item.Value(func(val []byte) error { 312 resQuery.Log = "exists" 313 resQuery.Value = val 314 return nil 315 }) 316 } 317 return nil 318 }) 319 if err != nil { 320 panic(err) 321 } 322 return 323 } 324 ``` 325 326 The complete specification can be found 327 [here](https://docs.tendermint.com/master/spec/abci/). 328 329 ## 1.4 Starting an application and a Tendermint Core instances 330 331 Put the following code into the "main.go" file: 332 333 ```go 334 package main 335 336 import ( 337 "flag" 338 "fmt" 339 "os" 340 "os/signal" 341 "syscall" 342 343 "github.com/dgraph-io/badger" 344 345 abciserver "github.com/franono/tendermint/abci/server" 346 "github.com/franono/tendermint/libs/log" 347 ) 348 349 var socketAddr string 350 351 func init() { 352 flag.StringVar(&socketAddr, "socket-addr", "unix://example.sock", "Unix domain socket address") 353 } 354 355 func main() { 356 db, err := badger.Open(badger.DefaultOptions("/tmp/badger")) 357 if err != nil { 358 fmt.Fprintf(os.Stderr, "failed to open badger db: %v", err) 359 os.Exit(1) 360 } 361 defer db.Close() 362 app := NewKVStoreApplication(db) 363 364 flag.Parse() 365 366 logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout)) 367 368 server := abciserver.NewSocketServer(socketAddr, app) 369 server.SetLogger(logger) 370 if err := server.Start(); err != nil { 371 fmt.Fprintf(os.Stderr, "error starting socket server: %v", err) 372 os.Exit(1) 373 } 374 defer server.Stop() 375 376 c := make(chan os.Signal, 1) 377 signal.Notify(c, os.Interrupt, syscall.SIGTERM) 378 <-c 379 os.Exit(0) 380 } 381 ``` 382 383 This is a huge blob of code, so let's break it down into pieces. 384 385 First, we initialize the Badger database and create an app instance: 386 387 ```go 388 db, err := badger.Open(badger.DefaultOptions("/tmp/badger")) 389 if err != nil { 390 fmt.Fprintf(os.Stderr, "failed to open badger db: %v", err) 391 os.Exit(1) 392 } 393 defer db.Close() 394 app := NewKVStoreApplication(db) 395 ``` 396 397 For **Windows** users, restarting this app will make badger throw an error as it requires value log to be truncated. For more information on this, visit [here](https://github.com/dgraph-io/badger/issues/744). 398 This can be avoided by setting the truncate option to true, like this: 399 400 ```go 401 db, err := badger.Open(badger.DefaultOptions("/tmp/badger").WithTruncate(true)) 402 ``` 403 404 Then we start the ABCI server and add some signal handling to gracefully stop 405 it upon receiving SIGTERM or Ctrl-C. Tendermint Core will act as a client, 406 which connects to our server and send us transactions and other messages. 407 408 ```go 409 server := abciserver.NewSocketServer(socketAddr, app) 410 server.SetLogger(logger) 411 if err := server.Start(); err != nil { 412 fmt.Fprintf(os.Stderr, "error starting socket server: %v", err) 413 os.Exit(1) 414 } 415 defer server.Stop() 416 417 c := make(chan os.Signal, 1) 418 signal.Notify(c, os.Interrupt, syscall.SIGTERM) 419 <-c 420 os.Exit(0) 421 ``` 422 423 ## 1.5 Getting Up and Running 424 425 We are going to use [Go modules](https://github.com/golang/go/wiki/Modules) for 426 dependency management. 427 428 ```sh 429 $ export GO111MODULE=on 430 $ go mod init github.com/me/example 431 $ go build 432 ``` 433 434 This should build the binary. 435 436 To create a default configuration, nodeKey and private validator files, let's 437 execute `tendermint init`. But before we do that, we will need to install 438 Tendermint Core. Please refer to [the official 439 guide](https://docs.tendermint.com/master/introduction/install.html). If you're 440 installing from source, don't forget to checkout the latest release (`git checkout vX.Y.Z`). 441 442 ```sh 443 $ rm -rf /tmp/example 444 $ TMHOME="/tmp/example" tendermint init 445 446 I[2019-07-16|18:20:36.480] Generated private validator module=main keyFile=/tmp/example/config/priv_validator_key.json stateFile=/tmp/example2/data/priv_validator_state.json 447 I[2019-07-16|18:20:36.481] Generated node key module=main path=/tmp/example/config/node_key.json 448 I[2019-07-16|18:20:36.482] Generated genesis file module=main path=/tmp/example/config/genesis.json 449 ``` 450 451 Feel free to explore the generated files, which can be found at 452 `/tmp/example/config` directory. Documentation on the config can be found 453 [here](https://docs.tendermint.com/master/tendermint-core/configuration.html). 454 455 We are ready to start our application: 456 457 ```sh 458 $ rm example.sock 459 $ ./example 460 461 badger 2019/07/16 18:25:11 INFO: All 0 tables opened in 0s 462 badger 2019/07/16 18:25:11 INFO: Replaying file id: 0 at offset: 0 463 badger 2019/07/16 18:25:11 INFO: Replay took: 300.4s 464 I[2019-07-16|18:25:11.523] Starting ABCIServer impl=ABCIServ 465 ``` 466 467 Then we need to start Tendermint Core and point it to our application. Staying 468 within the application directory execute: 469 470 ```sh 471 $ TMHOME="/tmp/example" tendermint node --proxy_app=unix://example.sock 472 473 I[2019-07-16|18:26:20.362] Version info module=main software=0.32.1 block=10 p2p=7 474 I[2019-07-16|18:26:20.383] Starting Node module=main impl=Node 475 E[2019-07-16|18:26:20.392] Couldn't connect to any seeds module=p2p 476 I[2019-07-16|18:26:20.394] Started node module=main nodeInfo="{ProtocolVersion:{P2P:7 Block:10 App:0} ID_:8dab80770ae8e295d4ce905d86af78c4ff634b79 ListenAddr:tcp://0.0.0.0:26656 Network:test-chain-nIO96P Version:0.32.1 Channels:4020212223303800 Moniker:app48.fun-box.ru Other:{TxIndex:on RPCAddress:tcp://127.0.0.1:26657}}" 477 I[2019-07-16|18:26:21.440] Executed block module=state height=1 validTxs=0 invalidTxs=0 478 I[2019-07-16|18:26:21.446] Committed state module=state height=1 txs=0 appHash= 479 ``` 480 481 This should start the full node and connect to our ABCI application. 482 483 ``` 484 I[2019-07-16|18:25:11.525] Waiting for new connection... 485 I[2019-07-16|18:26:20.329] Accepted a new connection 486 I[2019-07-16|18:26:20.329] Waiting for new connection... 487 I[2019-07-16|18:26:20.330] Accepted a new connection 488 I[2019-07-16|18:26:20.330] Waiting for new connection... 489 I[2019-07-16|18:26:20.330] Accepted a new connection 490 ``` 491 492 Now open another tab in your terminal and try sending a transaction: 493 494 ```sh 495 $ curl -s 'localhost:26657/broadcast_tx_commit?tx="tendermint=rocks"' 496 { 497 "jsonrpc": "2.0", 498 "id": "", 499 "result": { 500 "check_tx": { 501 "gasWanted": "1" 502 }, 503 "deliver_tx": {}, 504 "hash": "CDD3C6DFA0A08CAEDF546F9938A2EEC232209C24AA0E4201194E0AFB78A2C2BB", 505 "height": "33" 506 } 507 ``` 508 509 Response should contain the height where this transaction was committed. 510 511 Now let's check if the given key now exists and its value: 512 513 ``` 514 $ curl -s 'localhost:26657/abci_query?data="tendermint"' 515 { 516 "jsonrpc": "2.0", 517 "id": "", 518 "result": { 519 "response": { 520 "log": "exists", 521 "key": "dGVuZGVybWludA==", 522 "value": "cm9ja3My" 523 } 524 } 525 } 526 ``` 527 528 "dGVuZGVybWludA==" and "cm9ja3M=" are the base64-encoding of the ASCII of 529 "tendermint" and "rocks" accordingly. 530 531 ## Outro 532 533 I hope everything went smoothly and your first, but hopefully not the last, 534 Tendermint Core application is up and running. If not, please [open an issue on 535 Github](https://github.com/franono/tendermint/issues/new/choose). To dig 536 deeper, read [the docs](https://docs.tendermint.com/master/).