github.com/vipernet-xyz/tm@v0.34.24/docs/tutorials/go.md (about) 1 <!--- 2 order: 1 3 ---> 4 5 # Creating an application in Go 6 7 ## Guide Assumptions 8 9 This guide is designed for beginners who want to get started with a Tendermint 10 Core application from scratch. It does not assume that you have any prior 11 experience with Tendermint Core. 12 13 Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state 14 transition machine - written in any programming language - and securely 15 replicates it on many machines. 16 17 Although Tendermint Core is written in the Golang programming language, prior 18 knowledge of it is not required for this guide. You can learn it as we go due 19 to it's simplicity. However, you may want to go through [Learn X in Y minutes 20 Where X=Go](https://learnxinyminutes.com/docs/go/) first to familiarize 21 yourself with the syntax. 22 23 By following along with this guide, you'll create a Tendermint Core project 24 called kvstore, a (very) simple distributed BFT key-value store. 25 26 ## Built-in app vs external app 27 28 To get maximum performance it is better to run your application alongside 29 Tendermint Core. [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is written 30 this way. Please refer to [Writing a built-in Tendermint Core application in 31 Go](./go-built-in.md) guide for details. 32 33 Having a separate application might give you better security guarantees as two 34 processes would be communicating via established binary protocol. Tendermint 35 Core will not have access to application's state. 36 37 ## 1.1 Installing Go 38 39 Please refer to [the official guide for installing 40 Go](https://golang.org/doc/install). 41 42 Verify that you have the latest version of Go installed: 43 44 ```bash 45 $ go version 46 go version go1.15.x darwin/amd64 47 ``` 48 49 Make sure you have `$GOPATH` environment variable set: 50 51 ```bash 52 echo $GOPATH 53 /Users/melekes/go 54 ``` 55 56 ## 1.2 Creating a new Go project 57 58 We'll start by creating a new Go project. 59 60 ```bash 61 mkdir kvstore 62 cd kvstore 63 ``` 64 65 Inside the example directory create a `main.go` file with the following content: 66 67 ```go 68 package main 69 70 import ( 71 "fmt" 72 ) 73 74 func main() { 75 fmt.Println("Hello, Tendermint Core") 76 } 77 ``` 78 79 When run, this should print "Hello, Tendermint Core" to the standard output. 80 81 ```bash 82 go run main.go 83 Hello, Tendermint Core 84 ``` 85 86 ## 1.3 Writing a Tendermint Core application 87 88 Tendermint Core communicates with the application through the Application 89 BlockChain Interface (ABCI). All message types are defined in the [protobuf 90 file](https://github.com/vipernet-xyz/tm/blob/v0.34.x/proto/tendermint/abci/types.proto). 91 This allows Tendermint Core to run applications written in any programming 92 language. 93 94 Create a file called `app.go` with the following content: 95 96 ```go 97 package main 98 99 import ( 100 abcitypes "github.com/vipernet-xyz/tm/abci/types" 101 ) 102 103 type KVStoreApplication struct {} 104 105 var _ abcitypes.Application = (*KVStoreApplication)(nil) 106 107 func NewKVStoreApplication() *KVStoreApplication { 108 return &KVStoreApplication{} 109 } 110 111 func (KVStoreApplication) Info(req abcitypes.RequestInfo) abcitypes.ResponseInfo { 112 return abcitypes.ResponseInfo{} 113 } 114 115 func (KVStoreApplication) SetOption(req abcitypes.RequestSetOption) abcitypes.ResponseSetOption { 116 return abcitypes.ResponseSetOption{} 117 } 118 119 func (KVStoreApplication) DeliverTx(req abcitypes.RequestDeliverTx) abcitypes.ResponseDeliverTx { 120 return abcitypes.ResponseDeliverTx{Code: 0} 121 } 122 123 func (KVStoreApplication) CheckTx(req abcitypes.RequestCheckTx) abcitypes.ResponseCheckTx { 124 return abcitypes.ResponseCheckTx{Code: 0} 125 } 126 127 func (KVStoreApplication) Commit() abcitypes.ResponseCommit { 128 return abcitypes.ResponseCommit{} 129 } 130 131 func (KVStoreApplication) Query(req abcitypes.RequestQuery) abcitypes.ResponseQuery { 132 return abcitypes.ResponseQuery{Code: 0} 133 } 134 135 func (KVStoreApplication) InitChain(req abcitypes.RequestInitChain) abcitypes.ResponseInitChain { 136 return abcitypes.ResponseInitChain{} 137 } 138 139 func (KVStoreApplication) BeginBlock(req abcitypes.RequestBeginBlock) abcitypes.ResponseBeginBlock { 140 return abcitypes.ResponseBeginBlock{} 141 } 142 143 func (KVStoreApplication) EndBlock(req abcitypes.RequestEndBlock) abcitypes.ResponseEndBlock { 144 return abcitypes.ResponseEndBlock{} 145 } 146 147 func (KVStoreApplication) ListSnapshots(abcitypes.RequestListSnapshots) abcitypes.ResponseListSnapshots { 148 return abcitypes.ResponseListSnapshots{} 149 } 150 151 func (KVStoreApplication) OfferSnapshot(abcitypes.RequestOfferSnapshot) abcitypes.ResponseOfferSnapshot { 152 return abcitypes.ResponseOfferSnapshot{} 153 } 154 155 func (KVStoreApplication) LoadSnapshotChunk(abcitypes.RequestLoadSnapshotChunk) abcitypes.ResponseLoadSnapshotChunk { 156 return abcitypes.ResponseLoadSnapshotChunk{} 157 } 158 159 func (KVStoreApplication) ApplySnapshotChunk(abcitypes.RequestApplySnapshotChunk) abcitypes.ResponseApplySnapshotChunk { 160 return abcitypes.ResponseApplySnapshotChunk{} 161 } 162 ``` 163 164 Now I will go through each method explaining when it's called and adding 165 required business logic. 166 167 ### 1.3.1 CheckTx 168 169 When a new transaction is added to the Tendermint Core, it will ask the 170 application to check it (validate the format, signatures, etc.). 171 172 ```go 173 import "bytes" 174 175 func (app *KVStoreApplication) isValid(tx []byte) (code uint32) { 176 // check format 177 parts := bytes.Split(tx, []byte("=")) 178 if len(parts) != 2 { 179 return 1 180 } 181 182 key, value := parts[0], parts[1] 183 184 // check if the same key=value already exists 185 err := app.db.View(func(txn *badger.Txn) error { 186 item, err := txn.Get(key) 187 if err != nil && err != badger.ErrKeyNotFound { 188 return err 189 } 190 if err == nil { 191 return item.Value(func(val []byte) error { 192 if bytes.Equal(val, value) { 193 code = 2 194 } 195 return nil 196 }) 197 } 198 return nil 199 }) 200 if err != nil { 201 panic(err) 202 } 203 204 return code 205 } 206 207 func (app *KVStoreApplication) CheckTx(req abcitypes.RequestCheckTx) abcitypes.ResponseCheckTx { 208 code := app.isValid(req.Tx) 209 return abcitypes.ResponseCheckTx{Code: code, GasWanted: 1} 210 } 211 ``` 212 213 Don't worry if this does not compile yet. 214 215 If the transaction does not have a form of `{bytes}={bytes}`, we return `1` 216 code. When the same key=value already exist (same key and value), we return `2` 217 code. For others, we return a zero code indicating that they are valid. 218 219 Note that anything with non-zero code will be considered invalid (`-1`, `100`, 220 etc.) by Tendermint Core. 221 222 Valid transactions will eventually be committed given they are not too big and 223 have enough gas. To learn more about gas, check out ["the 224 specification"](https://github.com/vipernet-xyz/tm/blob/v0.34.x/spec/abci/apps.md#gas). 225 226 For the underlying key-value store we'll use 227 [badger](https://github.com/dgraph-io/badger), which is an embeddable, 228 persistent and fast key-value (KV) database. 229 230 ```go 231 import "github.com/dgraph-io/badger" 232 233 type KVStoreApplication struct { 234 db *badger.DB 235 currentBatch *badger.Txn 236 } 237 238 func NewKVStoreApplication(db *badger.DB) *KVStoreApplication { 239 return &KVStoreApplication{ 240 db: db, 241 } 242 } 243 ``` 244 245 ### 1.3.2 BeginBlock -> DeliverTx -> EndBlock -> Commit 246 247 When Tendermint Core has decided on the block, it's transferred to the 248 application in 3 parts: `BeginBlock`, one `DeliverTx` per transaction and 249 `EndBlock` in the end. DeliverTx are being transferred asynchronously, but the 250 responses are expected to come in order. 251 252 ```go 253 func (app *KVStoreApplication) BeginBlock(req abcitypes.RequestBeginBlock) abcitypes.ResponseBeginBlock { 254 app.currentBatch = app.db.NewTransaction(true) 255 return abcitypes.ResponseBeginBlock{} 256 } 257 258 ``` 259 260 Here we create a batch, which will store block's transactions. 261 262 ```go 263 func (app *KVStoreApplication) DeliverTx(req abcitypes.RequestDeliverTx) abcitypes.ResponseDeliverTx { 264 code := app.isValid(req.Tx) 265 if code != 0 { 266 return abcitypes.ResponseDeliverTx{Code: code} 267 } 268 269 parts := bytes.Split(req.Tx, []byte("=")) 270 key, value := parts[0], parts[1] 271 272 err := app.currentBatch.Set(key, value) 273 if err != nil { 274 panic(err) 275 } 276 277 return abcitypes.ResponseDeliverTx{Code: 0} 278 } 279 ``` 280 281 If the transaction is badly formatted or the same key=value already exist, we 282 again return the non-zero code. Otherwise, we add it to the current batch. 283 284 In the current design, a block can include incorrect transactions (those who 285 passed CheckTx, but failed DeliverTx or transactions included by the proposer 286 directly). This is done for performance reasons. 287 288 Note we can't commit transactions inside the `DeliverTx` because in such case 289 `Query`, which may be called in parallel, will return inconsistent data (i.e. 290 it will report that some value already exist even when the actual block was not 291 yet committed). 292 293 `Commit` instructs the application to persist the new state. 294 295 ```go 296 func (app *KVStoreApplication) Commit() abcitypes.ResponseCommit { 297 app.currentBatch.Commit() 298 return abcitypes.ResponseCommit{Data: []byte{}} 299 } 300 ``` 301 302 ### 1.3.3 Query 303 304 Now, when the client wants to know whenever a particular key/value exist, it 305 will call Tendermint Core RPC `/abci_query` endpoint, which in turn will call 306 the application's `Query` method. 307 308 Applications are free to provide their own APIs. But by using Tendermint Core 309 as a proxy, clients (including [light client 310 package](https://godoc.org/github.com/vipernet-xyz/tm/light)) can leverage 311 the unified API across different applications. Plus they won't have to call the 312 otherwise separate Tendermint Core API for additional proofs. 313 314 Note we don't include a proof here. 315 316 ```go 317 func (app *KVStoreApplication) Query(reqQuery abcitypes.RequestQuery) (resQuery abcitypes.ResponseQuery) { 318 resQuery.Key = reqQuery.Data 319 err := app.db.View(func(txn *badger.Txn) error { 320 item, err := txn.Get(reqQuery.Data) 321 if err != nil && err != badger.ErrKeyNotFound { 322 return err 323 } 324 if err == badger.ErrKeyNotFound { 325 resQuery.Log = "does not exist" 326 } else { 327 return item.Value(func(val []byte) error { 328 resQuery.Log = "exists" 329 resQuery.Value = val 330 return nil 331 }) 332 } 333 return nil 334 }) 335 if err != nil { 336 panic(err) 337 } 338 return 339 } 340 ``` 341 342 The complete specification can be found 343 [here](https://github.com/vipernet-xyz/tm/tree/v0.34.x/spec/abci/). 344 345 ## 1.4 Starting an application and a Tendermint Core instances 346 347 Put the following code into the "main.go" file: 348 349 ```go 350 package main 351 352 import ( 353 "flag" 354 "fmt" 355 "os" 356 "os/signal" 357 "syscall" 358 359 "github.com/dgraph-io/badger" 360 361 abciserver "github.com/vipernet-xyz/tm/abci/server" 362 "github.com/vipernet-xyz/tm/libs/log" 363 ) 364 365 var socketAddr string 366 367 func init() { 368 flag.StringVar(&socketAddr, "socket-addr", "unix://example.sock", "Unix domain socket address") 369 } 370 371 func main() { 372 db, err := badger.Open(badger.DefaultOptions("/tmp/badger")) 373 if err != nil { 374 fmt.Fprintf(os.Stderr, "failed to open badger db: %v", err) 375 os.Exit(1) 376 } 377 defer db.Close() 378 app := NewKVStoreApplication(db) 379 380 flag.Parse() 381 382 logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout)) 383 384 server := abciserver.NewSocketServer(socketAddr, app) 385 server.SetLogger(logger) 386 if err := server.Start(); err != nil { 387 fmt.Fprintf(os.Stderr, "error starting socket server: %v", err) 388 os.Exit(1) 389 } 390 defer server.Stop() 391 392 c := make(chan os.Signal, 1) 393 signal.Notify(c, os.Interrupt, syscall.SIGTERM) 394 <-c 395 os.Exit(0) 396 } 397 ``` 398 399 This is a huge blob of code, so let's break it down into pieces. 400 401 First, we initialize the Badger database and create an app instance: 402 403 ```go 404 db, err := badger.Open(badger.DefaultOptions("/tmp/badger")) 405 if err != nil { 406 fmt.Fprintf(os.Stderr, "failed to open badger db: %v", err) 407 os.Exit(1) 408 } 409 defer db.Close() 410 app := NewKVStoreApplication(db) 411 ``` 412 413 For **Windows** users, restarting this app will make badger throw an error as it requires value log to be truncated. For more information on this, visit [here](https://github.com/dgraph-io/badger/issues/744). 414 This can be avoided by setting the truncate option to true, like this: 415 416 ```go 417 db, err := badger.Open(badger.DefaultOptions("/tmp/badger").WithTruncate(true)) 418 ``` 419 420 Then we start the ABCI server and add some signal handling to gracefully stop 421 it upon receiving SIGTERM or Ctrl-C. Tendermint Core will act as a client, 422 which connects to our server and send us transactions and other messages. 423 424 ```go 425 server := abciserver.NewSocketServer(socketAddr, app) 426 server.SetLogger(logger) 427 if err := server.Start(); err != nil { 428 fmt.Fprintf(os.Stderr, "error starting socket server: %v", err) 429 os.Exit(1) 430 } 431 defer server.Stop() 432 433 c := make(chan os.Signal, 1) 434 signal.Notify(c, os.Interrupt, syscall.SIGTERM) 435 <-c 436 os.Exit(0) 437 ``` 438 439 ## 1.5 Getting Up and Running 440 441 We are going to use [Go modules](https://github.com/golang/go/wiki/Modules) for 442 dependency management. 443 444 ```bash 445 go mod init github.com/me/example 446 go get github.com/vipernet-xyz/tm/@v0.34.0 447 ``` 448 449 After running the above commands you will see two generated files, go.mod and go.sum. The go.mod file should look similar to: 450 451 ```go 452 module github.com/me/example 453 454 go 1.15 455 456 require ( 457 github.com/dgraph-io/badger v1.6.2 458 github.com/vipernet-xyz/tm v0.34.0 459 ) 460 ``` 461 462 Finally, we will build our binary: 463 464 ```sh 465 go build 466 ``` 467 468 To create a default configuration, nodeKey and private validator files, let's 469 execute `tendermint init`. But before we do that, we will need to install 470 Tendermint Core. Please refer to [the official 471 guide](https://docs.tendermint.com/v0.34/introduction/install.html). If you're 472 installing from source, don't forget to checkout the latest release (`git checkout vX.Y.Z`). 473 474 ```bash 475 rm -rf /tmp/example 476 TMHOME="/tmp/example" tendermint init 477 478 I[2019-07-16|18:20:36.480] Generated private validator module=main keyFile=/tmp/example/config/priv_validator_key.json stateFile=/tmp/example2/data/priv_validator_state.json 479 I[2019-07-16|18:20:36.481] Generated node key module=main path=/tmp/example/config/node_key.json 480 I[2019-07-16|18:20:36.482] Generated genesis file module=main path=/tmp/example/config/genesis.json 481 ``` 482 483 Feel free to explore the generated files, which can be found at 484 `/tmp/example/config` directory. Documentation on the config can be found 485 [here](https://docs.tendermint.com/v0.34/tendermint-core/configuration.html). 486 487 We are ready to start our application: 488 489 ```bash 490 rm example.sock 491 ./example 492 493 badger 2019/07/16 18:25:11 INFO: All 0 tables opened in 0s 494 badger 2019/07/16 18:25:11 INFO: Replaying file id: 0 at offset: 0 495 badger 2019/07/16 18:25:11 INFO: Replay took: 300.4s 496 I[2019-07-16|18:25:11.523] Starting ABCIServer impl=ABCIServ 497 ``` 498 499 Then we need to start Tendermint Core and point it to our application. Staying 500 within the application directory execute: 501 502 ```bash 503 TMHOME="/tmp/example" tendermint node --proxy_app=unix://example.sock 504 505 I[2019-07-16|18:26:20.362] Version info module=main software=0.32.1 block=10 p2p=7 506 I[2019-07-16|18:26:20.383] Starting Node module=main impl=Node 507 E[2019-07-16|18:26:20.392] Couldn't connect to any seeds module=p2p 508 I[2019-07-16|18:26:20.394] Started node module=main nodeInfo="{ProtocolVersion:{P2P:7 Block:10 App:0} ID_:8dab80770ae8e295d4ce905d86af78c4ff634b79 ListenAddr:tcp://0.0.0.0:26656 Network:test-chain-nIO96P Version:0.32.1 Channels:4020212223303800 Moniker:app48.fun-box.ru Other:{TxIndex:on RPCAddress:tcp://127.0.0.1:26657}}" 509 I[2019-07-16|18:26:21.440] Executed block module=state height=1 validTxs=0 invalidTxs=0 510 I[2019-07-16|18:26:21.446] Committed state module=state height=1 txs=0 appHash= 511 ``` 512 513 This should start the full node and connect to our ABCI application. 514 515 ```sh 516 I[2019-07-16|18:25:11.525] Waiting for new connection... 517 I[2019-07-16|18:26:20.329] Accepted a new connection 518 I[2019-07-16|18:26:20.329] Waiting for new connection... 519 I[2019-07-16|18:26:20.330] Accepted a new connection 520 I[2019-07-16|18:26:20.330] Waiting for new connection... 521 I[2019-07-16|18:26:20.330] Accepted a new connection 522 ``` 523 524 Now open another tab in your terminal and try sending a transaction: 525 526 ```json 527 curl -s 'localhost:26657/broadcast_tx_commit?tx="tendermint=rocks"' 528 { 529 "jsonrpc": "2.0", 530 "id": "", 531 "result": { 532 "check_tx": { 533 "gasWanted": "1" 534 }, 535 "deliver_tx": {}, 536 "hash": "CDD3C6DFA0A08CAEDF546F9938A2EEC232209C24AA0E4201194E0AFB78A2C2BB", 537 "height": "33" 538 } 539 ``` 540 541 Response should contain the height where this transaction was committed. 542 543 Now let's check if the given key now exists and its value: 544 545 ```json 546 curl -s 'localhost:26657/abci_query?data="tendermint"' 547 { 548 "jsonrpc": "2.0", 549 "id": "", 550 "result": { 551 "response": { 552 "log": "exists", 553 "key": "dGVuZGVybWludA==", 554 "value": "cm9ja3My" 555 } 556 } 557 } 558 ``` 559 560 "dGVuZGVybWludA==" and "cm9ja3M=" are the base64-encoding of the ASCII of 561 "tendermint" and "rocks" accordingly. 562 563 ## Outro 564 565 I hope everything went smoothly and your first, but hopefully not the last, 566 Tendermint Core application is up and running. If not, please [open an issue on 567 Github](https://github.com/vipernet-xyz/tm/issues/new/choose). To dig 568 deeper, read [the docs](https://docs.tendermint.com/v0.34/).