github.com/ari-anchor/sei-tendermint@v0.0.0-20230519144642-dc826b7b56bb/docs/tutorials/go-built-in.md (about) 1 <!--- 2 order: 2 3 ---> 4 5 # Creating a built-in application in Go 6 7 ## Guide assumptions 8 9 This guide is designed for beginners who want to get started with a Tendermint 10 Core application from scratch. It does not assume that you have any prior 11 experience with Tendermint Core. 12 13 Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state 14 transition machine - written in any programming language - and securely 15 replicates it on many machines. 16 17 Although Tendermint Core is written in the Golang programming language, prior 18 knowledge of it is not required for this guide. You can learn it as we go due 19 to it's simplicity. However, you may want to go through [Learn X in Y minutes 20 Where X=Go](https://learnxinyminutes.com/docs/go/) first to familiarize 21 yourself with the syntax. 22 23 By following along with this guide, you'll create a Tendermint Core project 24 called kvstore, a (very) simple distributed BFT key-value store. 25 26 > Note: please use a released version of Tendermint with this guide. The guides will work with the latest version. Please, do not use master. 27 28 ## Built-in app vs external app 29 30 Running your application inside the same process as Tendermint Core will give 31 you the best possible performance. 32 33 For other languages, your application have to communicate with Tendermint Core 34 through a TCP, Unix domain socket or gRPC. 35 36 ## 1.1 Installing Go 37 38 Please refer to [the official guide for installing 39 Go](https://golang.org/doc/install). 40 41 Verify that you have the latest version of Go installed: 42 43 ```bash 44 $ go version 45 go version go1.16.x darwin/amd64 46 ``` 47 48 ## 1.2 Creating a new Go project 49 50 We'll start by creating a new Go project. 51 52 ```bash 53 mkdir kvstore 54 cd kvstore 55 go mod init github.com/<github_username>/<repo_name> 56 ``` 57 58 Inside the example directory create a `main.go` file with the following content: 59 60 > Note: there is no need to clone or fork Tendermint in this tutorial. 61 62 ```go 63 package main 64 65 import ( 66 "fmt" 67 ) 68 69 func main() { 70 fmt.Println("Hello, Tendermint Core") 71 } 72 ``` 73 74 When run, this should print "Hello, Tendermint Core" to the standard output. 75 76 ```bash 77 $ go run main.go 78 Hello, Tendermint Core 79 ``` 80 81 ## 1.3 Writing a Tendermint Core application 82 83 Tendermint Core communicates with the application through the Application 84 BlockChain Interface (ABCI). All message types are defined in the [protobuf 85 file](https://github.com/tendermint/tendermint/blob/master/proto/tendermint/abci/types.proto). 86 This allows Tendermint Core to run applications written in any programming 87 language. 88 89 Create a file called `app.go` with the following content: 90 91 ```go 92 package main 93 94 import ( 95 abcitypes "github.com/tendermint/tendermint/abci/types" 96 ) 97 98 type KVStoreApplication struct {} 99 100 var _ abcitypes.Application = (*KVStoreApplication)(nil) 101 102 func NewKVStoreApplication() *KVStoreApplication { 103 return &KVStoreApplication{} 104 } 105 106 func (KVStoreApplication) Info(req abcitypes.RequestInfo) abcitypes.ResponseInfo { 107 return abcitypes.ResponseInfo{} 108 } 109 110 func (KVStoreApplication) DeliverTx(req abcitypes.RequestDeliverTx) abcitypes.ResponseDeliverTx { 111 return abcitypes.ResponseDeliverTx{Code: 0} 112 } 113 114 func (KVStoreApplication) CheckTx(req abcitypes.RequestCheckTx) abcitypes.ResponseCheckTx { 115 return abcitypes.ResponseCheckTx{Code: 0} 116 } 117 118 func (KVStoreApplication) Commit() abcitypes.ResponseCommit { 119 return abcitypes.ResponseCommit{} 120 } 121 122 func (KVStoreApplication) Query(req abcitypes.RequestQuery) abcitypes.ResponseQuery { 123 return abcitypes.ResponseQuery{Code: 0} 124 } 125 126 func (KVStoreApplication) InitChain(req abcitypes.RequestInitChain) abcitypes.ResponseInitChain { 127 return abcitypes.ResponseInitChain{} 128 } 129 130 func (KVStoreApplication) BeginBlock(req abcitypes.RequestBeginBlock) abcitypes.ResponseBeginBlock { 131 return abcitypes.ResponseBeginBlock{} 132 } 133 134 func (KVStoreApplication) EndBlock(req abcitypes.RequestEndBlock) abcitypes.ResponseEndBlock { 135 return abcitypes.ResponseEndBlock{} 136 } 137 138 func (KVStoreApplication) ListSnapshots(abcitypes.RequestListSnapshots) abcitypes.ResponseListSnapshots { 139 return abcitypes.ResponseListSnapshots{} 140 } 141 142 func (KVStoreApplication) OfferSnapshot(abcitypes.RequestOfferSnapshot) abcitypes.ResponseOfferSnapshot { 143 return abcitypes.ResponseOfferSnapshot{} 144 } 145 146 func (KVStoreApplication) LoadSnapshotChunk(abcitypes.RequestLoadSnapshotChunk) abcitypes.ResponseLoadSnapshotChunk { 147 return abcitypes.ResponseLoadSnapshotChunk{} 148 } 149 150 func (KVStoreApplication) ApplySnapshotChunk(abcitypes.RequestApplySnapshotChunk) abcitypes.ResponseApplySnapshotChunk { 151 return abcitypes.ResponseApplySnapshotChunk{} 152 } 153 ``` 154 155 Now I will go through each method explaining when it's called and adding 156 required business logic. 157 158 ### 1.3.1 CheckTx 159 160 When a new transaction is added to the Tendermint Core, it will ask the 161 application to check it (validate the format, signatures, etc.). 162 163 ```go 164 import "bytes" 165 166 func (app *KVStoreApplication) isValid(tx []byte) (code uint32) { 167 // check format 168 parts := bytes.Split(tx, []byte("=")) 169 if len(parts) != 2 { 170 return 1 171 } 172 173 key, value := parts[0], parts[1] 174 175 // check if the same key=value already exists 176 err := app.db.View(func(txn *badger.Txn) error { 177 item, err := txn.Get(key) 178 if err != nil && err != badger.ErrKeyNotFound { 179 return err 180 } 181 if err == nil { 182 return item.Value(func(val []byte) error { 183 if bytes.Equal(val, value) { 184 code = 2 185 } 186 return nil 187 }) 188 } 189 return nil 190 }) 191 if err != nil { 192 panic(err) 193 } 194 195 return code 196 } 197 198 func (app *KVStoreApplication) CheckTx(req abcitypes.RequestCheckTx) abcitypes.ResponseCheckTx { 199 code := app.isValid(req.Tx) 200 return abcitypes.ResponseCheckTx{Code: code, GasWanted: 1} 201 } 202 ``` 203 204 Don't worry if this does not compile yet. 205 206 If the transaction does not have a form of `{bytes}={bytes}`, we return `1` 207 code. When the same key=value already exist (same key and value), we return `2` 208 code. For others, we return a zero code indicating that they are valid. 209 210 Note that anything with non-zero code will be considered invalid (`-1`, `100`, 211 etc.) by Tendermint Core. 212 213 Valid transactions will eventually be committed given they are not too big and 214 have enough gas. To learn more about gas, check out ["the 215 specification"](https://github.com/tendermint/tendermint/blob/master/spec/abci/apps.md#gas). 216 217 For the underlying key-value store we'll use 218 [badger](https://github.com/dgraph-io/badger), which is an embeddable, 219 persistent and fast key-value (KV) database. 220 221 ```go 222 import "github.com/dgraph-io/badger" 223 224 type KVStoreApplication struct { 225 db *badger.DB 226 currentBatch *badger.Txn 227 } 228 229 func NewKVStoreApplication(db *badger.DB) *KVStoreApplication { 230 return &KVStoreApplication{ 231 db: db, 232 } 233 } 234 ``` 235 236 ### 1.3.2 BeginBlock -> DeliverTx -> EndBlock -> Commit 237 238 When Tendermint Core has decided on the block, it's transfered to the 239 application in 3 parts: `BeginBlock`, one `DeliverTx` per transaction and 240 `EndBlock` in the end. DeliverTx are being transfered asynchronously, but the 241 responses are expected to come in order. 242 243 ```go 244 func (app *KVStoreApplication) BeginBlock(req abcitypes.RequestBeginBlock) abcitypes.ResponseBeginBlock { 245 app.currentBatch = app.db.NewTransaction(true) 246 return abcitypes.ResponseBeginBlock{} 247 } 248 249 ``` 250 251 Here we create a batch, which will store block's transactions. 252 253 ```go 254 func (app *KVStoreApplication) DeliverTx(req abcitypes.RequestDeliverTx) abcitypes.ResponseDeliverTx { 255 code := app.isValid(req.Tx) 256 if code != 0 { 257 return abcitypes.ResponseDeliverTx{Code: code} 258 } 259 260 parts := bytes.Split(req.Tx, []byte("=")) 261 key, value := parts[0], parts[1] 262 263 err := app.currentBatch.Set(key, value) 264 if err != nil { 265 panic(err) 266 } 267 268 return abcitypes.ResponseDeliverTx{Code: 0} 269 } 270 ``` 271 272 If the transaction is badly formatted or the same key=value already exist, we 273 again return the non-zero code. Otherwise, we add it to the current batch. 274 275 In the current design, a block can include incorrect transactions (those who 276 passed CheckTx, but failed DeliverTx or transactions included by the proposer 277 directly). This is done for performance reasons. 278 279 Note we can't commit transactions inside the `DeliverTx` because in such case 280 `Query`, which may be called in parallel, will return inconsistent data (i.e. 281 it will report that some value already exist even when the actual block was not 282 yet committed). 283 284 `Commit` instructs the application to persist the new state. 285 286 ```go 287 func (app *KVStoreApplication) Commit() abcitypes.ResponseCommit { 288 app.currentBatch.Commit() 289 return abcitypes.ResponseCommit{Data: []byte{}} 290 } 291 ``` 292 293 ### 1.3.3 Query 294 295 Now, when the client wants to know whenever a particular key/value exist, it 296 will call Tendermint Core RPC `/abci_query` endpoint, which in turn will call 297 the application's `Query` method. 298 299 Applications are free to provide their own APIs. But by using Tendermint Core 300 as a proxy, clients (including [light client 301 package](https://godoc.org/github.com/tendermint/tendermint/light)) can leverage 302 the unified API across different applications. Plus they won't have to call the 303 otherwise separate Tendermint Core API for additional proofs. 304 305 Note we don't include a proof here. 306 307 ```go 308 func (app *KVStoreApplication) Query(reqQuery abcitypes.RequestQuery) (resQuery abcitypes.ResponseQuery) { 309 resQuery.Key = reqQuery.Data 310 err := app.db.View(func(txn *badger.Txn) error { 311 item, err := txn.Get(reqQuery.Data) 312 if err != nil && err != badger.ErrKeyNotFound { 313 return err 314 } 315 if err == badger.ErrKeyNotFound { 316 resQuery.Log = "does not exist" 317 } else { 318 return item.Value(func(val []byte) error { 319 resQuery.Log = "exists" 320 resQuery.Value = val 321 return nil 322 }) 323 } 324 return nil 325 }) 326 if err != nil { 327 panic(err) 328 } 329 return 330 } 331 ``` 332 333 The complete specification can be found 334 [here](https://github.com/tendermint/tendermint/tree/master/spec/abci/). 335 336 ## 1.4 Starting an application and a Tendermint Core instance in the same process 337 338 Put the following code into the "main.go" file: 339 340 ```go 341 package main 342 343 import ( 344 "flag" 345 "fmt" 346 "os" 347 "os/signal" 348 "path/filepath" 349 "syscall" 350 351 "github.com/dgraph-io/badger" 352 "github.com/spf13/viper" 353 354 abci "github.com/tendermint/tendermint/abci/types" 355 cfg "github.com/tendermint/tendermint/config" 356 tmflags "github.com/tendermint/tendermint/libs/cli/flags" 357 "github.com/tendermint/tendermint/libs/log" 358 nm "github.com/tendermint/tendermint/node" 359 "github.com/tendermint/tendermint/internal/p2p" 360 "github.com/tendermint/tendermint/privval" 361 "github.com/tendermint/tendermint/proxy" 362 ) 363 364 var configFile string 365 366 func init() { 367 flag.StringVar(&configFile, "config", "$HOME/.tendermint/config/config.toml", "Path to config.toml") 368 } 369 370 func main() { 371 db, err := badger.Open(badger.DefaultOptions("/tmp/badger")) 372 if err != nil { 373 fmt.Fprintf(os.Stderr, "failed to open badger db: %v", err) 374 os.Exit(1) 375 } 376 defer db.Close() 377 app := NewKVStoreApplication(db) 378 379 flag.Parse() 380 381 node, err := newTendermint(app, configFile) 382 if err != nil { 383 fmt.Fprintf(os.Stderr, "%v", err) 384 os.Exit(2) 385 } 386 387 node.Start() 388 defer func() { 389 node.Stop() 390 node.Wait() 391 }() 392 393 c := make(chan os.Signal, 1) 394 signal.Notify(c, os.Interrupt, syscall.SIGTERM) 395 <-c 396 } 397 398 func newTendermint(app abci.Application, configFile string) (*nm.Node, error) { 399 // read config 400 config := cfg.DefaultValidatorConfig() 401 config.RootDir = filepath.Dir(filepath.Dir(configFile)) 402 viper.SetConfigFile(configFile) 403 if err := viper.ReadInConfig(); err != nil { 404 return nil, fmt.Errorf("viper failed to read config file: %w", err) 405 } 406 if err := viper.Unmarshal(config); err != nil { 407 return nil, fmt.Errorf("viper failed to unmarshal config: %w", err) 408 } 409 if err := config.ValidateBasic(); err != nil { 410 return nil, fmt.Errorf("config is invalid: %w", err) 411 } 412 413 // create logger 414 logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout)) 415 var err error 416 logger, err = tmflags.ParseLogLevel(config.LogLevel, logger, cfg.DefaultLogLevel) 417 if err != nil { 418 return nil, fmt.Errorf("failed to parse log level: %w", err) 419 } 420 421 // read private validator 422 pv := privval.LoadFilePV( 423 config.PrivValidatorKeyFile(), 424 config.PrivValidatorStateFile(), 425 ) 426 427 // read node key 428 nodeKey, err := p2p.LoadNodeKey(config.NodeKeyFile()) 429 if err != nil { 430 return nil, fmt.Errorf("failed to load node's key: %w", err) 431 } 432 433 // create node 434 node, err := nm.NewNode( 435 config, 436 pv, 437 nodeKey, 438 abcicli.NewLocalClientCreator(app), 439 nm.DefaultGenesisDocProviderFunc(config), 440 nm.DefaultDBProvider, 441 nm.DefaultMetricsProvider(config.Instrumentation), 442 logger) 443 if err != nil { 444 return nil, fmt.Errorf("failed to create new Tendermint node: %w", err) 445 } 446 447 return node, nil 448 } 449 ``` 450 451 This is a huge blob of code, so let's break it down into pieces. 452 453 First, we initialize the Badger database and create an app instance: 454 455 ```go 456 db, err := badger.Open(badger.DefaultOptions("/tmp/badger")) 457 if err != nil { 458 fmt.Fprintf(os.Stderr, "failed to open badger db: %v", err) 459 os.Exit(1) 460 } 461 defer db.Close() 462 app := NewKVStoreApplication(db) 463 ``` 464 465 For **Windows** users, restarting this app will make badger throw an error as it requires value log to be truncated. For more information on this, visit [here](https://github.com/dgraph-io/badger/issues/744). 466 This can be avoided by setting the truncate option to true, like this: 467 468 ```go 469 db, err := badger.Open(badger.DefaultOptions("/tmp/badger").WithTruncate(true)) 470 ``` 471 472 Then we use it to create a Tendermint Core `Node` instance: 473 474 ```go 475 flag.Parse() 476 477 node, err := newTendermint(app, configFile) 478 if err != nil { 479 fmt.Fprintf(os.Stderr, "%v", err) 480 os.Exit(2) 481 } 482 483 ... 484 485 // create node 486 node, err := nm.NewNode( 487 config, 488 pv, 489 nodeKey, 490 abcicli.NewLocalClientCreator(app), 491 nm.DefaultGenesisDocProviderFunc(config), 492 nm.DefaultDBProvider, 493 nm.DefaultMetricsProvider(config.Instrumentation), 494 logger) 495 if err != nil { 496 return nil, fmt.Errorf("failed to create new Tendermint node: %w", err) 497 } 498 ``` 499 500 `NewNode` requires a few things including a configuration file, a private 501 validator, a node key and a few others in order to construct the full node. 502 503 Note we use `abcicli.NewLocalClientCreator` here to create a local client instead 504 of one communicating through a socket or gRPC. 505 506 [viper](https://github.com/spf13/viper) is being used for reading the config, 507 which we will generate later using the `tendermint init` command. 508 509 ```go 510 config := cfg.DefaultValidatorConfig() 511 config.RootDir = filepath.Dir(filepath.Dir(configFile)) 512 viper.SetConfigFile(configFile) 513 if err := viper.ReadInConfig(); err != nil { 514 return nil, fmt.Errorf("viper failed to read config file: %w", err) 515 } 516 if err := viper.Unmarshal(config); err != nil { 517 return nil, fmt.Errorf("viper failed to unmarshal config: %w", err) 518 } 519 if err := config.ValidateBasic(); err != nil { 520 return nil, fmt.Errorf("config is invalid: %w", err) 521 } 522 ``` 523 524 We use `FilePV`, which is a private validator (i.e. thing which signs consensus 525 messages). Normally, you would use `SignerRemote` to connect to an external 526 [HSM](https://kb.certus.one/hsm.html). 527 528 ```go 529 pv := privval.LoadFilePV( 530 config.PrivValidatorKeyFile(), 531 config.PrivValidatorStateFile(), 532 ) 533 534 ``` 535 536 `nodeKey` is needed to identify the node in a p2p network. 537 538 ```go 539 nodeKey, err := p2p.LoadNodeKey(config.NodeKeyFile()) 540 if err != nil { 541 return nil, fmt.Errorf("failed to load node's key: %w", err) 542 } 543 ``` 544 545 As for the logger, we use the build-in library, which provides a nice 546 abstraction over [go-kit's 547 logger](https://github.com/go-kit/kit/tree/master/log). 548 549 ```go 550 logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout)) 551 var err error 552 logger, err = tmflags.ParseLogLevel(config.LogLevel, logger, cfg.DefaultLogLevel()) 553 if err != nil { 554 return nil, fmt.Errorf("failed to parse log level: %w", err) 555 } 556 ``` 557 558 Finally, we start the node and add some signal handling to gracefully stop it 559 upon receiving SIGTERM or Ctrl-C. 560 561 ```go 562 node.Start() 563 defer func() { 564 node.Stop() 565 node.Wait() 566 }() 567 568 c := make(chan os.Signal, 1) 569 signal.Notify(c, os.Interrupt, syscall.SIGTERM) 570 <-c 571 ``` 572 573 ## 1.5 Getting Up and Running 574 575 We are going to use [Go modules](https://github.com/golang/go/wiki/Modules) for 576 dependency management. 577 578 ```bash 579 export GO111MODULE=on 580 go mod init github.com/me/example 581 ``` 582 583 This should create a `go.mod` file. The current tutorial only works with 584 the master branch of Tendermint. so let's make sure we're using the latest version: 585 586 ```sh 587 go get github.com/tendermint/tendermint@master 588 ``` 589 590 This will populate the `go.mod` with a release number followed by a hash for Tendermint. 591 592 ```go 593 module github.com/me/example 594 595 go 1.15 596 597 require ( 598 github.com/dgraph-io/badger v1.6.2 599 github.com/tendermint/tendermint <vX> 600 ) 601 ``` 602 603 Now we can build the binary: 604 605 ```bash 606 go build 607 ``` 608 609 To create a default configuration, nodeKey and private validator files, let's 610 execute `tendermint init validator`. But before we do that, we will need to install 611 Tendermint Core. Please refer to [the official 612 guide](https://docs.tendermint.com/master/introduction/install.html). If you're 613 installing from source, don't forget to checkout the latest release (`git 614 checkout vX.Y.Z`). Don't forget to check that the application uses the same 615 major version. 616 617 ```bash 618 $ rm -rf /tmp/example 619 $ TMHOME="/tmp/example" tendermint init validator 620 621 I[2019-07-16|18:40:36.480] Generated private validator module=main keyFile=/tmp/example/config/priv_validator_key.json stateFile=/tmp/example2/data/priv_validator_state.json 622 I[2019-07-16|18:40:36.481] Generated node key module=main path=/tmp/example/config/node_key.json 623 I[2019-07-16|18:40:36.482] Generated genesis file module=main path=/tmp/example/config/genesis.json 624 I[2019-07-16|18:40:36.483] Generated config module=main mode=validator 625 ``` 626 627 We are ready to start our application: 628 629 ```bash 630 $ ./example -config "/tmp/example/config/config.toml" 631 632 badger 2019/07/16 18:42:25 INFO: All 0 tables opened in 0s 633 badger 2019/07/16 18:42:25 INFO: Replaying file id: 0 at offset: 0 634 badger 2019/07/16 18:42:25 INFO: Replay took: 695.227s 635 E[2019-07-16|18:42:25.818] Couldn't connect to any seeds module=p2p 636 I[2019-07-16|18:42:26.853] Executed block module=state height=1 validTxs=0 invalidTxs=0 637 I[2019-07-16|18:42:26.865] Committed state module=state height=1 txs=0 appHash= 638 ``` 639 640 Now open another tab in your terminal and try sending a transaction: 641 642 ```bash 643 $ curl -s 'localhost:26657/broadcast_tx_commit?tx="tendermint=rocks"' 644 { 645 "check_tx": { 646 "gasWanted": "1", 647 ... 648 }, 649 "deliver_tx": { ... }, 650 "hash": "1B3C5A1093DB952C331B1749A21DCCBB0F6C7F4E0055CD04D16346472FC60EC6", 651 "height": "128" 652 } 653 ``` 654 655 Response should contain the height where this transaction was committed. 656 657 Now let's check if the given key now exists and its value: 658 659 ```json 660 $ curl -s 'localhost:26657/abci_query?data="tendermint"' 661 { 662 "response": { 663 "code": 0, 664 "log": "exists", 665 "info": "", 666 "index": "0", 667 "key": "dGVuZGVybWludA==", 668 "value": "cm9ja3M=", 669 "proofOps": null, 670 "height": "6", 671 "codespace": "" 672 } 673 } 674 ``` 675 676 "dGVuZGVybWludA==" and "cm9ja3M=" are the base64-encoding of the ASCII of 677 "tendermint" and "rocks" accordingly. 678 679 ## Outro 680 681 I hope everything went smoothly and your first, but hopefully not the last, 682 Tendermint Core application is up and running. If not, please [open an issue on 683 Github](https://github.com/tendermint/tendermint/issues/new/choose). To dig 684 deeper, read [the docs](https://docs.tendermint.com/master/).