github.com/fiagdao/tendermint@v0.32.11-0.20220824195748-2087fcc480c1/docs/guides/kotlin.md (about) 1 <!--- 2 order: 4 3 ---> 4 5 # Creating an application in Kotlin 6 7 ## Guide Assumptions 8 9 This guide is designed for beginners who want to get started with a Tendermint 10 Core application from scratch. It does not assume that you have any prior 11 experience with Tendermint Core. 12 13 Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state 14 transition machine (your application) - written in any programming language - and securely 15 replicates it on many machines. 16 17 By following along with this guide, you'll create a Tendermint Core project 18 called kvstore, a (very) simple distributed BFT key-value store. The application (which should 19 implementing the blockchain interface (ABCI)) will be written in Kotlin. 20 21 This guide assumes that you are not new to JVM world. If you are new please see [JVM Minimal Survival Guide](https://hadihariri.com/2013/12/29/jvm-minimal-survival-guide-for-the-dotnet-developer/#java-the-language-java-the-ecosystem-java-the-jvm) and [Gradle Docs](https://docs.gradle.org/current/userguide/userguide.html). 22 23 ## Built-in app vs external app 24 25 If you use Golang, you can run your app and Tendermint Core in the same process to get maximum performance. 26 [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is written this way. 27 Please refer to [Writing a built-in Tendermint Core application in Go](./go-built-in.md) guide for details. 28 29 If you choose another language, like we did in this guide, you have to write a separate app, 30 which will communicate with Tendermint Core via a socket (UNIX or TCP) or gRPC. 31 This guide will show you how to build external application using RPC server. 32 33 Having a separate application might give you better security guarantees as two 34 processes would be communicating via established binary protocol. Tendermint 35 Core will not have access to application's state. 36 37 ## 1.1 Installing Java and Gradle 38 39 Please refer to [the Oracle's guide for installing JDK](https://www.oracle.com/technetwork/java/javase/downloads/index.html). 40 41 Verify that you have installed Java successfully: 42 43 ```sh 44 $ java -version 45 java version "1.8.0_162" 46 Java(TM) SE Runtime Environment (build 1.8.0_162-b12) 47 Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode) 48 ``` 49 50 You can choose any version of Java higher or equal to 8. 51 In my case it is Java SE Development Kit 8. 52 53 Make sure you have `$JAVA_HOME` environment variable set: 54 55 ```sh 56 $ echo $JAVA_HOME 57 /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home 58 ``` 59 60 For Gradle installation, please refer to [their official guide](https://gradle.org/install/). 61 62 ## 1.2 Creating a new Kotlin project 63 64 We'll start by creating a new Gradle project. 65 66 ```sh 67 $ export KVSTORE_HOME=~/kvstore 68 $ mkdir $KVSTORE_HOME 69 $ cd $KVSTORE_HOME 70 ``` 71 72 Inside the example directory run: 73 74 ```sh 75 gradle init --dsl groovy --package io.example --project-name example --type kotlin-application 76 ``` 77 78 This will create a new project for you. The tree of files should look like: 79 80 ```sh 81 $ tree 82 . 83 |-- build.gradle 84 |-- gradle 85 | `-- wrapper 86 | |-- gradle-wrapper.jar 87 | `-- gradle-wrapper.properties 88 |-- gradlew 89 |-- gradlew.bat 90 |-- settings.gradle 91 `-- src 92 |-- main 93 | |-- kotlin 94 | | `-- io 95 | | `-- example 96 | | `-- App.kt 97 | `-- resources 98 `-- test 99 |-- kotlin 100 | `-- io 101 | `-- example 102 | `-- AppTest.kt 103 `-- resources 104 ``` 105 106 When run, this should print "Hello world." to the standard output. 107 108 ```sh 109 $ ./gradlew run 110 > Task :run 111 Hello world. 112 ``` 113 114 ## 1.3 Writing a Tendermint Core application 115 116 Tendermint Core communicates with the application through the Application 117 BlockChain Interface (ABCI). All message types are defined in the [protobuf 118 file](https://github.com/tendermint/tendermint/blob/master/abci/types/types.proto). 119 This allows Tendermint Core to run applications written in any programming 120 language. 121 122 ### 1.3.1 Compile .proto files 123 124 Add the following piece to the top of the `build.gradle`: 125 126 ```groovy 127 buildscript { 128 repositories { 129 mavenCentral() 130 } 131 dependencies { 132 classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.8' 133 } 134 } 135 ``` 136 137 Enable the protobuf plugin in the `plugins` section of the `build.gradle`: 138 139 ```groovy 140 plugins { 141 id 'com.google.protobuf' version '0.8.8' 142 } 143 ``` 144 145 Add the following code to `build.gradle`: 146 147 ```groovy 148 protobuf { 149 protoc { 150 artifact = "com.google.protobuf:protoc:3.7.1" 151 } 152 plugins { 153 grpc { 154 artifact = 'io.grpc:protoc-gen-grpc-java:1.22.1' 155 } 156 } 157 generateProtoTasks { 158 all()*.plugins { 159 grpc {} 160 } 161 } 162 } 163 ``` 164 165 Now we should be ready to compile the `*.proto` files. 166 167 Copy the necessary `.proto` files to your project: 168 169 ```sh 170 mkdir -p \ 171 $KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/abci/types \ 172 $KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/crypto/merkle \ 173 $KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/libs/kv \ 174 $KVSTORE_HOME/src/main/proto/github.com/gogo/protobuf/gogoproto 175 176 cp $GOPATH/src/github.com/tendermint/tendermint/abci/types/types.proto \ 177 $KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/abci/types/types.proto 178 cp $GOPATH/src/github.com/tendermint/tendermint/crypto/merkle/merkle.proto \ 179 $KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/crypto/merkle/merkle.proto 180 cp $GOPATH/src/github.com/tendermint/tendermint/libs/kv/types.proto \ 181 $KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/libs/kv/types.proto 182 cp $GOPATH/src/github.com/gogo/protobuf/gogoproto/gogo.proto \ 183 $KVSTORE_HOME/src/main/proto/github.com/gogo/protobuf/gogoproto/gogo.proto 184 ``` 185 186 Add these dependencies to `build.gradle`: 187 188 ```groovy 189 dependencies { 190 implementation 'io.grpc:grpc-protobuf:1.22.1' 191 implementation 'io.grpc:grpc-netty-shaded:1.22.1' 192 implementation 'io.grpc:grpc-stub:1.22.1' 193 } 194 ``` 195 196 To generate all protobuf-type classes run: 197 198 ```sh 199 ./gradlew generateProto 200 ``` 201 202 To verify that everything went smoothly, you can inspect the `build/generated/` directory: 203 204 ```sh 205 $ tree build/generated/ 206 build/generated/ 207 `-- source 208 `-- proto 209 `-- main 210 |-- grpc 211 | `-- types 212 | `-- ABCIApplicationGrpc.java 213 `-- java 214 |-- com 215 | `-- google 216 | `-- protobuf 217 | `-- GoGoProtos.java 218 |-- common 219 | `-- Types.java 220 |-- merkle 221 | `-- Merkle.java 222 `-- types 223 `-- Types.java 224 ``` 225 226 ### 1.3.2 Implementing ABCI 227 228 The resulting `$KVSTORE_HOME/build/generated/source/proto/main/grpc/types/ABCIApplicationGrpc.java` file 229 contains the abstract class `ABCIApplicationImplBase`, which is an interface we'll need to implement. 230 231 Create `$KVSTORE_HOME/src/main/kotlin/io/example/KVStoreApp.kt` file with the following content: 232 233 ```kotlin 234 package io.example 235 236 import io.grpc.stub.StreamObserver 237 import types.ABCIApplicationGrpc 238 import types.Types.* 239 240 class KVStoreApp : ABCIApplicationGrpc.ABCIApplicationImplBase() { 241 242 // methods implementation 243 244 } 245 ``` 246 247 Now I will go through each method of `ABCIApplicationImplBase` explaining when it's called and adding 248 required business logic. 249 250 ### 1.3.3 CheckTx 251 252 When a new transaction is added to the Tendermint Core, it will ask the 253 application to check it (validate the format, signatures, etc.). 254 255 ```kotlin 256 override fun checkTx(req: RequestCheckTx, responseObserver: StreamObserver<ResponseCheckTx>) { 257 val code = req.tx.validate() 258 val resp = ResponseCheckTx.newBuilder() 259 .setCode(code) 260 .setGasWanted(1) 261 .build() 262 responseObserver.onNext(resp) 263 responseObserver.onCompleted() 264 } 265 266 private fun ByteString.validate(): Int { 267 val parts = this.split('=') 268 if (parts.size != 2) { 269 return 1 270 } 271 val key = parts[0] 272 val value = parts[1] 273 274 // check if the same key=value already exists 275 val stored = getPersistedValue(key) 276 if (stored != null && stored.contentEquals(value)) { 277 return 2 278 } 279 280 return 0 281 } 282 283 private fun ByteString.split(separator: Char): List<ByteArray> { 284 val arr = this.toByteArray() 285 val i = (0 until this.size()).firstOrNull { arr[it] == separator.toByte() } 286 ?: return emptyList() 287 return listOf( 288 this.substring(0, i).toByteArray(), 289 this.substring(i + 1).toByteArray() 290 ) 291 } 292 ``` 293 294 Don't worry if this does not compile yet. 295 296 If the transaction does not have a form of `{bytes}={bytes}`, we return `1` 297 code. When the same key=value already exist (same key and value), we return `2` 298 code. For others, we return a zero code indicating that they are valid. 299 300 Note that anything with non-zero code will be considered invalid (`-1`, `100`, 301 etc.) by Tendermint Core. 302 303 Valid transactions will eventually be committed given they are not too big and 304 have enough gas. To learn more about gas, check out ["the 305 specification"](https://docs.tendermint.com/master/spec/abci/apps.html#gas). 306 307 For the underlying key-value store we'll use 308 [JetBrains Xodus](https://github.com/JetBrains/xodus), which is a transactional schema-less embedded high-performance database written in Java. 309 310 `build.gradle`: 311 312 ```groovy 313 dependencies { 314 implementation 'org.jetbrains.xodus:xodus-environment:1.3.91' 315 } 316 ``` 317 318 ```kotlin 319 ... 320 import jetbrains.exodus.ArrayByteIterable 321 import jetbrains.exodus.env.Environment 322 import jetbrains.exodus.env.Store 323 import jetbrains.exodus.env.StoreConfig 324 import jetbrains.exodus.env.Transaction 325 326 class KVStoreApp( 327 private val env: Environment 328 ) : ABCIApplicationGrpc.ABCIApplicationImplBase() { 329 330 private var txn: Transaction? = null 331 private var store: Store? = null 332 333 ... 334 335 private fun getPersistedValue(k: ByteArray): ByteArray? { 336 return env.computeInReadonlyTransaction { txn -> 337 val store = env.openStore("store", StoreConfig.WITHOUT_DUPLICATES, txn) 338 store.get(txn, ArrayByteIterable(k))?.bytesUnsafe 339 } 340 } 341 } 342 ``` 343 344 ### 1.3.4 BeginBlock -> DeliverTx -> EndBlock -> Commit 345 346 When Tendermint Core has decided on the block, it's transferred to the 347 application in 3 parts: `BeginBlock`, one `DeliverTx` per transaction and 348 `EndBlock` in the end. `DeliverTx` are being transferred asynchronously, but the 349 responses are expected to come in order. 350 351 ```kotlin 352 override fun beginBlock(req: RequestBeginBlock, responseObserver: StreamObserver<ResponseBeginBlock>) { 353 txn = env.beginTransaction() 354 store = env.openStore("store", StoreConfig.WITHOUT_DUPLICATES, txn!!) 355 val resp = ResponseBeginBlock.newBuilder().build() 356 responseObserver.onNext(resp) 357 responseObserver.onCompleted() 358 } 359 ``` 360 361 Here we begin a new transaction, which will accumulate the block's transactions and open the corresponding store. 362 363 ```kotlin 364 override fun deliverTx(req: RequestDeliverTx, responseObserver: StreamObserver<ResponseDeliverTx>) { 365 val code = req.tx.validate() 366 if (code == 0) { 367 val parts = req.tx.split('=') 368 val key = ArrayByteIterable(parts[0]) 369 val value = ArrayByteIterable(parts[1]) 370 store!!.put(txn!!, key, value) 371 } 372 val resp = ResponseDeliverTx.newBuilder() 373 .setCode(code) 374 .build() 375 responseObserver.onNext(resp) 376 responseObserver.onCompleted() 377 } 378 ``` 379 380 If the transaction is badly formatted or the same key=value already exist, we 381 again return the non-zero code. Otherwise, we add it to the store. 382 383 In the current design, a block can include incorrect transactions (those who 384 passed `CheckTx`, but failed `DeliverTx` or transactions included by the proposer 385 directly). This is done for performance reasons. 386 387 Note we can't commit transactions inside the `DeliverTx` because in such case 388 `Query`, which may be called in parallel, will return inconsistent data (i.e. 389 it will report that some value already exist even when the actual block was not 390 yet committed). 391 392 `Commit` instructs the application to persist the new state. 393 394 ```kotlin 395 override fun commit(req: RequestCommit, responseObserver: StreamObserver<ResponseCommit>) { 396 txn!!.commit() 397 val resp = ResponseCommit.newBuilder() 398 .setData(ByteString.copyFrom(ByteArray(8))) 399 .build() 400 responseObserver.onNext(resp) 401 responseObserver.onCompleted() 402 } 403 ``` 404 405 ### 1.3.5 Query 406 407 Now, when the client wants to know whenever a particular key/value exist, it 408 will call Tendermint Core RPC `/abci_query` endpoint, which in turn will call 409 the application's `Query` method. 410 411 Applications are free to provide their own APIs. But by using Tendermint Core 412 as a proxy, clients (including [light client 413 package](https://godoc.org/github.com/tendermint/tendermint/lite)) can leverage 414 the unified API across different applications. Plus they won't have to call the 415 otherwise separate Tendermint Core API for additional proofs. 416 417 Note we don't include a proof here. 418 419 ```kotlin 420 override fun query(req: RequestQuery, responseObserver: StreamObserver<ResponseQuery>) { 421 val k = req.data.toByteArray() 422 val v = getPersistedValue(k) 423 val builder = ResponseQuery.newBuilder() 424 if (v == null) { 425 builder.log = "does not exist" 426 } else { 427 builder.log = "exists" 428 builder.key = ByteString.copyFrom(k) 429 builder.value = ByteString.copyFrom(v) 430 } 431 responseObserver.onNext(builder.build()) 432 responseObserver.onCompleted() 433 } 434 ``` 435 436 The complete specification can be found 437 [here](https://docs.tendermint.com/master/spec/abci/). 438 439 ## 1.4 Starting an application and a Tendermint Core instances 440 441 Put the following code into the `$KVSTORE_HOME/src/main/kotlin/io/example/App.kt` file: 442 443 ```kotlin 444 package io.example 445 446 import jetbrains.exodus.env.Environments 447 448 fun main() { 449 Environments.newInstance("tmp/storage").use { env -> 450 val app = KVStoreApp(env) 451 val server = GrpcServer(app, 26658) 452 server.start() 453 server.blockUntilShutdown() 454 } 455 } 456 ``` 457 458 It is the entry point of the application. 459 Here we create a special object `Environment`, which knows where to store the application state. 460 Then we create and start the gRPC server to handle Tendermint Core requests. 461 462 Create `$KVSTORE_HOME/src/main/kotlin/io/example/GrpcServer.kt` file with the following content: 463 464 ```kotlin 465 package io.example 466 467 import io.grpc.BindableService 468 import io.grpc.ServerBuilder 469 470 class GrpcServer( 471 private val service: BindableService, 472 private val port: Int 473 ) { 474 private val server = ServerBuilder 475 .forPort(port) 476 .addService(service) 477 .build() 478 479 fun start() { 480 server.start() 481 println("gRPC server started, listening on $port") 482 Runtime.getRuntime().addShutdownHook(object : Thread() { 483 override fun run() { 484 println("shutting down gRPC server since JVM is shutting down") 485 this@GrpcServer.stop() 486 println("server shut down") 487 } 488 }) 489 } 490 491 fun stop() { 492 server.shutdown() 493 } 494 495 /** 496 * Await termination on the main thread since the grpc library uses daemon threads. 497 */ 498 fun blockUntilShutdown() { 499 server.awaitTermination() 500 } 501 502 } 503 ``` 504 505 ## 1.5 Getting Up and Running 506 507 To create a default configuration, nodeKey and private validator files, let's 508 execute `tendermint init`. But before we do that, we will need to install 509 Tendermint Core. 510 511 ```sh 512 $ rm -rf /tmp/example 513 $ cd $GOPATH/src/github.com/tendermint/tendermint 514 $ make install 515 $ TMHOME="/tmp/example" tendermint init 516 517 I[2019-07-16|18:20:36.480] Generated private validator module=main keyFile=/tmp/example/config/priv_validator_key.json stateFile=/tmp/example2/data/priv_validator_state.json 518 I[2019-07-16|18:20:36.481] Generated node key module=main path=/tmp/example/config/node_key.json 519 I[2019-07-16|18:20:36.482] Generated genesis file module=main path=/tmp/example/config/genesis.json 520 ``` 521 522 Feel free to explore the generated files, which can be found at 523 `/tmp/example/config` directory. Documentation on the config can be found 524 [here](https://docs.tendermint.com/master/tendermint-core/configuration.html). 525 526 We are ready to start our application: 527 528 ```sh 529 ./gradlew run 530 531 gRPC server started, listening on 26658 532 ``` 533 534 Then we need to start Tendermint Core and point it to our application. Staying 535 within the application directory execute: 536 537 ```sh 538 $ TMHOME="/tmp/example" tendermint node --abci grpc --proxy_app tcp://127.0.0.1:26658 539 540 I[2019-07-28|15:44:53.632] Version info module=main software=0.32.1 block=10 p2p=7 541 I[2019-07-28|15:44:53.677] Starting Node module=main impl=Node 542 I[2019-07-28|15:44:53.681] Started node module=main nodeInfo="{ProtocolVersion:{P2P:7 Block:10 App:0} ID_:7639e2841ccd47d5ae0f5aad3011b14049d3f452 ListenAddr:tcp://0.0.0.0:26656 Network:test-chain-Nhl3zk Version:0.32.1 Channels:4020212223303800 Moniker:Ivans-MacBook-Pro.local Other:{TxIndex:on RPCAddress:tcp://127.0.0.1:26657}}" 543 I[2019-07-28|15:44:54.801] Executed block module=state height=8 validTxs=0 invalidTxs=0 544 I[2019-07-28|15:44:54.814] Committed state module=state height=8 txs=0 appHash=0000000000000000 545 ``` 546 547 Now open another tab in your terminal and try sending a transaction: 548 549 ```sh 550 $ curl -s 'localhost:26657/broadcast_tx_commit?tx="tendermint=rocks"' 551 { 552 "jsonrpc": "2.0", 553 "id": "", 554 "result": { 555 "check_tx": { 556 "gasWanted": "1" 557 }, 558 "deliver_tx": {}, 559 "hash": "CDD3C6DFA0A08CAEDF546F9938A2EEC232209C24AA0E4201194E0AFB78A2C2BB", 560 "height": "33" 561 } 562 ``` 563 564 Response should contain the height where this transaction was committed. 565 566 Now let's check if the given key now exists and its value: 567 568 ```sh 569 $ curl -s 'localhost:26657/abci_query?data="tendermint"' 570 { 571 "jsonrpc": "2.0", 572 "id": "", 573 "result": { 574 "response": { 575 "log": "exists", 576 "key": "dGVuZGVybWludA==", 577 "value": "cm9ja3My" 578 } 579 } 580 } 581 ``` 582 583 `dGVuZGVybWludA==` and `cm9ja3M=` are the base64-encoding of the ASCII of `tendermint` and `rocks` accordingly. 584 585 ## Outro 586 587 I hope everything went smoothly and your first, but hopefully not the last, 588 Tendermint Core application is up and running. If not, please [open an issue on 589 Github](https://github.com/tendermint/tendermint/issues/new/choose). To dig 590 deeper, read [the docs](https://docs.tendermint.com/master/). 591 592 The full source code of this example project can be found [here](https://github.com/climber73/tendermint-abci-grpc-kotlin).