github.com/osdi23p228/fabric@v0.0.0-20221218062954-77808885f5db/docs/source/couchdb_tutorial.rst (about) 1 2 Using CouchDB 3 ============= 4 5 This tutorial will describe the steps required to use CouchDB as the state 6 database with Hyperledger Fabric. By now, you should be familiar with Fabric 7 concepts and have explored some of the samples and tutorials. 8 9 .. note:: These instructions use the new Fabric chaincode lifecycle introduced 10 in the Fabric v2.0 release. If you would like to use the previous 11 lifecycle model to use indexes with chaincode, visit the v1.4 12 version of the `Using CouchDB <https://hyperledger-fabric.readthedocs.io/en/release-1.4/couchdb_tutorial.html>`__. 13 14 The tutorial will take you through the following steps: 15 16 #. :ref:`cdb-enable-couch` 17 #. :ref:`cdb-create-index` 18 #. :ref:`cdb-add-index` 19 #. :ref:`cdb-install-deploy` 20 #. :ref:`cdb-query` 21 #. :ref:`cdb-best` 22 #. :ref:`cdb-pagination` 23 #. :ref:`cdb-update-index` 24 #. :ref:`cdb-delete-index` 25 26 For a deeper dive into CouchDB refer to :doc:`couchdb_as_state_database` 27 and for more information on the Fabric ledger refer to the `Ledger <ledger/ledger.html>`_ 28 topic. Follow the tutorial below for details on how to leverage CouchDB in your 29 blockchain network. 30 31 Throughout this tutorial, we will use the `Asset transfer ledger queries sample <https://github.com/hyperledger/fabric-samples/blob/{BRANCH}/asset-transfer-ledger-queries/chaincode-go>`__ 32 as our use case to demonstrate how to use CouchDB with Fabric, including the 33 execution of JSON queries against the state database. You should have completed the task 34 :doc:`install`. 35 36 Why CouchDB? 37 ~~~~~~~~~~~~ 38 39 Fabric supports two types of peer state databases. LevelDB is the default state 40 database embedded in the peer node. LevelDB stores chaincode data as simple 41 key-value pairs and only supports key, key range, and composite key queries. 42 CouchDB is an optional, alternate state database that allows you to model data 43 on the ledger as JSON and issue rich queries against data values rather than 44 the keys. The CouchDB support also allows you to deploy indexes with your chaincode to make 45 queries more efficient and enable you to query large datasets. 46 47 In order to leverage the benefits of CouchDB, namely content-based JSON 48 queries, your data must be modeled in JSON format. You must decide whether to use 49 LevelDB or CouchDB before setting up your network. Switching a peer from using 50 LevelDB to CouchDB is not supported due to data compatibility issues. All peers 51 on the network must use the same database type. If you have a mix of JSON and 52 binary data values, you can still use CouchDB, however the binary values can 53 only be queried based on key, key range, and composite key queries. 54 55 .. _cdb-enable-couch: 56 57 Enable CouchDB in Hyperledger Fabric 58 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 59 60 CouchDB runs as a separate database process alongside the peer. There 61 are additional considerations in terms of setup, management, and operations. 62 A Docker image of `CouchDB <https://hub.docker.com/_/couchdb/>`__ 63 is available and we recommend that it be run on the same server as the 64 peer. You will need to setup one CouchDB container per peer 65 and update each peer container by changing the configuration found in 66 ``core.yaml`` to point to the CouchDB container. The ``core.yaml`` 67 file must be located in the directory specified by the environment variable 68 FABRIC_CFG_PATH: 69 70 * For Docker deployments, ``core.yaml`` is pre-configured and located in the peer 71 container ``FABRIC_CFG_PATH`` folder. However, when using Docker environments, 72 you can pass environment variables to override the core.yaml properties, for 73 example ``CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS`` to set the CouchDB address. 74 75 * For native binary deployments, ``core.yaml`` is included with the release artifact 76 distribution. 77 78 Edit the ``stateDatabase`` section of ``core.yaml``. Specify ``CouchDB`` as the 79 ``stateDatabase`` and fill in the associated ``couchDBConfig`` properties. For 80 more information, see `CouchDB configuration <couchdb_as_state_database.html#couchdb-configuration>`__. 81 82 .. _cdb-create-index: 83 84 Create an index 85 ~~~~~~~~~~~~~~~ 86 87 Why are indexes important? 88 89 Indexes allow a database to be queried without having to examine every row 90 with every query, making them run faster and more efficiently. Normally, 91 indexes are built for frequently occurring query criteria allowing the data to 92 be queried more efficiently. To leverage the major benefit of CouchDB -- the 93 ability to perform rich queries against JSON data -- indexes are not required, 94 but they are strongly recommended for performance. Also, if sorting is required 95 in a query, CouchDB requires an index that includes the sorted fields. 96 97 .. note:: 98 99 JSON queries that do not have an index may work but will throw a warning 100 in the peer log that the index was not found. However, if a rich query 101 includes a sort specification, then an index on that field is required; 102 otherwise, the query will fail and an error will be thrown. 103 104 To demonstrate building an index, we will use the data from the `Asset transfer ledger queries 105 sample <https://github.com/hyperledger/fabric-samples/blob/{BRANCH}/asset-transfer-ledger-queries/chaincode-go/asset_transfer_ledger_chaincode.go>`__. 106 In this example, the Asset data structure is defined as: 107 108 .. code:: javascript 109 110 type Asset struct { 111 DocType string `json:"docType"` //docType is used to distinguish the various types of objects in state database 112 ID string `json:"ID"` //the field tags are needed to keep case from bouncing around 113 Color string `json:"color"` 114 Size int `json:"size"` 115 Owner string `json:"owner"` 116 AppraisedValue int `json:"appraisedValue"` 117 } 118 119 120 In this structure, the attributes (``docType``, ``ID``, ``color``, ``size``, 121 ``owner``, ``appraisedValue``) define the ledger data associated with the asset. The attribute 122 ``docType`` is a pattern that can be used in chaincode to differentiate different data 123 types within the chaincode namespace that may need to be queried separately. When using CouchDB, 124 each chaincode is represented as its own CouchDB database, that is, each chaincode has its own namespace for keys. 125 126 With respect to the Asset data structure, ``docType`` is used to identify 127 that this JSON document represents an asset. Potentially there could be other 128 JSON document types in the chaincode namespace. Any of the JSON fields can be 129 used in CouchDB JSON queries. 130 131 When defining an index for use in chaincode queries, each one must be defined 132 in its own text file with the extension `*.json` and the index definition must 133 be formatted in the CouchDB index JSON format. 134 135 To define an index, three pieces of information are required: 136 137 * `fields`: these are the fields to query 138 * `name`: name of the index 139 * `type`: always "json" in this context 140 141 For example, a simple index named ``foo-index`` for a field named ``foo``. 142 143 .. code:: json 144 145 { 146 "index": { 147 "fields": ["foo"] 148 }, 149 "name" : "foo-index", 150 "type" : "json" 151 } 152 153 Optionally the design document attribute ``ddoc`` can be specified on the index 154 definition. A `design document <http://guide.couchdb.org/draft/design.html>`__ is 155 a CouchDB construct designed to contain indexes. Indexes can be grouped into 156 design documents for efficiency but CouchDB recommends one index per design 157 document. 158 159 .. tip:: When defining an index it is a good practice to include the ``ddoc`` 160 attribute and value along with the index name. It is important to 161 include this attribute to ensure that you can update the index later 162 if needed. Also it gives you the ability to explicitly specify which 163 index to use on a query. 164 165 166 Here is another example of an index definition from the Asset transfer ledger queries sample with 167 the index name ``indexOwner`` using multiple fields ``docType`` and ``owner`` 168 and includes the ``ddoc`` attribute: 169 170 .. _indexExample: 171 172 .. code:: json 173 174 { 175 "index":{ 176 "fields":["docType","owner"] // Names of the fields to be queried 177 }, 178 "ddoc":"indexOwnerDoc", // (optional) Name of the design document in which the index will be created. 179 "name":"indexOwner", 180 "type":"json" 181 } 182 183 In the example above, if the design document ``indexOwnerDoc`` does not already 184 exist, it is automatically created when the index is deployed. An index can be 185 constructed with one or more attributes specified in the list of fields and 186 any combination of attributes can be specified. An attribute can exist in 187 multiple indexes for the same docType. In the following example, ``index1`` 188 only includes the attribute ``owner``, ``index2`` includes the attributes 189 ``owner`` and ``color``, and ``index3`` includes the attributes ``owner``, ``color``, and 190 ``size``. Also, notice each index definition has its own ``ddoc`` value, following 191 the CouchDB recommended practice. 192 193 .. code:: json 194 195 { 196 "index":{ 197 "fields":["owner"] // Names of the fields to be queried 198 }, 199 "ddoc":"index1Doc", // (optional) Name of the design document in which the index will be created. 200 "name":"index1", 201 "type":"json" 202 } 203 204 { 205 "index":{ 206 "fields":["owner", "color"] // Names of the fields to be queried 207 }, 208 "ddoc":"index2Doc", // (optional) Name of the design document in which the index will be created. 209 "name":"index2", 210 "type":"json" 211 } 212 213 { 214 "index":{ 215 "fields":["owner", "color", "size"] // Names of the fields to be queried 216 }, 217 "ddoc":"index3Doc", // (optional) Name of the design document in which the index will be created. 218 "name":"index3", 219 "type":"json" 220 } 221 222 223 In general, you should model index fields to match the fields that will be used 224 in query filters and sorts. For more details on building an index in JSON 225 format refer to the `CouchDB documentation <http://docs.couchdb.org/en/latest/api/database/find.html#db-index>`__. 226 227 A final word on indexing, Fabric takes care of indexing the documents in the 228 database using a pattern called ``index warming``. CouchDB does not typically 229 index new or updated documents until the next query. Fabric ensures that 230 indexes stay 'warm' by requesting an index update after every block of data is 231 committed. This ensures queries are fast because they do not have to index 232 documents before running the query. This process keeps the index current 233 and refreshed every time new records are added to the state database. 234 235 .. _cdb-add-index: 236 237 238 Add the index to your chaincode folder 239 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 240 241 Once you finalize an index, you need to package it with your chaincode for 242 deployment by placing it in the appropriate metadata folder. You can package and install the 243 chaincode using the :doc:`commands/peerlifecycle` commands. The JSON index files 244 must be located under the path ``META-INF/statedb/couchdb/indexes`` which is 245 located inside the directory where the chaincode resides. 246 247 The `Asset transfer ledger queries sample <https://github.com/hyperledger/fabric-samples/tree/{BRANCH}/asset-transfer-ledger-queries/chaincode-go>`__ below illustrates how the index 248 is packaged with the chaincode. 249 250 .. image:: images/couchdb_tutorial_pkg_example.png 251 :scale: 100% 252 :align: center 253 :alt: Marbles Chaincode Index Package 254 255 This sample includes one index named indexOwnerDoc, to support queries by asset owner: 256 257 .. code:: json 258 259 {"index":{"fields":["docType","owner"]},"ddoc":"indexOwnerDoc", "name":"indexOwner","type":"json"} 260 261 262 Start the network 263 ----------------- 264 265 :guilabel:`Try it yourself` 266 267 268 We will bring up the Fabric test network and use it to deploy the asset transfer ledger queries 269 chaincode. Use the following command to navigate to the `test-network` directory 270 in the Fabric samples: 271 272 .. code:: bash 273 274 cd fabric-samples/test-network 275 276 For this tutorial, we want to operate from a known initial state. The following 277 command will kill any active or stale Docker containers and remove previously 278 generated artifacts: 279 280 .. code:: bash 281 282 ./network.sh down 283 284 If you have not run through the tutorial before, you will need to vendor the 285 chaincode dependencies before we can deploy it to the network. Run the 286 following commands: 287 288 .. code:: bash 289 290 cd ../asset-transfer-ledger-queries/chaincode-go 291 GO111MODULE=on go mod vendor 292 cd ../../test-network 293 294 From the `test-network` directory, deploy the test network with CouchDB with the 295 following command: 296 297 .. code:: bash 298 299 ./network.sh up createChannel -s couchdb 300 301 This will create two fabric peer nodes that use CouchDB as the state database. 302 It will also create one ordering node and a single channel named ``mychannel``. 303 304 .. _cdb-install-deploy: 305 306 Deploy the smart contract 307 ~~~~~~~~~~~~~~~~~~~~~~~~~ 308 309 You can use the test network script to deploy the asset transfer ledger queries smart contract to the channel. 310 Run the following command to deploy the smart contract to `mychannel`: 311 312 .. code:: bash 313 314 ./network.sh deployCC -ccn ledger -ccp ../asset-transfer-ledger-queries/chaincode-go/ -ccl go -ccep "OR('Org1MSP.peer','Org2MSP.peer')" 315 316 Note that we are using the `-ccep` flag to deploy the smart contract with an endorsement policy of 317 `"OR('Org1MSP.peer','Org2MSP.peer')"`. This allows either organization to create an asset without 318 receiving an endorsement from the other organization. 319 320 Verify index was deployed 321 ------------------------- 322 323 Indexes will be deployed to each peer's CouchDB state database once the 324 chaincode has been installed on the peer and deployed to the channel. You can 325 verify that the CouchDB index was created successfully by examining the peer log 326 in the Docker container. 327 328 :guilabel:`Try it yourself` 329 330 To view the logs in the peer Docker container, open a new Terminal window and 331 run the following command to grep for message confirmation that the index was 332 created. 333 334 :: 335 336 docker logs peer0.org1.example.com 2>&1 | grep "CouchDB index" 337 338 339 You should see a result that looks like the following: 340 341 :: 342 343 [couchdb] createIndex -> INFO 072 Created CouchDB index [indexOwner] in state database [mychannel_ledger] using design document [_design/indexOwnerDoc] 344 345 .. _cdb-query: 346 347 Query the CouchDB State Database 348 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 349 350 Now that the index has been defined in the JSON file and deployed alongside the 351 chaincode, chaincode functions can execute JSON queries against the CouchDB 352 state database. 353 354 Specifying an index name on a query is optional. If not specified, and an index 355 already exists for the fields being queried, the existing index will be 356 automatically used. 357 358 .. tip:: It is a good practice to explicitly include an index name on a 359 query using the ``use_index`` keyword. Without it, CouchDB may pick a 360 less optimal index. Also CouchDB may not use an index at all and you 361 may not realize it, at the low volumes during testing. Only upon 362 higher volumes you may realize slow performance because CouchDB is not 363 using an index. 364 365 366 Build the query in chaincode 367 ---------------------------- 368 369 You can perform JSON queries against the data on the ledger using 370 queries defined within your chaincode. The `Asset transfer ledger queries sample 371 <https://github.com/hyperledger/fabric-samples/blob/{BRANCH}/asset-transfer-ledger-queries/chaincode-go/asset_transfer_ledger_chaincode.go>`__ 372 includes two JSON query functions: 373 374 * **QueryAssets** 375 376 Example of an **ad hoc JSON query**. This is a query 377 where a selector JSON query string can be passed into the function. This query 378 would be useful to client applications that need to dynamically build 379 their own queries at runtime. For more information on query selectors refer 380 to `CouchDB selector syntax <http://docs.couchdb.org/en/latest/api/database/find.html#find-selectors>`__. 381 382 383 * **QueryAssetsByOwner** 384 385 Example of a **parameterized query** where the 386 query is defined in the chaincode but allows a query parameter to be passed in. 387 In this case the function accepts a single argument, the asset owner. It then queries the state database for 388 JSON documents matching the docType of “asset” and the owner id using the 389 JSON query syntax. 390 391 392 Run the query using the peer command 393 ------------------------------------ 394 395 In absence of a client application, we can use the peer command to test the 396 queries defined in the chaincode. We will use the `peer chaincode query <commands/peerchaincode.html?%20chaincode%20query#peer-chaincode-query>`__ 397 command to use the Assets index ``indexOwner`` and query for all assets owned 398 by "tom" using the ``QueryAssets`` function. 399 400 :guilabel:`Try it yourself` 401 402 Before querying the database, we should add some data. Run the following 403 command as Org1 to create a asset owned by "tom": 404 405 .. code:: bash 406 407 export CORE_PEER_LOCALMSPID="Org1MSP" 408 export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt 409 export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp 410 export CORE_PEER_ADDRESS=localhost:7051 411 peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile ${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n ledger -c '{"Args":["CreateAsset","asset1","blue","5","tom","35"]}' 412 413 Next, query for all assets owned by tom: 414 415 .. code:: bash 416 417 // Rich Query with index name explicitly specified: 418 peer chaincode query -C mychannel -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}"]}' 419 420 Delving into the query command above, there are three arguments of interest: 421 422 * ``QueryAssets`` 423 424 Name of the function in the Assets chaincode. As you can see in the chaincode function 425 below, QueryAssets() calls ``getQueryResultForQueryString()``, which then passes the 426 queryString to the ``getQueryResult()`` shim API that executes the JSON query against the state database. 427 428 .. code:: bash 429 430 func (t *SimpleChaincode) QueryAssets(ctx contractapi.TransactionContextInterface, queryString string) ([]*Asset, error) { 431 return getQueryResultForQueryString(ctx, queryString) 432 } 433 434 435 * ``{"selector":{"docType":"asset","owner":"tom"}`` 436 437 This is an example of an **ad hoc selector** string which query for all documents 438 of type ``asset`` where the ``owner`` attribute has a value of ``tom``. 439 440 441 * ``"use_index":["_design/indexOwnerDoc", "indexOwner"]`` 442 443 Specifies both the design doc name ``indexOwnerDoc`` and index name 444 ``indexOwner``. In this example the selector query explicitly includes the 445 index name, specified by using the ``use_index`` keyword. Recalling the 446 index definition above :ref:`cdb-create-index`, it contains a design doc, 447 ``"ddoc":"indexOwnerDoc"``. With CouchDB, if you plan to explicitly include 448 the index name on the query, then the index definition must include the 449 ``ddoc`` value, so it can be referenced with the ``use_index`` keyword. 450 451 452 The query runs successfully and the index is leveraged with the following results: 453 454 .. code:: json 455 456 [{"docType":"asset","ID":"asset1","color":"blue","size":5,"owner":"tom","appraisedValue":35}] 457 458 .. _cdb-best: 459 460 Use best practices for queries and indexes 461 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 462 463 Queries that use indexes will complete faster, without having to scan the full 464 database in CouchDB. Understanding indexes will allow you to write your queries 465 for better performance and help your application handle larger amounts 466 of data. 467 468 It is also important to plan the indexes you install with your chaincode. You 469 should install only a few indexes per chaincode that support most of your queries. 470 Adding too many indexes, or using an excessive number of fields in an index, will 471 degrade the performance of your network. This is because the indexes are updated 472 after each block is committed. The more indexes that need to be updated through 473 "index warming", the longer it will take for transactions to complete. 474 475 The examples in this section will help demonstrate how queries use indexes and 476 what type of queries will have the best performance. Remember the following 477 when writing your queries: 478 479 * All fields in the index must also be in the selector or sort sections of your query 480 for the index to be used. 481 * More complex queries will have a lower performance and will be less likely to 482 use an index. 483 * You should avoid operators that will result in a full table scan or a 484 full index scan such as ``$or``, ``$in`` and ``$regex``. 485 486 In the previous section of this tutorial, you issued the following query against 487 the assets chaincode: 488 489 .. code:: bash 490 491 // Example one: query fully supported by the index 492 export CHANNEL_NAME=mychannel 493 peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}' 494 495 The asset transfer ledger queries chaincode was installed with the ``indexOwnerDoc`` index: 496 497 .. code:: json 498 499 {"index":{"fields":["docType","owner"]},"ddoc":"indexOwnerDoc", "name":"indexOwner","type":"json"} 500 501 Notice that both the fields in the query, ``docType`` and ``owner``, are 502 included in the index, making it a fully supported query. As a result this 503 query will be able to use the data in the index, without having to search the 504 full database. Fully supported queries such as this one will return faster than 505 other queries from your chaincode. 506 507 If you add extra fields to the query above, it will still use the index. 508 However, the query will additionally have to scan the database for the 509 extra fields, resulting in a longer response time. As an example, the query 510 below will still use the index, but will take a longer time to return than the 511 previous example. 512 513 .. code:: bash 514 515 // Example two: query fully supported by the index with additional data 516 peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\",\"color\":\"blue\"}, \"use_index\":[\"/indexOwnerDoc\", \"indexOwner\"]}"]}' 517 518 A query that does not include all fields in the index will have to scan the full 519 database instead. For example, the query below searches for the owner, without 520 specifying the type of item owned. Since the ownerIndexDoc contains both 521 the ``owner`` and ``docType`` fields, this query will not be able to use the 522 index. 523 524 .. code:: bash 525 526 // Example three: query not supported by the index 527 peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"owner\":\"tom\"}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}' 528 529 In general, more complex queries will have a longer response time, and have a 530 lower chance of being supported by an index. Operators such as ``$or``, ``$in``, 531 and ``$regex`` will often cause the query to scan the full index or not use the 532 index at all. 533 534 As an example, the query below contains an ``$or`` term that will search for every 535 asset and every item owned by tom. 536 537 .. code:: bash 538 539 // Example four: query with $or supported by the index 540 peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"$or\":[{\"docType\":\"asset\"},{\"owner\":\"tom\"}]}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}' 541 542 This query will still use the index because it searches for fields that are 543 included in ``indexOwnerDoc``. However, the ``$or`` condition in the query 544 requires a scan of all the items in the index, resulting in a longer response 545 time. 546 547 Below is an example of a complex query that is not supported by the index. 548 549 .. code:: bash 550 551 // Example five: Query with $or not supported by the index 552 peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"$or\":[{\"docType\":\"asset\",\"owner\":\"tom\"},{\"color\":\"yellow\"}]}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}' 553 554 The query searches for all assets owned by tom or any other items that are 555 yellow. This query will not use the index because it will need to search the 556 entire table to meet the ``$or`` condition. Depending the amount of data on your 557 ledger, this query will take a long time to respond or may timeout. 558 559 While it is important to follow best practices with your queries, using indexes 560 is not a solution for collecting large amounts of data. The blockchain data 561 structure is optimized to validate and confirm transactions and is not suited 562 for data analytics or reporting. If you want to build a dashboard as part 563 of your application or analyze the data from your network, the best practice is 564 to query an off chain database that replicates the data from your peers. This 565 will allow you to understand the data on the blockchain without degrading the 566 performance of your network or disrupting transactions. 567 568 You can use block or chaincode events from your application to write transaction 569 data to an off-chain database or analytics engine. For each block received, the block 570 listener application would iterate through the block transactions and build a data 571 store using the key/value writes from each valid transaction's ``rwset``. The 572 :doc:`peer_event_services` provide replayable events to ensure the integrity of 573 downstream data stores. For an example of how you can use an event listener to write 574 data to an external database, visit the `Off chain data sample <https://github.com/hyperledger/fabric-samples/tree/{BRANCH}/off_chain_data>`__ 575 in the Fabric Samples. 576 577 .. _cdb-pagination: 578 579 Query the CouchDB State Database With Pagination 580 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 581 582 When large result sets are returned by CouchDB queries, a set of APIs is 583 available which can be called by chaincode to paginate the list of results. 584 Pagination provides a mechanism to partition the result set by 585 specifying a ``pagesize`` and a start point -- a ``bookmark`` which indicates 586 where to begin the result set. The client application iteratively invokes the 587 chaincode that executes the query until no more results are returned. For more information refer to 588 this `topic on pagination with CouchDB <couchdb_as_state_database.html#couchdb-pagination>`__. 589 590 591 We will use the `Asset transfer ledger queries sample <https://github.com/hyperledger/fabric-samples/blob/{BRANCH}/asset-transfer-ledger-queries/chaincode-go/asset_transfer_ledger_chaincode.go>`__ 592 function ``QueryAssetsWithPagination`` to demonstrate how 593 pagination can be implemented in chaincode and the client application. 594 595 * **QueryAssetsWithPagination** -- 596 597 Example of an **ad hoc JSON query with pagination**. This is a query 598 where a selector string can be passed into the function similar to the 599 above example. In this case, a ``pageSize`` is also included with the query as 600 well as a ``bookmark``. 601 602 In order to demonstrate pagination, more data is required. This example assumes 603 that you have already added asset1 from above. Run the following commands in 604 the peer container to create four more assets owned by "tom", to create a 605 total of five assets owned by "tom": 606 607 :guilabel:`Try it yourself` 608 609 .. code:: bash 610 611 export CORE_PEER_LOCALMSPID="Org1MSP" 612 export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt 613 export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp 614 export CORE_PEER_ADDRESS=localhost:7051 615 peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile ${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n ledger -c '{"Args":["CreateAsset","asset2","yellow","5","tom","35"]}' 616 peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile ${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n ledger -c '{"Args":["CreateAsset","asset3","green","6","tom","20"]}' 617 peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile ${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n ledger -c '{"Args":["CreateAsset","asset4","purple","7","tom","20"]}' 618 peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile ${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n ledger -c '{"Args":["CreateAsset","asset5","blue","8","tom","40"]}' 619 620 In addition to the arguments for the query in the previous example, 621 QueryAssetsWithPagination adds ``pagesize`` and ``bookmark``. ``PageSize`` 622 specifies the number of records to return per query. The ``bookmark`` is an 623 "anchor" telling couchDB where to begin the page. (Each page of results returns 624 a unique bookmark.) 625 626 * ``QueryAssetsWithPagination`` 627 628 As you can see in the chaincode function below, QueryAssetsWithPagination() calls 629 ``getQueryResultForQueryStringWithPagination()``, which then passes the 630 queryString as well as the bookmark and pagesize to the ``GetQueryResultWithPagination()`` 631 shim API that executes the paginated JSON query against the state database. 632 633 .. code:: bash 634 635 func (t *SimpleChaincode) QueryAssetsWithPagination( 636 ctx contractapi.TransactionContextInterface, 637 queryString, 638 bookmark string, 639 pageSize int) ([]*Asset, error) { 640 641 return getQueryResultForQueryStringWithPagination(ctx, queryString, int32(pageSize), bookmark) 642 } 643 644 The following example is a peer command which calls QueryAssetsWithPagination 645 with a pageSize of ``3`` and no bookmark specified. 646 647 .. tip:: When no bookmark is specified, the query starts with the "first" 648 page of records. 649 650 :guilabel:`Try it yourself` 651 652 .. code:: bash 653 654 // Rich Query with index name explicitly specified and a page size of 3: 655 peer chaincode query -C mychannel -n ledger -c '{"Args":["QueryAssetsWithPagination", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}","","3"]}' 656 657 The following response is received (carriage returns added for clarity), three 658 of the five assets are returned because the ``pagsize`` was set to ``3``: 659 660 .. code:: bash 661 662 [{"docType":"asset","ID":"asset1","color":"blue","size":5,"owner":"tom","appraisedValue":35}, 663 {"docType":"asset","ID":"asset2","color":"yellow","size":5,"owner":"tom","appraisedValue":35}, 664 {"docType":"asset","ID":"asset3","color":"green","size":6,"owner":"tom","appraisedValue":20}] 665 666 667 .. note:: Bookmarks are uniquely generated by CouchDB for each query and 668 represent a placeholder in the result set. Pass the 669 returned bookmark on the subsequent iteration of the query to 670 retrieve the next set of results. 671 672 The following is a peer command to call QueryAssetsWithPagination with a 673 pageSize of ``3``. Notice this time, the query includes the bookmark returned 674 from the previous query. 675 676 :guilabel:`Try it yourself` 677 678 .. code:: bash 679 680 peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssetsWithPagination", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}","g1AAAABLeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqz5yYWJeWkGoOkOWDSOSANIFk2iCyIyVySn5uVBQAGEhRz","3"]}' 681 682 The following response is received (carriage returns added for clarity). The 683 last two records are retrieved: 684 685 .. code:: bash 686 687 [{"Key":"asset4", "Record":{"color":"purple","docType":"asset","name":"asset4","size":"7","owner":"tom","appraisedValue":20}}, 688 {"Key":"asset5", "Record":{"color":"blue","docType":"asset","name":"asset5","size":"8","owner":"tom","appraisedValue":40}}] 689 [{"ResponseMetadata":{"RecordsCount":"2", 690 "Bookmark":"g1AAAABLeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqz5yYWJeWkmoKkOWDSOSANIFk2iCyIyVySn5uVBQAGYhR1"}}] 691 692 The final command is a peer command to call QueryAssetsWithPagination with 693 a pageSize of ``3`` and with the bookmark from the previous query. 694 695 :guilabel:`Try it yourself` 696 697 .. code:: bash 698 699 peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssetsWithPagination", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}","g1AAAABLeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqz5yYWJeWkmoKkOWDSOSANIFk2iCyIyVySn5uVBQAGYhR1","3"]}' 700 701 The following response is received (carriage returns added for clarity). 702 No records are returned, indicating that all pages have been retrieved: 703 704 .. code:: bash 705 706 [] 707 [{"ResponseMetadata":{"RecordsCount":"0", 708 "Bookmark":"g1AAAABLeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqz5yYWJeWkmoKkOWDSOSANIFk2iCyIyVySn5uVBQAGYhR1"}}] 709 710 For an example of how a client application can iterate over 711 the result sets using pagination, search for the ``getQueryResultForQueryStringWithPagination`` 712 function in the `Asset transfer ledger queries sample <https://github.com/hyperledger/fabric-samples/blob/{BRANCH}/asset-transfer-ledger-queries/chaincode-go/asset_transfer_ledger_chaincode.go>`__. 713 714 .. _cdb-update-index: 715 716 Update an Index 717 ~~~~~~~~~~~~~~~ 718 719 It may be necessary to update an index over time. The same index may exist in 720 subsequent versions of the chaincode that gets installed. In order for an index 721 to be updated, the original index definition must have included the design 722 document ``ddoc`` attribute and an index name. To update an index definition, 723 use the same index name but alter the index definition. Simply edit the index 724 JSON file and add or remove fields from the index. Fabric only supports the 725 index type JSON. Changing the index type is not supported. The updated 726 index definition gets redeployed to the peer’s state database when the chaincode 727 definition is committed to the channel. Changes to the index name or ``ddoc`` 728 attributes will result in a new index being created and the original index remains 729 unchanged in CouchDB until it is removed. 730 731 .. note:: If the state database has a significant volume of data, it will take 732 some time for the index to be re-built, during which time chaincode 733 invokes that issue queries may fail or timeout. 734 735 Iterating on your index definition 736 ---------------------------------- 737 738 If you have access to your peer's CouchDB state database in a development 739 environment, you can iteratively test various indexes in support of 740 your chaincode queries. Any changes to chaincode though would require 741 redeployment. Use the `CouchDB Fauxton interface <http://docs.couchdb.org/en/latest/fauxton/index.html>`__ or a command 742 line curl utility to create and update indexes. 743 744 .. note:: The Fauxton interface is a web UI for the creation, update, and 745 deployment of indexes to CouchDB. If you want to try out this 746 interface, there is an example of the format of the Fauxton version 747 of the index in Assets sample. If you have deployed the test network 748 with CouchDB, the Fauxton interface can be loaded by opening a browser 749 and navigating to ``http://localhost:5984/_utils``. 750 751 Alternatively, if you prefer not use the Fauxton UI, the following is an example 752 of a curl command which can be used to create the index on the database 753 ``mychannel_ledger``: 754 755 .. code:: bash 756 757 // Index for docType, owner. 758 // Example curl command line to define index in the CouchDB channel_chaincode database 759 curl -i -X POST -H "Content-Type: application/json" -d 760 "{\"index\":{\"fields\":[\"docType\",\"owner\"]}, 761 \"name\":\"indexOwner\", 762 \"ddoc\":\"indexOwnerDoc\", 763 \"type\":\"json\"}" http://hostname:port/mychannel_ledger/_index 764 765 .. note:: If you are using the test network configured with CouchDB, replace 766 hostname:port with ``localhost:5984``. 767 768 .. _cdb-delete-index: 769 770 Delete an Index 771 ~~~~~~~~~~~~~~~ 772 773 Index deletion is not managed by Fabric tooling. If you need to delete an index, 774 manually issue a curl command against the database or delete it using the 775 Fauxton interface. 776 777 The format of the curl command to delete an index would be: 778 779 .. code:: bash 780 781 curl -X DELETE http://localhost:5984/{database_name}/_index/{design_doc}/json/{index_name} -H "accept: */*" -H "Host: localhost:5984" 782 783 784 To delete the index used in this tutorial, the curl command would be: 785 786 .. code:: bash 787 788 curl -X DELETE http://localhost:5984/mychannel_ledger/_index/indexOwnerDoc/json/indexOwner -H "accept: */*" -H "Host: localhost:5984" 789 790 791 792 .. Licensed under Creative Commons Attribution 4.0 International License 793 https://creativecommons.org/licenses/by/4.0/