github.com/osdi23p228/fabric@v0.0.0-20221218062954-77808885f5db/docs/source/couchdb_tutorial.rst (about)

     1  
     2  Using CouchDB
     3  =============
     4  
     5  This tutorial will describe the steps required to use CouchDB as the state
     6  database with Hyperledger Fabric. By now, you should be familiar with Fabric
     7  concepts and have explored some of the samples and tutorials.
     8  
     9  .. note:: These instructions use the new Fabric chaincode lifecycle introduced
    10            in the Fabric v2.0 release. If you would like to use the previous
    11            lifecycle model to use indexes with chaincode, visit the v1.4
    12            version of the `Using CouchDB <https://hyperledger-fabric.readthedocs.io/en/release-1.4/couchdb_tutorial.html>`__.
    13  
    14  The tutorial will take you through the following steps:
    15  
    16  #. :ref:`cdb-enable-couch`
    17  #. :ref:`cdb-create-index`
    18  #. :ref:`cdb-add-index`
    19  #. :ref:`cdb-install-deploy`
    20  #. :ref:`cdb-query`
    21  #. :ref:`cdb-best`
    22  #. :ref:`cdb-pagination`
    23  #. :ref:`cdb-update-index`
    24  #. :ref:`cdb-delete-index`
    25  
    26  For a deeper dive into CouchDB refer to :doc:`couchdb_as_state_database`
    27  and for more information on the Fabric ledger refer to the `Ledger <ledger/ledger.html>`_
    28  topic. Follow the tutorial below for details on how to leverage CouchDB in your
    29  blockchain network.
    30  
    31  Throughout this tutorial, we will use the `Asset transfer ledger queries sample <https://github.com/hyperledger/fabric-samples/blob/{BRANCH}/asset-transfer-ledger-queries/chaincode-go>`__
    32  as our use case to demonstrate how to use CouchDB with Fabric, including the
    33  execution of JSON queries against the state database. You should have completed the task
    34  :doc:`install`.
    35  
    36  Why CouchDB?
    37  ~~~~~~~~~~~~
    38  
    39  Fabric supports two types of peer state databases. LevelDB is the default state
    40  database embedded in the peer node. LevelDB stores chaincode data as simple
    41  key-value pairs and only supports key, key range, and composite key queries.
    42  CouchDB is an optional, alternate state database that allows you to model data
    43  on the ledger as JSON and issue rich queries against data values rather than
    44  the keys. The CouchDB support also allows you to deploy indexes with your chaincode to make
    45  queries more efficient and enable you to query large datasets.
    46  
    47  In order to leverage the benefits of CouchDB, namely content-based JSON
    48  queries, your data must be modeled in JSON format. You must decide whether to use
    49  LevelDB or CouchDB before setting up your network. Switching a peer from using
    50  LevelDB to CouchDB is not supported due to data compatibility issues. All peers
    51  on the network must use the same database type. If you have a mix of JSON and
    52  binary data values, you can still use CouchDB, however the binary values can
    53  only be queried based on key, key range, and composite key queries.
    54  
    55  .. _cdb-enable-couch:
    56  
    57  Enable CouchDB in Hyperledger Fabric
    58  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    59  
    60  CouchDB runs as a separate database process alongside the peer. There
    61  are additional considerations in terms of setup, management, and operations.
    62  A Docker image of `CouchDB <https://hub.docker.com/_/couchdb/>`__
    63  is available and we recommend that it be run on the same server as the
    64  peer. You will need to setup one CouchDB container per peer
    65  and update each peer container by changing the configuration found in
    66  ``core.yaml`` to point to the CouchDB container. The ``core.yaml``
    67  file must be located in the directory specified by the environment variable
    68  FABRIC_CFG_PATH:
    69  
    70  * For Docker deployments, ``core.yaml`` is pre-configured and located in the peer
    71    container ``FABRIC_CFG_PATH`` folder. However, when using Docker environments,
    72    you can pass environment variables to override the core.yaml properties, for
    73    example ``CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS`` to set the CouchDB address.
    74  
    75  * For native binary deployments, ``core.yaml`` is included with the release artifact
    76    distribution.
    77  
    78  Edit the ``stateDatabase`` section of ``core.yaml``. Specify ``CouchDB`` as the
    79  ``stateDatabase`` and fill in the associated ``couchDBConfig`` properties. For
    80  more information, see `CouchDB configuration <couchdb_as_state_database.html#couchdb-configuration>`__.
    81  
    82  .. _cdb-create-index:
    83  
    84  Create an index
    85  ~~~~~~~~~~~~~~~
    86  
    87  Why are indexes important?
    88  
    89  Indexes allow a database to be queried without having to examine every row
    90  with every query, making them run faster and more efficiently. Normally,
    91  indexes are built for frequently occurring query criteria allowing the data to
    92  be queried more efficiently. To leverage the major benefit of CouchDB -- the
    93  ability to perform rich queries against JSON data -- indexes are not required,
    94  but they are strongly recommended for performance. Also, if sorting is required
    95  in a query, CouchDB requires an index that includes the sorted fields.
    96  
    97  .. note::
    98  
    99     JSON queries that do not have an index may work but will throw a warning
   100     in the peer log that the index was not found. However, if a rich query
   101     includes a sort specification, then an index on that field is required;
   102     otherwise, the query will fail and an error will be thrown.
   103  
   104  To demonstrate building an index, we will use the data from the `Asset transfer ledger queries
   105  sample <https://github.com/hyperledger/fabric-samples/blob/{BRANCH}/asset-transfer-ledger-queries/chaincode-go/asset_transfer_ledger_chaincode.go>`__.
   106  In this example, the Asset data structure is defined as:
   107  
   108  .. code:: javascript
   109  
   110      type Asset struct {
   111              DocType        string `json:"docType"` //docType is used to distinguish the various types of objects in state database
   112              ID             string `json:"ID"`      //the field tags are needed to keep case from bouncing around
   113              Color          string `json:"color"`
   114              Size           int    `json:"size"`
   115              Owner          string `json:"owner"`
   116              AppraisedValue int    `json:"appraisedValue"`
   117      }
   118  
   119  
   120  In this structure, the attributes (``docType``, ``ID``, ``color``, ``size``,
   121  ``owner``, ``appraisedValue``) define the ledger data associated with the asset. The attribute
   122  ``docType`` is a pattern that can be used in chaincode to differentiate different data
   123  types within the chaincode namespace that may need to be queried separately. When using CouchDB,
   124  each chaincode is represented as its own CouchDB database, that is, each chaincode has its own namespace for keys.
   125  
   126  With respect to the Asset data structure, ``docType`` is used to identify
   127  that this JSON document represents an asset. Potentially there could be other
   128  JSON document types in the chaincode namespace. Any of the JSON fields can be
   129  used in CouchDB JSON queries.
   130  
   131  When defining an index for use in chaincode queries, each one must be defined
   132  in its own text file with the extension `*.json` and the index definition must
   133  be formatted in the CouchDB index JSON format.
   134  
   135  To define an index, three pieces of information are required:
   136  
   137    * `fields`: these are the fields to query
   138    * `name`: name of the index
   139    * `type`: always "json" in this context
   140  
   141  For example, a simple index named ``foo-index`` for a field named ``foo``.
   142  
   143  .. code:: json
   144  
   145      {
   146          "index": {
   147              "fields": ["foo"]
   148          },
   149          "name" : "foo-index",
   150          "type" : "json"
   151      }
   152  
   153  Optionally the design document  attribute ``ddoc`` can be specified on the index
   154  definition. A `design document <http://guide.couchdb.org/draft/design.html>`__ is
   155  a CouchDB construct designed to contain indexes. Indexes can be grouped into
   156  design documents for efficiency but CouchDB recommends one index per design
   157  document.
   158  
   159  .. tip:: When defining an index it is a good practice to include the ``ddoc``
   160           attribute and value along with the index name. It is important to
   161           include this attribute to ensure that you can update the index later
   162           if needed. Also it gives you the ability to explicitly specify which
   163           index to use on a query.
   164  
   165  
   166  Here is another example of an index definition from the Asset transfer ledger queries sample with
   167  the index name ``indexOwner`` using multiple fields ``docType`` and ``owner``
   168  and includes the ``ddoc`` attribute:
   169  
   170  .. _indexExample:
   171  
   172  .. code:: json
   173  
   174    {
   175      "index":{
   176          "fields":["docType","owner"] // Names of the fields to be queried
   177      },
   178      "ddoc":"indexOwnerDoc", // (optional) Name of the design document in which the index will be created.
   179      "name":"indexOwner",
   180      "type":"json"
   181    }
   182  
   183  In the example above, if the design document ``indexOwnerDoc`` does not already
   184  exist, it is automatically created when the index is deployed. An index can be
   185  constructed with one or more attributes specified in the list of fields and
   186  any combination of attributes can be specified. An attribute can exist in
   187  multiple indexes for the same docType. In the following example, ``index1``
   188  only includes the attribute ``owner``, ``index2`` includes the attributes
   189  ``owner`` and ``color``, and ``index3`` includes the attributes ``owner``, ``color``, and
   190  ``size``. Also, notice each index definition has its own ``ddoc`` value, following
   191  the CouchDB recommended practice.
   192  
   193  .. code:: json
   194  
   195    {
   196      "index":{
   197          "fields":["owner"] // Names of the fields to be queried
   198      },
   199      "ddoc":"index1Doc", // (optional) Name of the design document in which the index will be created.
   200      "name":"index1",
   201      "type":"json"
   202    }
   203  
   204    {
   205      "index":{
   206          "fields":["owner", "color"] // Names of the fields to be queried
   207      },
   208      "ddoc":"index2Doc", // (optional) Name of the design document in which the index will be created.
   209      "name":"index2",
   210      "type":"json"
   211    }
   212  
   213    {
   214      "index":{
   215          "fields":["owner", "color", "size"] // Names of the fields to be queried
   216      },
   217      "ddoc":"index3Doc", // (optional) Name of the design document in which the index will be created.
   218      "name":"index3",
   219      "type":"json"
   220    }
   221  
   222  
   223  In general, you should model index fields to match the fields that will be used
   224  in query filters and sorts. For more details on building an index in JSON
   225  format refer to the `CouchDB documentation <http://docs.couchdb.org/en/latest/api/database/find.html#db-index>`__.
   226  
   227  A final word on indexing, Fabric takes care of indexing the documents in the
   228  database using a pattern called ``index warming``. CouchDB does not typically
   229  index new or updated documents until the next query. Fabric ensures that
   230  indexes stay 'warm' by requesting an index update after every block of data is
   231  committed.  This ensures queries are fast because they do not have to index
   232  documents before running the query. This process keeps the index current
   233  and refreshed every time new records are added to the state database.
   234  
   235  .. _cdb-add-index:
   236  
   237  
   238  Add the index to your chaincode folder
   239  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   240  
   241  Once you finalize an index, you need to package it with your chaincode for
   242  deployment by placing it in the appropriate metadata folder. You can package and install the
   243  chaincode using the :doc:`commands/peerlifecycle` commands. The JSON index files
   244  must be located under the path ``META-INF/statedb/couchdb/indexes`` which is
   245  located inside the directory where the chaincode resides.
   246  
   247  The `Asset transfer ledger queries sample <https://github.com/hyperledger/fabric-samples/tree/{BRANCH}/asset-transfer-ledger-queries/chaincode-go>`__  below illustrates how the index
   248  is packaged with the chaincode.
   249  
   250  .. image:: images/couchdb_tutorial_pkg_example.png
   251    :scale: 100%
   252    :align: center
   253    :alt: Marbles Chaincode Index Package
   254  
   255  This sample includes one index named indexOwnerDoc, to support queries by asset owner:
   256  
   257  .. code:: json
   258  
   259    {"index":{"fields":["docType","owner"]},"ddoc":"indexOwnerDoc", "name":"indexOwner","type":"json"}
   260  
   261  
   262  Start the network
   263  -----------------
   264  
   265  :guilabel:`Try it yourself`
   266  
   267  
   268  We will bring up the Fabric test network and use it to deploy the asset transfer ledger queries
   269  chaincode. Use the following command to navigate to the `test-network` directory
   270  in the Fabric samples:
   271  
   272  .. code:: bash
   273  
   274      cd fabric-samples/test-network
   275  
   276  For this tutorial, we want to operate from a known initial state. The following
   277  command will kill any active or stale Docker containers and remove previously
   278  generated artifacts:
   279  
   280  .. code:: bash
   281  
   282      ./network.sh down
   283  
   284  If you have not run through the tutorial before, you will need to vendor the
   285  chaincode dependencies before we can deploy it to the network. Run the
   286  following commands:
   287  
   288  .. code:: bash
   289  
   290      cd ../asset-transfer-ledger-queries/chaincode-go
   291      GO111MODULE=on go mod vendor
   292      cd ../../test-network
   293  
   294  From the `test-network` directory, deploy the test network with CouchDB with the
   295  following command:
   296  
   297  .. code:: bash
   298  
   299      ./network.sh up createChannel -s couchdb
   300  
   301  This will create two fabric peer nodes that use CouchDB as the state database.
   302  It will also create one ordering node and a single channel named ``mychannel``.
   303  
   304  .. _cdb-install-deploy:
   305  
   306  Deploy the smart contract
   307  ~~~~~~~~~~~~~~~~~~~~~~~~~
   308  
   309  You can use the test network script to deploy the asset transfer ledger queries smart contract to the channel.
   310  Run the following command to deploy the smart contract to `mychannel`:
   311  
   312  .. code:: bash
   313  
   314    ./network.sh deployCC -ccn ledger -ccp ../asset-transfer-ledger-queries/chaincode-go/ -ccl go -ccep "OR('Org1MSP.peer','Org2MSP.peer')"
   315  
   316  Note that we are using the `-ccep` flag to deploy the smart contract with an endorsement policy of
   317  `"OR('Org1MSP.peer','Org2MSP.peer')"`. This allows either organization to create an asset without
   318  receiving an endorsement from the other organization.
   319  
   320  Verify index was deployed
   321  -------------------------
   322  
   323  Indexes will be deployed to each peer's CouchDB state database once the
   324  chaincode has been installed on the peer and deployed to the channel. You can
   325  verify that the CouchDB index was created successfully by examining the peer log
   326  in the Docker container.
   327  
   328  :guilabel:`Try it yourself`
   329  
   330  To view the logs in the peer Docker container, open a new Terminal window and
   331  run the following command to grep for message confirmation that the index was
   332  created.
   333  
   334  ::
   335  
   336     docker logs peer0.org1.example.com  2>&1 | grep "CouchDB index"
   337  
   338  
   339  You should see a result that looks like the following:
   340  
   341  ::
   342  
   343     [couchdb] createIndex -> INFO 072 Created CouchDB index [indexOwner] in state database [mychannel_ledger] using design document [_design/indexOwnerDoc]
   344  
   345  .. _cdb-query:
   346  
   347  Query the CouchDB State Database
   348  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   349  
   350  Now that the index has been defined in the JSON file and deployed alongside the
   351  chaincode, chaincode functions can execute JSON queries against the CouchDB
   352  state database.
   353  
   354  Specifying an index name on a query is optional. If not specified, and an index
   355  already exists for the fields being queried, the existing index will be
   356  automatically used.
   357  
   358  .. tip:: It is a good practice to explicitly include an index name on a
   359           query using the ``use_index`` keyword. Without it, CouchDB may pick a
   360           less optimal index. Also CouchDB may not use an index at all and you
   361           may not realize it, at the low volumes during testing. Only upon
   362           higher volumes you may realize slow performance because CouchDB is not
   363           using an index.
   364  
   365  
   366  Build the query in chaincode
   367  ----------------------------
   368  
   369  You can perform JSON queries against the data on the ledger using
   370  queries defined within your chaincode. The `Asset transfer ledger queries sample
   371  <https://github.com/hyperledger/fabric-samples/blob/{BRANCH}/asset-transfer-ledger-queries/chaincode-go/asset_transfer_ledger_chaincode.go>`__
   372  includes two JSON query functions:
   373  
   374    * **QueryAssets**
   375  
   376        Example of an **ad hoc JSON query**. This is a query
   377        where a selector JSON query string can be passed into the function. This query
   378        would be useful to client applications that need to dynamically build
   379        their own queries at runtime. For more information on query selectors refer
   380        to `CouchDB selector syntax <http://docs.couchdb.org/en/latest/api/database/find.html#find-selectors>`__.
   381  
   382  
   383    * **QueryAssetsByOwner**
   384  
   385        Example of a **parameterized query** where the
   386        query is defined in the chaincode but allows a query parameter to be passed in.
   387        In this case the function accepts a single argument, the asset owner. It then queries the state database for
   388        JSON documents matching the docType of “asset” and the owner id using the
   389        JSON query syntax.
   390  
   391  
   392  Run the query using the peer command
   393  ------------------------------------
   394  
   395  In absence of a client application, we can use the peer command to test the
   396  queries defined in the chaincode. We will use the `peer chaincode query <commands/peerchaincode.html?%20chaincode%20query#peer-chaincode-query>`__
   397  command to use the Assets index ``indexOwner`` and query for all assets owned
   398  by "tom" using the ``QueryAssets`` function.
   399  
   400  :guilabel:`Try it yourself`
   401  
   402  Before querying the database, we should add some data. Run the following
   403  command as Org1 to create a asset owned by "tom":
   404  
   405  .. code:: bash
   406  
   407      export CORE_PEER_LOCALMSPID="Org1MSP"
   408      export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
   409      export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
   410      export CORE_PEER_ADDRESS=localhost:7051
   411      peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile ${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n ledger -c '{"Args":["CreateAsset","asset1","blue","5","tom","35"]}'
   412  
   413  Next, query for all assets owned by tom:
   414  
   415  .. code:: bash
   416  
   417     // Rich Query with index name explicitly specified:
   418     peer chaincode query -C mychannel -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}"]}'
   419  
   420  Delving into the query command above, there are three arguments of interest:
   421  
   422  *  ``QueryAssets``
   423  
   424    Name of the function in the Assets chaincode. As you can see in the chaincode function
   425    below, QueryAssets() calls ``getQueryResultForQueryString()``, which then passes the
   426    queryString to the ``getQueryResult()`` shim API that executes the JSON query against the state database.
   427  
   428  .. code:: bash
   429  
   430      func (t *SimpleChaincode) QueryAssets(ctx contractapi.TransactionContextInterface, queryString string) ([]*Asset, error) {
   431              return getQueryResultForQueryString(ctx, queryString)
   432      }
   433  
   434  
   435  *  ``{"selector":{"docType":"asset","owner":"tom"}``
   436  
   437    This is an example of an **ad hoc selector** string which query for all documents
   438    of type ``asset`` where the ``owner`` attribute has a value of ``tom``.
   439  
   440  
   441  *  ``"use_index":["_design/indexOwnerDoc", "indexOwner"]``
   442  
   443    Specifies both the design doc name  ``indexOwnerDoc`` and index name
   444    ``indexOwner``. In this example the selector query explicitly includes the
   445    index name, specified by using the ``use_index`` keyword. Recalling the
   446    index definition above :ref:`cdb-create-index`, it contains a design doc,
   447    ``"ddoc":"indexOwnerDoc"``. With CouchDB, if you plan to explicitly include
   448    the index name on the query, then the index definition must include the
   449    ``ddoc`` value, so it can be referenced with the ``use_index`` keyword.
   450  
   451  
   452  The query runs successfully and the index is leveraged with the following results:
   453  
   454  .. code:: json
   455  
   456    [{"docType":"asset","ID":"asset1","color":"blue","size":5,"owner":"tom","appraisedValue":35}]
   457  
   458  .. _cdb-best:
   459  
   460  Use best practices for queries and indexes
   461  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   462  
   463  Queries that use indexes will complete faster, without having to scan the full
   464  database in CouchDB. Understanding indexes will allow you to write your queries
   465  for better performance and help your application handle larger amounts
   466  of data.
   467  
   468  It is also important to plan the indexes you install with your chaincode. You
   469  should install only a few indexes per chaincode that support most of your queries.
   470  Adding too many indexes, or using an excessive number of fields in an index, will
   471  degrade the performance of your network. This is because the indexes are updated
   472  after each block is committed. The more indexes that need to be updated through
   473  "index warming", the longer it will take for transactions to complete.
   474  
   475  The examples in this section will help demonstrate how queries use indexes and
   476  what type of queries will have the best performance. Remember the following
   477  when writing your queries:
   478  
   479  * All fields in the index must also be in the selector or sort sections of your query
   480    for the index to be used.
   481  * More complex queries will have a lower performance and will be less likely to
   482    use an index.
   483  * You should avoid operators that will result in a full table scan or a
   484    full index scan such as ``$or``, ``$in`` and ``$regex``.
   485  
   486  In the previous section of this tutorial, you issued the following query against
   487  the assets chaincode:
   488  
   489  .. code:: bash
   490  
   491    // Example one: query fully supported by the index
   492    export CHANNEL_NAME=mychannel
   493    peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}'
   494  
   495  The asset transfer ledger queries chaincode was installed with the ``indexOwnerDoc`` index:
   496  
   497  .. code:: json
   498  
   499    {"index":{"fields":["docType","owner"]},"ddoc":"indexOwnerDoc", "name":"indexOwner","type":"json"}
   500  
   501  Notice that both the fields in the query, ``docType`` and ``owner``, are
   502  included in the index, making it a fully supported query. As a result this
   503  query will be able to use the data in the index, without having to search the
   504  full database. Fully supported queries such as this one will return faster than
   505  other queries from your chaincode.
   506  
   507  If you add extra fields to the query above, it will still use the index.
   508  However, the query will additionally have to scan the database for the
   509  extra fields, resulting in a longer response time. As an example, the query
   510  below will still use the index, but will take a longer time to return than the
   511  previous example.
   512  
   513  .. code:: bash
   514  
   515    // Example two: query fully supported by the index with additional data
   516    peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\",\"color\":\"blue\"}, \"use_index\":[\"/indexOwnerDoc\", \"indexOwner\"]}"]}'
   517  
   518  A query that does not include all fields in the index will have to scan the full
   519  database instead. For example, the query below searches for the owner, without
   520  specifying the type of item owned. Since the ownerIndexDoc contains both
   521  the ``owner`` and ``docType`` fields, this query will not be able to use the
   522  index.
   523  
   524  .. code:: bash
   525  
   526    // Example three: query not supported by the index
   527    peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"owner\":\"tom\"}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}'
   528  
   529  In general, more complex queries will have a longer response time, and have a
   530  lower chance of being supported by an index. Operators such as ``$or``, ``$in``,
   531  and ``$regex`` will often cause the query to scan the full index or not use the
   532  index at all.
   533  
   534  As an example, the query below contains an ``$or`` term that will search for every
   535  asset and every item owned by tom.
   536  
   537  .. code:: bash
   538  
   539    // Example four: query with $or supported by the index
   540    peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"$or\":[{\"docType\":\"asset\"},{\"owner\":\"tom\"}]}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}'
   541  
   542  This query will still use the index because it searches for fields that are
   543  included in ``indexOwnerDoc``. However, the ``$or`` condition in the query
   544  requires a scan of all the items in the index, resulting in a longer response
   545  time.
   546  
   547  Below is an example of a complex query that is not supported by the index.
   548  
   549  .. code:: bash
   550  
   551    // Example five: Query with $or not supported by the index
   552    peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"$or\":[{\"docType\":\"asset\",\"owner\":\"tom\"},{\"color\":\"yellow\"}]}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}'
   553  
   554  The query searches for all assets owned by tom or any other items that are
   555  yellow. This query will not use the index because it will need to search the
   556  entire table to meet the ``$or`` condition. Depending the amount of data on your
   557  ledger, this query will take a long time to respond or may timeout.
   558  
   559  While it is important to follow best practices with your queries, using indexes
   560  is not a solution for collecting large amounts of data. The blockchain data
   561  structure is optimized to validate and confirm transactions and is not suited
   562  for data analytics or reporting. If you want to build a dashboard as part
   563  of your application or analyze the data from your network, the best practice is
   564  to query an off chain database that replicates the data from your peers. This
   565  will allow you to understand the data on the blockchain without degrading the
   566  performance of your network or disrupting transactions.
   567  
   568  You can use block or chaincode events from your application to write transaction
   569  data to an off-chain database or analytics engine. For each block received, the block
   570  listener application would iterate through the block transactions and build a data
   571  store using the key/value writes from each valid transaction's ``rwset``. The
   572  :doc:`peer_event_services` provide replayable events to ensure the integrity of
   573  downstream data stores. For an example of how you can use an event listener to write
   574  data to an external database, visit the `Off chain data sample <https://github.com/hyperledger/fabric-samples/tree/{BRANCH}/off_chain_data>`__
   575  in the Fabric Samples.
   576  
   577  .. _cdb-pagination:
   578  
   579  Query the CouchDB State Database With Pagination
   580  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   581  
   582  When large result sets are returned by CouchDB queries, a set of APIs is
   583  available which can be called by chaincode to paginate the list of results.
   584  Pagination provides a mechanism to partition the result set by
   585  specifying a ``pagesize`` and a start point -- a ``bookmark`` which indicates
   586  where to begin the result set. The client application iteratively invokes the
   587  chaincode that executes the query until no more results are returned. For more information refer to
   588  this `topic on pagination with CouchDB <couchdb_as_state_database.html#couchdb-pagination>`__.
   589  
   590  
   591  We will use the `Asset transfer ledger queries sample <https://github.com/hyperledger/fabric-samples/blob/{BRANCH}/asset-transfer-ledger-queries/chaincode-go/asset_transfer_ledger_chaincode.go>`__
   592  function ``QueryAssetsWithPagination`` to  demonstrate how
   593  pagination can be implemented in chaincode and the client application.
   594  
   595  * **QueryAssetsWithPagination** --
   596  
   597      Example of an **ad hoc JSON query with pagination**. This is a query
   598      where a selector string can be passed into the function similar to the
   599      above example.  In this case, a ``pageSize`` is also included with the query as
   600      well as a ``bookmark``.
   601  
   602  In order to demonstrate pagination, more data is required. This example assumes
   603  that you have already added asset1 from above. Run the following commands in
   604  the peer container to create four more assets owned by "tom", to create a
   605  total of five assets owned by "tom":
   606  
   607  :guilabel:`Try it yourself`
   608  
   609  .. code:: bash
   610  
   611      export CORE_PEER_LOCALMSPID="Org1MSP"
   612      export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
   613      export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
   614      export CORE_PEER_ADDRESS=localhost:7051
   615      peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile  ${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n ledger -c '{"Args":["CreateAsset","asset2","yellow","5","tom","35"]}'
   616      peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile  ${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n ledger -c '{"Args":["CreateAsset","asset3","green","6","tom","20"]}'
   617      peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile  ${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n ledger -c '{"Args":["CreateAsset","asset4","purple","7","tom","20"]}'
   618      peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile  ${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n ledger -c '{"Args":["CreateAsset","asset5","blue","8","tom","40"]}'
   619  
   620  In addition to the arguments for the query in the previous example,
   621  QueryAssetsWithPagination adds ``pagesize`` and ``bookmark``. ``PageSize``
   622  specifies the number of records to return per query.  The ``bookmark`` is an
   623  "anchor" telling couchDB where to begin the page. (Each page of results returns
   624  a unique bookmark.)
   625  
   626  *  ``QueryAssetsWithPagination``
   627  
   628    As you can see in the chaincode function below, QueryAssetsWithPagination() calls
   629    ``getQueryResultForQueryStringWithPagination()``, which then passes the
   630    queryString as well as the bookmark and pagesize to the ``GetQueryResultWithPagination()``
   631    shim API that executes the paginated JSON query against the state database.
   632  
   633  .. code:: bash
   634  
   635      func (t *SimpleChaincode) QueryAssetsWithPagination(
   636              ctx contractapi.TransactionContextInterface,
   637              queryString,
   638              bookmark string,
   639              pageSize int) ([]*Asset, error) {
   640  
   641              return getQueryResultForQueryStringWithPagination(ctx, queryString, int32(pageSize), bookmark)
   642      }
   643  
   644  The following example is a peer command which calls QueryAssetsWithPagination
   645  with a pageSize of ``3`` and no bookmark specified.
   646  
   647  .. tip:: When no bookmark is specified, the query starts with the "first"
   648           page of records.
   649  
   650  :guilabel:`Try it yourself`
   651  
   652  .. code:: bash
   653  
   654    // Rich Query with index name explicitly specified and a page size of 3:
   655    peer chaincode query -C mychannel -n ledger -c '{"Args":["QueryAssetsWithPagination", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}","","3"]}'
   656  
   657  The following response is received (carriage returns added for clarity), three
   658  of the five assets are returned because the ``pagsize`` was set to ``3``:
   659  
   660  .. code:: bash
   661  
   662    [{"docType":"asset","ID":"asset1","color":"blue","size":5,"owner":"tom","appraisedValue":35},
   663    {"docType":"asset","ID":"asset2","color":"yellow","size":5,"owner":"tom","appraisedValue":35},
   664    {"docType":"asset","ID":"asset3","color":"green","size":6,"owner":"tom","appraisedValue":20}]
   665  
   666  
   667  .. note::  Bookmarks are uniquely generated by CouchDB for each query and
   668             represent a placeholder in the result set. Pass the
   669             returned bookmark on the subsequent iteration of the query to
   670             retrieve the next set of results.
   671  
   672  The following is a peer command to call QueryAssetsWithPagination with a
   673  pageSize of ``3``. Notice this time, the query includes the bookmark returned
   674  from the previous query.
   675  
   676  :guilabel:`Try it yourself`
   677  
   678  .. code:: bash
   679  
   680    peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssetsWithPagination", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}","g1AAAABLeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqz5yYWJeWkGoOkOWDSOSANIFk2iCyIyVySn5uVBQAGEhRz","3"]}'
   681  
   682  The following response is received (carriage returns added for clarity).  The
   683  last two records are retrieved:
   684  
   685  .. code:: bash
   686  
   687    [{"Key":"asset4", "Record":{"color":"purple","docType":"asset","name":"asset4","size":"7","owner":"tom","appraisedValue":20}},
   688     {"Key":"asset5", "Record":{"color":"blue","docType":"asset","name":"asset5","size":"8","owner":"tom","appraisedValue":40}}]
   689    [{"ResponseMetadata":{"RecordsCount":"2",
   690    "Bookmark":"g1AAAABLeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqz5yYWJeWkmoKkOWDSOSANIFk2iCyIyVySn5uVBQAGYhR1"}}]
   691  
   692  The final command is a peer command to call QueryAssetsWithPagination with
   693  a pageSize of ``3`` and with the bookmark from the previous query.
   694  
   695  :guilabel:`Try it yourself`
   696  
   697  .. code:: bash
   698  
   699      peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssetsWithPagination", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}","g1AAAABLeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqz5yYWJeWkmoKkOWDSOSANIFk2iCyIyVySn5uVBQAGYhR1","3"]}'
   700  
   701  The following response is received (carriage returns added for clarity).
   702  No records are returned, indicating that all pages have been retrieved:
   703  
   704  .. code:: bash
   705  
   706      []
   707      [{"ResponseMetadata":{"RecordsCount":"0",
   708      "Bookmark":"g1AAAABLeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqz5yYWJeWkmoKkOWDSOSANIFk2iCyIyVySn5uVBQAGYhR1"}}]
   709  
   710  For an example of how a client application can iterate over
   711  the result sets using pagination, search for the ``getQueryResultForQueryStringWithPagination``
   712  function in the `Asset transfer ledger queries sample <https://github.com/hyperledger/fabric-samples/blob/{BRANCH}/asset-transfer-ledger-queries/chaincode-go/asset_transfer_ledger_chaincode.go>`__.
   713  
   714  .. _cdb-update-index:
   715  
   716  Update an Index
   717  ~~~~~~~~~~~~~~~
   718  
   719  It may be necessary to update an index over time. The same index may exist in
   720  subsequent versions of the chaincode that gets installed. In order for an index
   721  to be updated, the original index definition must have included the design
   722  document ``ddoc`` attribute and an index name. To update an index definition,
   723  use the same index name but alter the index definition. Simply edit the index
   724  JSON file and add or remove fields from the index. Fabric only supports the
   725  index type JSON. Changing the index type is not supported. The updated
   726  index definition gets redeployed to the peer’s state database when the chaincode
   727  definition is committed to the channel. Changes to the index name or ``ddoc``
   728  attributes will result in a new index being created and the original index remains
   729  unchanged in CouchDB until it is removed.
   730  
   731  .. note:: If the state database has a significant volume of data, it will take
   732            some time for the index to be re-built, during which time chaincode
   733            invokes that issue queries may fail or timeout.
   734  
   735  Iterating on your index definition
   736  ----------------------------------
   737  
   738  If you have access to your peer's CouchDB state database in a development
   739  environment, you can iteratively test various indexes in support of
   740  your chaincode queries. Any changes to chaincode though would require
   741  redeployment. Use the `CouchDB Fauxton interface <http://docs.couchdb.org/en/latest/fauxton/index.html>`__ or a command
   742  line curl utility to create and update indexes.
   743  
   744  .. note:: The Fauxton interface is a web UI for the creation, update, and
   745            deployment of indexes to CouchDB. If you want to try out this
   746            interface, there is an example of the format of the Fauxton version
   747            of the index in Assets sample. If you have deployed the test network
   748            with CouchDB, the Fauxton interface can be loaded by opening a browser
   749            and navigating to ``http://localhost:5984/_utils``.
   750  
   751  Alternatively, if you prefer not use the Fauxton UI, the following is an example
   752  of a curl command which can be used to create the index on the database
   753  ``mychannel_ledger``:
   754  
   755  .. code:: bash
   756  
   757    // Index for docType, owner.
   758    // Example curl command line to define index in the CouchDB channel_chaincode database
   759     curl -i -X POST -H "Content-Type: application/json" -d
   760            "{\"index\":{\"fields\":[\"docType\",\"owner\"]},
   761              \"name\":\"indexOwner\",
   762              \"ddoc\":\"indexOwnerDoc\",
   763              \"type\":\"json\"}" http://hostname:port/mychannel_ledger/_index
   764  
   765  .. note:: If you are using the test network configured with CouchDB, replace
   766      hostname:port with ``localhost:5984``.
   767  
   768  .. _cdb-delete-index:
   769  
   770  Delete an Index
   771  ~~~~~~~~~~~~~~~
   772  
   773  Index deletion is not managed by Fabric tooling. If you need to delete an index,
   774  manually issue a curl command against the database or delete it using the
   775  Fauxton interface.
   776  
   777  The format of the curl command to delete an index would be:
   778  
   779  .. code:: bash
   780  
   781     curl -X DELETE http://localhost:5984/{database_name}/_index/{design_doc}/json/{index_name} -H  "accept: */*" -H  "Host: localhost:5984"
   782  
   783  
   784  To delete the index used in this tutorial, the curl command would be:
   785  
   786  .. code:: bash
   787  
   788     curl -X DELETE http://localhost:5984/mychannel_ledger/_index/indexOwnerDoc/json/indexOwner -H  "accept: */*" -H  "Host: localhost:5984"
   789  
   790  
   791  
   792  .. Licensed under Creative Commons Attribution 4.0 International License
   793     https://creativecommons.org/licenses/by/4.0/