github.com/cosmos/cosmos-sdk@v0.50.10/docs/architecture/adr-055-orm.md (about)

     1  # ADR 055: ORM
     2  
     3  ## Changelog
     4  
     5  * 2022-04-27: First draft
     6  
     7  ## Status
     8  
     9  ACCEPTED Implemented
    10  
    11  ## Abstract
    12  
    13  In order to make it easier for developers to build Cosmos SDK modules and for clients to query, index and verify proofs
    14  against state data, we have implemented an ORM (object-relational mapping) layer for the Cosmos SDK.
    15  
    16  ## Context
    17  
    18  Historically modules in the Cosmos SDK have always used the key-value store directly and created various handwritten
    19  functions for managing key format as well as constructing secondary indexes. This consumes a significant amount of
    20  time when building a module and is error-prone. Because key formats are non-standard, sometimes poorly documented,
    21  and subject to change, it is hard for clients to generically index, query and verify merkle proofs against state data.
    22  
    23  The known first instance of an "ORM" in the Cosmos ecosystem was in [weave](https://github.com/iov-one/weave/tree/master/orm).
    24  A later version was built for [regen-ledger](https://github.com/regen-network/regen-ledger/tree/157181f955823149e1825263a317ad8e16096da4/orm) for
    25  use in the group module and later [ported to the SDK](https://github.com/cosmos/cosmos-sdk/tree/35d3312c3be306591fcba39892223f1244c8d108/x/group/internal/orm)
    26  just for that purpose.
    27  
    28  While these earlier designs made it significantly easier to write state machines, they still required a lot of manual
    29  configuration, didn't expose state format directly to clients, and were limited in their support of different types
    30  of index keys, composite keys, and range queries.
    31  
    32  Discussions about the design continued in https://github.com/cosmos/cosmos-sdk/discussions/9156 and more
    33  sophisticated proofs of concept were created in https://github.com/allinbits/cosmos-sdk-poc/tree/master/runtime/orm
    34  and https://github.com/cosmos/cosmos-sdk/pull/10454.
    35  
    36  ## Decision
    37  
    38  These prior efforts culminated in the creation of the Cosmos SDK `orm` go module which uses protobuf annotations
    39  for specifying ORM table definitions. This ORM is based on the new `google.golang.org/protobuf/reflect/protoreflect`
    40  API and supports:
    41  
    42  * sorted indexes for all simple protobuf types (except `bytes`, `enum`, `float`, `double`) as well as `Timestamp` and `Duration`
    43  * unsorted `bytes` and `enum` indexes
    44  * composite primary and secondary keys
    45  * unique indexes
    46  * auto-incrementing `uint64` primary keys
    47  * complex prefix and range queries
    48  * paginated queries
    49  * complete logical decoding of KV-store data
    50  
    51  Almost all the information needed to decode state directly is specified in .proto files. Each table definition specifies
    52  an ID which is unique per .proto file and each index within a table is unique within that table. Clients then only need
    53  to know the name of a module and the prefix ORM data for a specific .proto file within that module in order to decode
    54  state data directly. This additional information will be exposed directly through app configs which will be explained
    55  in a future ADR related to app wiring.
    56  
    57  The ORM makes optimizations around storage space by not repeating values in the primary key in the key value
    58  when storing primary key records. For example, if the object `{"a":0,"b":1}` has the primary key `a`, it will
    59  be stored in the key value store as `Key: '0', Value: {"b":1}` (with more efficient protobuf binary encoding).
    60  Also, the generated code from https://github.com/cosmos/cosmos-proto does optimizations around the
    61  `google.golang.org/protobuf/reflect/protoreflect` API to improve performance.
    62  
    63  A code generator is included with the ORM which creates type safe wrappers around the ORM's dynamic `Table`
    64  implementation and is the recommended way for modules to use the ORM.
    65  
    66  The ORM tests provide a simplified bank module demonstration which illustrates:
    67  * [ORM proto options](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/internal/testpb/bank.proto)
    68  * [Generated Code](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/internal/testpb/bank.cosmos_orm.go)
    69  * [Example Usage in a Module Keeper](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/model/ormdb/module_test.go)
    70  
    71  ## Consequences
    72  
    73  ### Backwards Compatibility
    74  
    75  State machine code that adopts the ORM will need migrations as the state layout is generally backwards incompatible.
    76  These state machines will also need to migrate to https://github.com/cosmos/cosmos-proto at least for state data.
    77  
    78  ### Positive
    79  
    80  * easier to build modules
    81  * easier to add secondary indexes to state
    82  * possible to write a generic indexer for ORM state
    83  * easier to write clients that do state proofs
    84  * possible to automatically write query layers rather than needing to manually implement gRPC queries
    85  
    86  ### Negative
    87  
    88  * worse performance than handwritten keys (for now). See [Further Discussions](#further-discussions)
    89  for potential improvements
    90  
    91  ### Neutral
    92  
    93  ## Further Discussions
    94  
    95  Further discussions will happen within the Cosmos SDK Framework Working Group. Current planned and ongoing work includes:
    96  
    97  * automatically generate client-facing query layer
    98  * client-side query libraries that transparently verify light client proofs
    99  * index ORM data to SQL databases
   100  * improve performance by:
   101      * optimizing existing reflection based code to avoid unnecessary gets when doing deletes & updates of simple tables
   102      * more sophisticated code generation such as making fast path reflection even faster (avoiding `switch` statements),
   103    or even fully generating code that equals handwritten performance
   104  
   105  
   106  ## References
   107  
   108  * https://github.com/iov-one/weave/tree/master/orm).
   109  * https://github.com/regen-network/regen-ledger/tree/157181f955823149e1825263a317ad8e16096da4/orm
   110  * https://github.com/cosmos/cosmos-sdk/tree/35d3312c3be306591fcba39892223f1244c8d108/x/group/internal/orm
   111  * https://github.com/cosmos/cosmos-sdk/discussions/9156
   112  * https://github.com/allinbits/cosmos-sdk-poc/tree/master/runtime/orm
   113  * https://github.com/cosmos/cosmos-sdk/pull/10454