github.com/cosmos/cosmos-sdk@v0.50.10/docs/architecture/adr-044-protobuf-updates-guidelines.md (about)

     1  # ADR 044: Guidelines for Updating Protobuf Definitions
     2  
     3  ## Changelog
     4  
     5  * 28.06.2021: Initial Draft
     6  * 02.12.2021: Add `Since:` comment for new fields
     7  * 21.07.2022: Remove the rule of no new `Msg` in the same proto version.
     8  
     9  ## Status
    10  
    11  Draft
    12  
    13  ## Abstract
    14  
    15  This ADR provides guidelines and recommended practices when updating Protobuf definitions. These guidelines are targeting module developers.
    16  
    17  ## Context
    18  
    19  The Cosmos SDK maintains a set of [Protobuf definitions](https://github.com/cosmos/cosmos-sdk/tree/main/proto/cosmos). It is important to correctly design Protobuf definitions to avoid any breaking changes within the same version. The reasons are to not break tooling (including indexers and explorers), wallets and other third-party integrations.
    20  
    21  When making changes to these Protobuf definitions, the Cosmos SDK currently only follows [Buf's](https://docs.buf.build/) recommendations. We noticed however that Buf's recommendations might still result in breaking changes in the SDK in some cases. For example:
    22  
    23  * Adding fields to `Msg`s. Adding fields is a not a Protobuf spec-breaking operation. However, when adding new fields to `Msg`s, the unknown field rejection will throw an error when sending the new `Msg` to an older node.
    24  * Marking fields as `reserved`. Protobuf proposes the `reserved` keyword for removing fields without the need to bump the package version. However, by doing so, client backwards compatibility is broken as Protobuf doesn't generate anything for `reserved` fields. See [#9446](https://github.com/cosmos/cosmos-sdk/issues/9446) for more details on this issue.
    25  
    26  Moreover, module developers often face other questions around Protobuf definitions such as "Can I rename a field?" or "Can I deprecate a field?" This ADR aims to answer all these questions by providing clear guidelines about allowed updates for Protobuf definitions.
    27  
    28  ## Decision
    29  
    30  We decide to keep [Buf's](https://docs.buf.build/) recommendations with the following exceptions:
    31  
    32  * `UNARY_RPC`: the Cosmos SDK currently does not support streaming RPCs.
    33  * `COMMENT_FIELD`: the Cosmos SDK allows fields with no comments.
    34  * `SERVICE_SUFFIX`: we use the `Query` and `Msg` service naming convention, which doesn't use the `-Service` suffix.
    35  * `PACKAGE_VERSION_SUFFIX`: some packages, such as `cosmos.crypto.ed25519`, don't use a version suffix.
    36  * `RPC_REQUEST_STANDARD_NAME`: Requests for the `Msg` service don't have the `-Request` suffix to keep backwards compatibility.
    37  
    38  On top of Buf's recommendations we add the following guidelines that are specific to the Cosmos SDK.
    39  
    40  ### Updating Protobuf Definition Without Bumping Version
    41  
    42  #### 1. Module developers MAY add new Protobuf definitions
    43  
    44  Module developers MAY add new `message`s, new `Service`s, new `rpc` endpoints, and new fields to existing messages. This recommendation follows the Protobuf specification, but is added in this document for clarity, as the SDK requires one additional change.
    45  
    46  The SDK requires the Protobuf comment of the new addition to contain one line with the following format:
    47  
    48  ```protobuf
    49  // Since: cosmos-sdk <version>{, <version>...}
    50  ```
    51  
    52  Where each `version` denotes a minor ("0.45") or patch ("0.44.5") version from which the field is available. This will greatly help client libraries, who can optionally use reflection or custom code generation to show/hide these fields depending on the targetted node version.
    53  
    54  As examples, the following comments are valid:
    55  
    56  ```protobuf
    57  // Since: cosmos-sdk 0.44
    58  
    59  // Since: cosmos-sdk 0.42.11, 0.44.5
    60  ```
    61  
    62  and the following ones are NOT valid:
    63  
    64  ```protobuf
    65  // Since cosmos-sdk v0.44
    66  
    67  // since: cosmos-sdk 0.44
    68  
    69  // Since: cosmos-sdk 0.42.11 0.44.5
    70  
    71  // Since: Cosmos SDK 0.42.11, 0.44.5
    72  ```
    73  
    74  #### 2. Fields MAY be marked as `deprecated`, and nodes MAY implement a protocol-breaking change for handling these fields
    75  
    76  Protobuf supports the [`deprecated` field option](https://developers.google.com/protocol-buffers/docs/proto#options), and this option MAY be used on any field, including `Msg` fields. If a node handles a Protobuf message with a non-empty deprecated field, the node MAY change its behavior upon processing it, even in a protocol-breaking way. When possible, the node MUST handle backwards compatibility without breaking the consensus (unless we increment the proto version).
    77  
    78  As an example, the Cosmos SDK v0.42 to v0.43 update contained two Protobuf-breaking changes, listed below. Instead of bumping the package versions from `v1beta1` to `v1`, the SDK team decided to follow this guideline, by reverting the breaking changes, marking those changes as deprecated, and modifying the node implementation when processing messages with deprecated fields. More specifically:
    79  
    80  * The Cosmos SDK recently removed support for [time-based software upgrades](https://github.com/cosmos/cosmos-sdk/pull/8849). As such, the `time` field has been marked as deprecated in `cosmos.upgrade.v1beta1.Plan`. Moreover, the node will reject any proposal containing an upgrade Plan whose `time` field is non-empty.
    81  * The Cosmos SDK now supports [governance split votes](./adr-037-gov-split-vote.md). When querying for votes, the returned `cosmos.gov.v1beta1.Vote` message has its `option` field (used for 1 vote option) deprecated in favor of its `options` field (allowing multiple vote options). Whenever possible, the SDK still populates the deprecated `option` field, that is, if and only if the `len(options) == 1` and `options[0].Weight == 1.0`.
    82  
    83  #### 3. Fields MUST NOT be renamed
    84  
    85  Whereas the official Protobuf recommendations do not prohibit renaming fields, as it does not break the Protobuf binary representation, the SDK explicitly forbids renaming fields in Protobuf structs. The main reason for this choice is to avoid introducing breaking changes for clients, which often rely on hard-coded fields from generated types. Moreover, renaming fields will lead to client-breaking JSON representations of Protobuf definitions, used in REST endpoints and in the CLI.
    86  
    87  ### Incrementing Protobuf Package Version
    88  
    89  TODO, needs architecture review. Some topics:
    90  
    91  * Bumping versions frequency
    92  * When bumping versions, should the Cosmos SDK support both versions?
    93      * i.e. v1beta1 -> v1, should we have two folders in the Cosmos SDK, and handlers for both versions?
    94  * mention ADR-023 Protobuf naming
    95  
    96  ## Consequences
    97  
    98  > This section describes the resulting context, after applying the decision. All consequences should be listed here, not just the "positive" ones. A particular decision may have positive, negative, and neutral consequences, but all of them affect the team and project in the future.
    99  
   100  ### Backwards Compatibility
   101  
   102  > All ADRs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The ADR must explain how the author proposes to deal with these incompatibilities. ADR submissions without a sufficient backwards compatibility treatise may be rejected outright.
   103  
   104  ### Positive
   105  
   106  * less pain to tool developers
   107  * more compatibility in the ecosystem
   108  * ...
   109  
   110  ### Negative
   111  
   112  {negative consequences}
   113  
   114  ### Neutral
   115  
   116  * more rigor in Protobuf review
   117  
   118  ## Further Discussions
   119  
   120  This ADR is still in the DRAFT stage, and the "Incrementing Protobuf Package Version" will be filled in once we make a decision on how to correctly do it.
   121  
   122  ## Test Cases [optional]
   123  
   124  Test cases for an implementation are mandatory for ADRs that are affecting consensus changes. Other ADRs can choose to include links to test cases if applicable.
   125  
   126  ## References
   127  
   128  * [#9445](https://github.com/cosmos/cosmos-sdk/issues/9445) Release proto definitions v1
   129  * [#9446](https://github.com/cosmos/cosmos-sdk/issues/9446) Address v1beta1 proto breaking changes