github.com/ari-anchor/sei-tendermint@v0.0.0-20230519144642-dc826b7b56bb/docs/architecture/adr-060-go-api-stability.md (about) 1 # ADR 060: Go API Stability 2 3 ## Changelog 4 5 - 2020-09-08: Initial version. (@erikgrinaker) 6 7 - 2020-09-09: Tweak accepted changes, add initial public API packages, add consequences. (@erikgrinaker) 8 9 - 2020-09-17: Clarify initial public API. (@erikgrinaker) 10 11 ## Context 12 13 With the release of Tendermint 1.0 we will adopt [semantic versioning](https://semver.org). One major implication is a guarantee that we will not make backwards-incompatible changes until Tendermint 2.0 (except in pre-release versions). In order to provide this guarantee for our Go API, we must clearly define which of our APIs are public, and what changes are considered backwards-compatible. 14 15 Currently, we list packages that we consider public in our [README](https://github.com/tendermint/tendermint#versioning), but since we are still at version 0.x we do not provide any backwards compatiblity guarantees at all. 16 17 ### Glossary 18 19 * **External project:** a different Git/VCS repository or code base. 20 21 * **External package:** a different Go package, can be a child or sibling package in the same project. 22 23 * **Internal code:** code not intended for use in external projects. 24 25 * **Internal directory:** code under `internal/` which cannot be imported in external projects. 26 27 * **Exported:** a Go identifier starting with an uppercase letter, which can therefore be accessed by an external package. 28 29 * **Private:** a Go identifier starting with a lowercase letter, which therefore cannot be accessed by an external package unless via an exported field, variable, or function/method return value. 30 31 * **Public API:** any Go identifier that can be imported or accessed by an external project, except test code in `_test.go` files. 32 33 * **Private API:** any Go identifier that is not accessible via a public API, including all code in the internal directory. 34 35 ## Alternative Approaches 36 37 - Split all public APIs out to separate Go modules in separate Git repositories, and consider all Tendermint code internal and not subject to API backwards compatibility at all. This was rejected, since it has been attempted by the Tendermint project earlier, resulting in too much dependency management overhead. 38 39 - Simply document which APIs are public and which are private. This is the current approach, but users should not be expected to self-enforce this, the documentation is not always up-to-date, and external projects will often end up depending on internal code anyway. 40 41 ## Decision 42 43 From Tendermint 1.0, all internal code (except private APIs) will be placed in a root-level [`internal` directory](https://golang.org/cmd/go/#hdr-Internal_Directories), which the Go compiler will block for use by external projects. All exported items outside of the `internal` directory are considered a public API and subject to backwards compatibility guarantees, except files ending in `_test.go`. 44 45 The `crypto` package may be split out to a separate module in a separate repo. This is the main general-purpose package used by external projects, and is the only Tendermint dependency in e.g. IAVL which can cause some problems for projects depending on both IAVL and Tendermint. This will be decided after further discussion. 46 47 The `tm-db` package will remain a separate module in a separate repo. The `crypto` package may possibly be split out, pending further discussion, as this is the main general-purpose package used by other projects. 48 49 ## Detailed Design 50 51 ### Public API 52 53 When preparing our public API for 1.0, we should keep these principles in mind: 54 55 - Limit the number of public APIs that we start out with - we can always add new APIs later, but we can't change or remove APIs once they're made public. 56 57 - Before an API is made public, do a thorough review of the API to make sure it covers any future needs, can accomodate expected changes, and follows good API design practices. 58 59 The following is the minimum set of public APIs that will be included in 1.0, in some form: 60 61 - `abci` 62 - packages used for constructing nodes `config`, `libs/log`, and `version` 63 - Client APIs, i.e. `rpc/client`, `light`, and `privval`. 64 - `crypto` (possibly as a separate repo) 65 66 We may offer additional APIs as well, following further discussions internally and with other stakeholders. However, public APIs for providing custom components (e.g. reactors and mempools) are not planned for 1.0, but may be added in a later 1.x version if this is something we want to offer. 67 68 For comparison, the following are the number of Tendermint imports in the Cosmos SDK (excluding tests), which should be mostly satisfied by the planned APIs. 69 70 ``` 71 1 github.com/tendermint/tendermint/abci/server 72 73 github.com/tendermint/tendermint/abci/types 73 2 github.com/tendermint/tendermint/cmd/tendermint/commands 74 7 github.com/tendermint/tendermint/config 75 68 github.com/tendermint/tendermint/crypto 76 1 github.com/tendermint/tendermint/crypto/armor 77 10 github.com/tendermint/tendermint/crypto/ed25519 78 2 github.com/tendermint/tendermint/crypto/encoding 79 3 github.com/tendermint/tendermint/crypto/merkle 80 3 github.com/tendermint/tendermint/crypto/sr25519 81 8 github.com/tendermint/tendermint/crypto/tmhash 82 1 github.com/tendermint/tendermint/crypto/xsalsa20symmetric 83 11 github.com/tendermint/tendermint/libs/bytes 84 2 github.com/tendermint/tendermint/libs/bytes.HexBytes 85 15 github.com/tendermint/tendermint/libs/cli 86 2 github.com/tendermint/tendermint/libs/cli/flags 87 2 github.com/tendermint/tendermint/libs/json 88 30 github.com/tendermint/tendermint/libs/log 89 1 github.com/tendermint/tendermint/libs/math 90 11 github.com/tendermint/tendermint/libs/os 91 4 github.com/tendermint/tendermint/libs/rand 92 1 github.com/tendermint/tendermint/libs/strings 93 5 github.com/tendermint/tendermint/light 94 1 github.com/tendermint/tendermint/internal/mempool 95 3 github.com/tendermint/tendermint/node 96 5 github.com/tendermint/tendermint/internal/p2p 97 4 github.com/tendermint/tendermint/privval 98 10 github.com/tendermint/tendermint/proto/tendermint/crypto 99 1 github.com/tendermint/tendermint/proto/tendermint/libs/bits 100 24 github.com/tendermint/tendermint/proto/tendermint/types 101 3 github.com/tendermint/tendermint/proto/tendermint/version 102 2 github.com/tendermint/tendermint/proxy 103 3 github.com/tendermint/tendermint/rpc/client 104 1 github.com/tendermint/tendermint/rpc/client/http 105 2 github.com/tendermint/tendermint/rpc/client/local 106 3 github.com/tendermint/tendermint/rpc/core/types 107 1 github.com/tendermint/tendermint/rpc/jsonrpc/server 108 33 github.com/tendermint/tendermint/types 109 2 github.com/tendermint/tendermint/types/time 110 1 github.com/tendermint/tendermint/version 111 ``` 112 113 ### Backwards-Compatible Changes 114 115 In Go, [almost all API changes are backwards-incompatible](https://blog.golang.org/module-compatibility) and thus exported items in public APIs generally cannot be changed until Tendermint 2.0. The only backwards-compatible changes we can make to public APIs are: 116 117 - Adding a package. 118 119 - Adding a new identifier to the package scope (e.g. const, var, func, struct, interface, etc.). 120 121 - Adding a new method to a struct. 122 123 - Adding a new field to a struct, if the zero-value preserves any old behavior. 124 125 - Changing the order of fields in a struct. 126 127 - Adding a variadic parameter to a named function or struct method, if the function type itself is not assignable in any public APIs (e.g. a callback). 128 129 - Adding a new method to an interface, or a variadic parameter to an interface method, _if the interface already has a private method_ (which prevents external packages from implementing it). 130 131 - Widening a numeric type as long as it is a named type (e.g. `type Number int32` can change to `int64`, but not `int8` or `uint32`). 132 133 Note that public APIs can expose private types (e.g. via an exported variable, field, or function/method return value), in which case the exported fields and methods on these private types are also part of the public API and covered by its backwards compatiblity guarantees. In general, private types should never be accessible via public APIs unless wrapped in an exported interface. 134 135 Also note that if we accept, return, export, or embed types from a dependency, we assume the backwards compatibility responsibility for that dependency, and must make sure any dependency upgrades comply with the above constraints. 136 137 We should run CI linters for minor version branches to enforce this, e.g. [apidiff](https://go.googlesource.com/exp/+/refs/heads/master/apidiff/README.md), [breakcheck](https://github.com/gbbr/breakcheck), and [apicombat](https://github.com/bradleyfalzon/apicompat). 138 139 #### Accepted Breakage 140 141 The above changes can still break programs in a few ways - these are _not_ considered backwards-incompatible changes, and users are advised to avoid this usage: 142 143 - If a program uses unkeyed struct literals (e.g. `Foo{"bar", "baz"}`) and we add fields or change the field order, the program will no longer compile or may have logic errors. 144 145 - If a program embeds two structs in a struct, and we add a new field or method to an embedded Tendermint struct which also exists in the other embedded struct, the program will no longer compile. 146 147 - If a program compares two structs (e.g. with `==`), and we add a new field of an incomparable type (slice, map, func, or struct that contains these) to a Tendermint struct which is compared, the program will no longer compile. 148 149 - If a program assigns a Tendermint function to an identifier, and we add a variadic parameter to the function signature, the program will no longer compile. 150 151 ### Strategies for API Evolution 152 153 The API guarantees above can be fairly constraining, but are unavoidable given the Go language design. The following tricks can be employed where appropriate to allow us to make changes to the API: 154 155 - We can add a new function or method with a different name that takes additional parameters, and have the old function call the new one. 156 157 - Functions and methods can take an options struct instead of separate parameters, to allow adding new options - this is particularly suitable for functions that take many parameters and are expected to be extended, and especially for interfaces where we cannot add new methods with different parameters at all. 158 159 - Interfaces can include a private method, e.g. `interface { private() }`, to make them unimplementable by external packages and thus allow us to add new methods to the interface without breaking other programs. Of course, this can't be used for interfaces that should be implementable externally. 160 161 - We can use [interface upgrades](https://avtok.com/2014/11/05/interface-upgrades.html) to allow implementers of an existing interface to also implement a new interface, as long as the old interface can still be used - e.g. the new interface `BetterReader` may have a method `ReadBetter()`, and a function that takes a `Reader` interface as an input can check if the implementer also implements `BetterReader` and in that case call `ReadBetter()` instead of `Read()`. 162 163 ## Status 164 165 Accepted 166 167 ## Consequences 168 169 ### Positive 170 171 - Users can safely upgrade with less fear of applications breaking, and know whether an upgrade only includes bug fixes or also functional enhancements 172 173 - External developers have a predictable and well-defined API to build on that will be supported for some time 174 175 - Less synchronization between teams, since there is a clearer contract and timeline for changes and they happen less frequently 176 177 - More documentation will remain accurate, since it's not chasing a moving target 178 179 - Less time will be spent on code churn and more time spent on functional improvements, both for the community and for our teams 180 181 ### Negative 182 183 - Many improvements, changes, and bug fixes will have to be postponed until the next major version, possibly for a year or more 184 185 - The pace of development will slow down, since we must work within the existing API constraints, and spend more time planning public APIs 186 187 - External developers may lose access to some currently exported APIs and functionality 188 189 ## References 190 191 - [#4451: Place internal APIs under internal package](https://github.com/tendermint/tendermint/issues/4451) 192 193 - [On Pluggability](https://docs.google.com/document/d/1G08LnwSyb6BAuCVSMF3EKn47CGdhZ5wPZYJQr4-bw58/edit?ts=5f609f11)