github.com/cockroachdb/cockroach@v20.2.0-alpha.1+incompatible/docs/RFCS/20151207_grpc.md (about)

     1  - Feature Name: grpc
     2  - Status: completed
     3  - Start Date: 2015-12-07
     4  - RFC PR: [#3352](https://github.com/cockroachdb/cockroach/pull/3352)
     5  - Cockroach Issue: [#2381](https://github.com/cockroachdb/cockroach/pull/2381),
     6                     [#3013](https://github.com/cockroachdb/cockroach/issues/3013),
     7                     [#3421](https://github.com/cockroachdb/cockroach/pull/3421)
     8  
     9  # Summary
    10  
    11  Replace the custom protocol in `cockroach/rpc/codec` with GRPC. This
    12  will affect both internal communication and potential client-facing
    13  APIs (assuming we offer other public APIs besides `pgwire`). GRPC was
    14  previously discussed in #2381 for client-facing APIs; this RFC extends
    15  the proposal to internal RPCs as well.
    16  
    17  # Motivation
    18  
    19  The primary motivation is to minimize the impact that raft snapshots
    20  have on other RPCs (#3013). Our RPC codec transmits each request as a
    21  single chunk on the network, which can block other requests for a
    22  significant period of time (leading to more severe consequences if
    23  raft heartbeats or range lease operations are blocked for too long).
    24  Since GRPC is based on HTTP/2, it has built-in support for
    25  multiplexing large messages.
    26  
    27  Secondary benefits of GRPC include support for bidirectional streaming
    28  channels, which are a good fit for raft and gossip messages that do
    29  not fit the "one request, one response" pattern, and may be useful for
    30  future distributed SQL workloads. There are also benefits to adopting
    31  a more widely-used protocol instead of our current custom one.
    32  
    33  # Detailed design
    34  
    35  Remove all use of `cockroach/rpc/codec` and `net/rpc`, replacing them
    36  with GRPC. `cockroach/rpc` be removed, or may become a thin wrapper
    37  around GRPC. In either case, interfaces will need to be extended to
    38  allow streaming; it probably makes more sense to pass GRPC objects
    39  around everywhere than to attempt to cover them with our own
    40  abstraction.
    41  
    42  # Drawbacks
    43  
    44  GRPC is currently slower than our RPC codec ([benchmark results](
    45  https://github.com/cockroachdb/rpc-bench)). It
    46  [looks like](https://github.com/grpc/grpc-go/issues/89) the Go
    47  implementation of GRPC has not yet seen significant performance work.
    48  It should be possible to improve performance, but GRPC is more complex
    49  and it may be difficult to match the performance of our custom codec.
    50  
    51  Investigation so far indicates that the performance difference is
    52  primarily due to the fact that GRPC performs two `Write` syscalls per
    53  server response (one for the headers and one for the body) while our
    54  own codec does one. GRPC also spends a bit more time in garbage
    55  collection than we do.
    56  
    57  The `grpc-go` server does not currently support serving other HTTP
    58  requests on the same port as GRPC (grpc/grpc-go#75). We would need to
    59  either fix this upstream or listen on two ports (or hack around it: an
    60  HTTP handshake with an `Upgrade` header like our current RPC codec is
    61  possible, but would make us incompatible with standard GRPC).
    62  
    63  # Alternatives
    64  
    65  We could improve the streaming/multiplexing capabilities of our own
    66  RPC codec, or work around the problem by manually splitting messages
    67  at a higher level (#3421 contains the beginning of this work).
    68  
    69  Using protobufs over plain HTTP/2 is in some ways simpler than using
    70  GRPC (in that it removes a layer), and gives us some of the key
    71  benefits including better multiplexing for snapshots. This is likely
    72  to have similar performance to GRPC. For internal use GRPC should be
    73  better than plain HTTP, but for external APIs plain HTTP will be
    74  easier to use from languages where no high-quality GRPC implementation
    75  exists.
    76  
    77  # Unresolved questions