github.com/m3db/m3@v1.5.0/src/dbnode/client/README.md (about)

     1  # `client` Implementation details
     2  
     3  This file lists details and design choices about the implementation of the `client` package.
     4  
     5  
     6  ## FetchTagged/FetchTaggedIDs
     7  First point to note, `FetchTagged`/`FetchTaggedIDs` share a majority of code, so the following applies to both.
     8  The document points out where they differ as applicable.
     9  
    10  First a quick glossary of the types used in this flow:
    11  - `session`: the implementation backing the `client.Session` interface, it's topology aware and does
    12  the fan-out/coordination of the request/response lifecycle.
    13  - `op`: generic interface used by the `session` as a unit of work. It has a `completionFn` specified based
    14  on the backing implementation. It is executed by the `hostQueue`.
    15  - `completionFn`: the callback specified on each `op` to return a response to the caller.
    16  - `hostQueue`: work queue maintained per host by the `session`. It does the execution of a rpc request
    17  asynchronously, and executing any callbacks specified on response/timeout.
    18  - `fetchState`: struct corresponding to a single attempt of a `FetchTagged` rpc request.
    19  - `fetchTaggedResultsAccumulator`: struct used by a `fetchState` for accumulating responses and converting to
    20  the appropriate response types eventually.
    21  - `fetchTaggedShardConsistencyResult`: struct used by the `fetchTaggedResultsAccumulator` for response
    22  consistency calculations.
    23  - `fetchTaggedOp`: implementation of `op` for the `FetchTagged` method calls.
    24  
    25  Easiest way to understand this is to follow the control flow for the execution of a FetchTagged method.
    26  
    27  Sequence of steps:
    28  1. A user of the API calls `session.FetchTagged(...)`
    29  2. `session` validates the request, and converts to the rpc type; terminating early if this fails.
    30  3. Once the request is validated, the `session` retrieves a `fetchState`, and `fetchTaggedOp`
    31  from their respective pools. We create a single `fetchState` and `fetchTaggedOp` per attempt.
    32  4. The `fetchTaggedOp` is enqueued into each known host's `hostQueue`.
    33  5. The `session` then calls `fetchState.Wait()`, which is backed by a condition variable, while
    34  waiting for responses to come back. Note: the calling go-routine will keep waiting until sufficient
    35  responses have been received to fullfil the consistency requirement of the request, or if we receive
    36  responses from all hosts and are still unable to meet the constraint.
    37  6. In the background, each `hostQueue` is executing the appropriate rpc and calling the `completionFn`
    38  with success/error/timeout.
    39  7. For each execution of the `completionFn`, the `fetchState` passes the response to the
    40  `fetchTaggedResultsAccumulator` which does the consistency calculation and updates its internal state.
    41  The `fetchTaggedResultsAccumulator` returns whether it's hit a terminating condition to to the `fetchState`,
    42  which acts upon it and if it has, calls `Signal()` to indicate to the original go-routine waiting on the
    43  condition variable that a response is available.
    44  
    45  ### Nuances about lifecycle
    46  - The `fetchTaggedOp` retains a reference to the `fetchState`, so the `fetchState` has to remain alive
    47  until all hosts have returned a response/timed-out.
    48  - The `fetchState` created by the session in steps during the execution of a `FetchTagged()` method can
    49  remain alive even after the response. This is because we can return success even if we haven't received
    50  responses from all hosts depending on the consistency requirement. As a result, the final caller to
    51  `fetchState.decRef()` actually cleans it up and returns it to the pool. This will be a hostQueue in case
    52  of early success, or the go-routine calling the the `FetchTagged()` in case of error.