github.com/tickoalcantara12/micro/v3@v3.0.0-20221007104245-9d75b9bcbab9/docs/v2/design/framework/store.md (about)

     1  # Store
     2  
     3  Store is an abstraction for key-value storage.
     4  
     5  ## Overview
     6  
     7  For the majority of time microservices are considered stateless and storage is offloaded to a database. 
     8  Considering that we provide a framework, storage and distributed storage needs to be a core concern.
     9  Micro provides a Store interface for key-value storage and a micro store service as the RPC layer 
    10  abstraction.
    11  
    12  ## Design
    13  
    14  The interface is:
    15  
    16  ```go
    17  // Store is the interface for data storage
    18  type Store interface {
    19  	Init(...Option)                          error
    20  	Options()                                Options
    21  	Read(key string, opts ...ReadOption)     ([]*Record, error)
    22  	Write(*Record, opts ...WriteOption)      error
    23  	Delete(key string, opts ...DeleteOption) error
    24  	List(opts ...ListOption)                 ([]string, error)
    25  	String()                                 string
    26  }
    27  
    28  // Record is the data stored by the store
    29  type Record struct {
    30  	// The key for the record
    31  	Key    string
    32  	// The encoded database
    33  	Value  []byte
    34  	// Associated metadata
    35  	Metadata map[string]interface{}
    36  	// Time at which the record expires
    37  	Expiry time.Duration
    38  }
    39  ```
    40  
    41  ### Init
    42  
    43  `Init()` initialises the store. It must any required setup on the backing storage
    44  implementation and check that it is ready for use, returning any errors.
    45  `Init()` **must** be called successfully before the store is used.
    46  
    47  #### Option
    48  
    49  ```go
    50  type Options struct {
    51  	Nodes     []string
    52  	Namespace string
    53  	Prefix    string
    54  	Suffix    string
    55  	Context   context.Context
    56  }
    57  
    58  type Option func(o *Options)
    59  ```
    60  
    61  `Nodes` contains the addresses or other connection information of the backing
    62  storage. For example, an etcd implementation would contain the nodes of the
    63  cluster. A SQL implementation could contain one or more connection strings.
    64  
    65  `Namespace` allows multiple isolated stores to be kept in one backend, if supported.
    66  For example, multiple tables in a SQL store.
    67  
    68  `Prefix` and `Suffix` set a global prefix/suffix on all keys.
    69  
    70  `Context` should contain all implementation specific options, using
    71  [`context.WithValue`](https://pkg.go.dev/context?tab=doc#WithValue) as a KV store.
    72  
    73  ### Options
    74  
    75  `Options()` returns the current options
    76  
    77  ### String
    78  
    79  `String()` returns the name of the implementation, e.g. `memory`. Useful for logging purposes.
    80  
    81  ### Read
    82  
    83  `Read()` takes a single key name and optional `ReadOption`s. It returns matching `*Record`s or an error.
    84  
    85  #### ReadOption
    86  
    87  ```go
    88  type ReadOptions struct {
    89  	Prefix bool
    90  	Suffix bool
    91  }
    92  
    93  type ReadOption func(r *ReadOptions)
    94  ```
    95  
    96  `Prefix` and `Suffix` return all keys with matching prefix or suffix.
    97  
    98  ### Write
    99  
   100  `Write()` writes a record to the store, and returns an error if the record was not written.
   101  
   102  #### WriteOption
   103  
   104  ```go
   105  type WriteOptions struct {
   106    Expiry time.Time
   107    TTL    time.Duration
   108  }
   109  
   110  type WriteOption func(w *WriteOptions)
   111  ```
   112  
   113  If Expiry or TTL are passed as options, overwrite the record's expiry before writing.
   114  
   115  ### Delete
   116  
   117  `Delete()` removes the record with the corresponding key from the store.
   118  
   119  No options are defined yet.
   120  
   121  ### List
   122  
   123  `List()` returns any keys that match, or an empty list with no error if none matched.
   124  
   125  #### ListOption
   126  
   127  ```go
   128  type ListOptions struct {
   129    Prefix string
   130    Suffix string
   131  }
   132  
   133  type ListOption func(l *ListOptions)
   134  ```
   135  
   136  If Prefix and / or Suffix are set, the returned list is limited to keys that have the prefix or suffix.
   137  
   138  ## Caching
   139  
   140  Caching is a layer to be built on top of the store in store/cache much like registry/cache.
   141  
   142  Caching needs to take into consideration cache coherence and invalidation
   143  - https://en.m.wikipedia.org/wiki/Cache_coherence
   144  - https://en.m.wikipedia.org/wiki/Cache_invalidation
   145  
   146  ## Indexing
   147  
   148  The store supports indexing via metadata field values. These values can be scanned to quickly access 
   149  records that otherwise might of a larger size and more costly to decode. 
   150  
   151  In the case of a system like cockroachdb we store metadata as a separate field that uses JSONB format 
   152  so that it can be queried. See reference here https://www.cockroachlabs.com/docs/stable/jsonb.html.
   153  
   154  Storage and querying would be of the form
   155  
   156  ```go
   157  
   158  type Fields map[string]string
   159  
   160  store.Write(&Record{
   161  	Key: "user:1",
   162  	Value: []byte(...),
   163  	Metadata: map[string]interface{
   164  		"name": "John",
   165  		"email": "john@example.com",
   166  	},
   167  }
   168  
   169  
   170  wrote.Read("", store.ReadWhere(&store.Fields{
   171  	"name": "john",
   172  })
   173  ```