go.mercari.io/datastore@v1.8.2/doc.go (about)

     1  /*
     2  Package datastore has an abstract representation of (AppEngine | Cloud) Datastore.
     3  
     4  repository https://github.com/mercari/datastore
     5  
     6  Let's read https://cloud.google.com/datastore/docs/ or https://cloud.google.com/appengine/docs/standard/go/datastore/ .
     7  You should also check https://godoc.org/cloud.google.com/go/datastore or https://godoc.org/google.golang.org/appengine/datastore as datastore original library.
     8  
     9  Japanese version https://github.com/mercari/datastore/blob/master/doc_ja.go
    10  
    11  
    12  Basic usage
    13  
    14  Please see https://godoc.org/go.mercari.io/datastore/clouddatastore or https://godoc.org/go.mercari.io/datastore/aedatastore .
    15  Create a Client using the FromContext function of each package.
    16  
    17  Later in this document, notes on migration from each package are summarized.
    18  Please see also there.
    19  
    20  This package is based on the newly designed Cloud Datastore API.
    21  We are introducing flatten tags that only exist in Cloud Datastore, we need to be careful when migrating from AE Datastore.
    22  Details will be described later.
    23  If you are worried, you may have a clue to the solution at https://godoc.org/go.mercari.io/datastore/clouddatastore .
    24  
    25  
    26  The purpose of this package
    27  
    28  This package has three main objectives.
    29  
    30  	1. Provide a middleware layer, and reduce the code that are not directly related to application value.
    31  	2. AppEngine, Cloud, to provide the same interface for both Datastore.
    32  	3. Enable batch processing for Single Get, Signle Put, etc.
    33  
    34  
    35  Middleware layer
    36  
    37  We are forced to make functions that are not directly related to the value of the application for speed, stability and operation.
    38  Such functions can be abstracted and used as middleware.
    39  
    40  Let's think about this case.
    41  Put Entity to Datastore and set it to Memcache or Redis.
    42  Next, when getting from Datastore, Get from Memcache first, Get it again from Datastore if it fails.
    43  It is very troublesome to provide these operations for all Kind and all Entity operations.
    44  However, if the middleware intervenes with all Datastore RPCs, you can transparently process without affecting the application code.
    45  
    46  As another case, RPC sometimes fails.
    47  If it fails, the process often succeeds simply by retrying.
    48  For easy RET retry with all RPCs, it is better to implement it as middleware.
    49  
    50  Please refer to https://godoc.org/go.mercari.io/datastore/dsmiddleware if you want to know the middleware already provided.
    51  
    52  
    53  Provide the same interface between AppEngine and Cloud Datastore
    54  
    55  The same interface is provided for AppEngine Datastore and Cloud Datastore.
    56  These two are compatible, you can run it with exactly the same code after creating the Client.
    57  
    58  For example, you can use AE Datastore in a production environment and Cloud Datastore Emulator in UnitTest.
    59  If you can avoid goapp, tests may be faster and IDE may be more vulnerable to debugging.
    60  You can also read data from the local environment via Cloud Datastore for systems running on AE Datastore.
    61  
    62  Caution.
    63  Although the storage bodies of RPCs of AE Datastore and Cloud Datastore are shared, there is a difference in expressiveness at the API level.
    64  Please carefully read the data written in AE Datastore carelessly on Cloud Datastore and do not update it.
    65  It may become impossible to read from the API of AE Datastore side.
    66  About this, we have not strictly tested.
    67  
    68  
    69  Batch processing
    70  
    71  The operation of Datastore has very little latency with respect to RPC's network.
    72  When acquiring 10 entities it means that GetMulti one time is better than getting 10 times using loops.
    73  However, we are not good at putting together multiple processes at once.
    74  Suppose, for example, you want to query on Post Kind, use the list of Comment IDs of the resulting Post, and get a list of Comments.
    75  For example, you can query Post Kind and get a list of Post.
    76  In addition, consider using CommentIDs of Post and getting a list of Comment.
    77  This is enough Query + 1 GetMulti is enough if you write very clever code.
    78  However, after acquiring the data, it is necessary to link the Comment list with the appropriate Post.
    79  On the other hand, you can easily write a code that throws a query once and then GetMulti the Comment as many as Post.
    80  In summary, it is convenient to have Put or Get queued, and there is a mechanism to execute it collectively later.
    81  
    82  Batch() is it!
    83  You can find the example at https://godoc.org/go.mercari.io/datastore/#pkg-examples .
    84  
    85  
    86  Boom replacing goon
    87  
    88  I love goon.
    89  So I made https://godoc.org/go.mercari.io/datastore/boom which can be used in conjunction with this package.
    90  
    91  
    92  How to migrate to this library
    93  
    94  Here's an overview of what you need to do to migrate your existing code.
    95  
    96  	replace *datastore.Key to datastore.Key.
    97  	replace *datastore.Query to datastore.Query.
    98  	replace *datastore.Iterator to datastore.Iterator.
    99  
   100  from AE Datastore
   101  
   102  	import go.mercari.io/datastore and go.mercari.io/datastore/aedatastore both.
   103  	rewrite those using functions of datastore package to FromContext function and Client method calls.
   104  	replace err.(appengine.MultiError) to err.(datastore.MultiError) .
   105  	Stop using appengine.BlobKey and replace with string.
   106  	replace google.golang.org/appengine/datastore.Done to google.golang.org/api/iterator.Done .
   107  	replace key.IntID() to key.ID() .
   108  	replace key.StringID() to key.Name() .
   109  	When nesting a struct, apply `datastore:", flatten "` to the corresponding field.
   110  	Delete datastore.TransactionOptions, it is not supported.
   111  	If using google.golang.org/appengine/datastore , replace to go.mercari.io/datastore .
   112  
   113  from Cloud Datastore
   114  
   115  	import go.mercari.io/datastore and go.mercari.io/datastore/clouddatastore .
   116  	rewrite those using functions of datastore package to FromContext function and Client method calls.
   117  	replace *datastore.Commit to datastore.Commit .
   118  	If using cloud.google.com/go/datastore , replace to go.mercari.io/datastore .
   119  
   120  from goon to boom
   121  
   122  	replace *goon.Goon to *boom.Boom .
   123  	replace goon.FromContext(ctx) to ds, _ := aedatastore.FromContext(ctx); boom.FromClient(ctx, ds) .
   124  */
   125  package datastore // import "go.mercari.io/datastore"