vitess.io/vitess@v0.16.2/go/vt/wrangler/doc_test.md (about)

     1  ## Wrangler test framework
     2  
     3  This document has some initial details on how the wrangler unit tests are setup. Updating tests is currently tedious,
     4  especially if sql queries get added/modified. We hope to simplify the framework "soon".
     5  
     6  We will illustrate by walking through a test.
     7  
     8  ### TestCanSwitch
     9  
    10  We first create mock tablets for source and targets and the associated database mocks. `traffic_switcher_env_test.go`
    11  and `fake_tablet_test.go` contains the framework code.  `newTestTableMigrater` is called to setup the
    12  workflow. `newTestTableMigrater` is just a wrapper on `newTestTableMigraterCustom` to create a workflow with two source
    13  shards "-40", "40-" and two target shards "-80", "80-". The filter it uses is a `select *` signifying MoveTables (rather
    14  than a Materialize).
    15  
    16  #### `newTestTableMigraterCustom`
    17  
    18  * setups a topo (memorytopo)
    19  * instantiates a wrangler with this topo, logging to the console
    20  * instantiates a mock db from `fakesqldb`
    21  * creates a fake tablet, one (primary) per shard
    22  * start tablets (see below)
    23  * creates db clients (the `fakeDBClient` for VR engine queries) and the VR engines using this mock db
    24  * sets starting gtid positions
    25  * setups a MoveTables: on the targets create vreplication stream entries for MoveTables and also create the reverse
    26    workflow on the sources. Expected sql queries for the vreplication table are also added here. Routing rules are setup.
    27  
    28  #### The fake tablet
    29  
    30  `go/vt/wrangler/fake_tablet_test.go`
    31  
    32  Fake tablets are "active": they run a goroutine listening to the tablet grpc port and are registered in the topo. The
    33  database however is a mock db and not an actual mysql server. Starting an actual mysql server is very slow and hence we
    34  prefer the mock db.
    35  
    36  The fake tablet has, as attributes, keyspace, shard and tablet uid. The key functionality is the action loop which
    37  creates listeners on the http and grpc ports and instantiates a `TabletManager` using the test topo server and fake
    38  mysql daemon and registers it with the grpc server.
    39  
    40  This ensures that the grpc calls made during the test are actually handled by tablet manager code and any vrengine
    41  database calls are intercepted by the mock. This setup is good enough for the tests that just setup the workflow and
    42  test the workflow state machine. There is no actual data being vreplicated.
    43  
    44  #### The fake MySQLDaemon
    45  
    46  `go/vt/mysqlctl/fakemysqldaemon/fakemysqldaemon.go`
    47  
    48  Used to set primary positions to provide/validate gtids.
    49  
    50  #### The fake db client
    51  
    52  `go/vt/wrangler/fake_dbclient_test.go`
    53  
    54  This defines a mock db which is limited in scope to the vreplication engine. All queries that it mocks are related to
    55  the `_vt` database only. The queries specified serve to validate that the expected set of queries were generated. When
    56  updating/adding tests, you will need to tell each test what queries are valid. For queries that can happen often and
    57  asynchronously, like updating heartbeats or setting gtid positions, we have the mechanism to ignore them or return a
    58  fixed result.
    59  
    60  ### Pain points / Possible improvements
    61  
    62  * Multiple mock db frameworks are used (different ones for wrangler, materialize) resulting in too many paradigms and
    63    duplication of functionality
    64  * sequence of queries matters. This was introduced for precise testing of logic when VReplication was first invented and
    65    many moving parts had to be validated. Do we still need this or can we switch to a query map based mock?
    66  
    67  Another option is to use a db server with a mysql API, ideally an in-memory one so that we can test based on actual
    68  data. We won't have to define the queries each time they change. We can of course just setup a regular mysql server.
    69  This is done in e2e tests, but it might be unacceptable for unit tests.  
    70