github.com/pingcap/tiflow@v0.0.0-20240520035814-5bf52d54e205/pkg/orchestrator/doc.go (about)

     1  // Copyright 2021 PingCAP, Inc.
     2  //
     3  // Licensed under the Apache License, Version 2.0 (the "License");
     4  // you may not use this file except in compliance with the License.
     5  // You may obtain a copy of the License at
     6  //
     7  //     http://www.apache.org/licenses/LICENSE-2.0
     8  //
     9  // Unless required by applicable law or agreed to in writing, software
    10  // distributed under the License is distributed on an "AS IS" BASIS,
    11  // See the License for the specific language governing permissions and
    12  // limitations under the License.
    13  
    14  /*
    15  Package orchestrator mainly implements a ETCD worker.
    16  A ETCD worker is used to read/write data from ETCD servers based on snapshot and data patches.
    17  Here is a detailed description of how the ETCD worker works:
    18  
    19  				   ETCD Servers
    20  					|       ^
    21  					|       |
    22  		   1. Watch |       | 5. Txn
    23  					|       |
    24  					v       |
    25  				    EtcdWorker
    26  					|       ^
    27  					|       |
    28  		   2. Update|       | 4. DataPatch
    29  		   +--------+       +-------+
    30  		   |                        |
    31  		   |                        |
    32  		   v         3.Tick         |
    33  		ReactorState ----------> Reactor
    34  
    35  	 1. EtcdWorker watches the txn modification log from ETCD servers
    36  	 2. EtcdWorker updates the txn modification listened from ETCD servers by calling the Update function of ReactorState
    37  	 3. EtcdWorker calls the Tick function of Reactor, and EtcdWorker make sure the state of ReactorState is a consistent snapshot of ETCD servers
    38  	 4. Reactor is implemented by the upper layer application. Usually, Reactor will produce DataPatches when the Tick function called
    39  	    EtcdWorker apply all the DataPatches produced by Reactor
    40  	 5. EtcdWorker commits a txn to ETCD according to DataPatches
    41  
    42  The upper layer application which is a user of EtcdWorker only need to implement Reactor and ReactorState interface.
    43  The ReactorState is used to maintenance status of ETCD, and the Reactor can produce DataPatches differently according to the ReactorState.
    44  The EtcdWorker make sure any ReactorState which perceived by Reactor must be a consistent snapshot of ETCD servers.
    45  */
    46  package orchestrator