github.com/pingcap/ticdc@v0.0.0-20220526033649-485a10ef2652/integration/README.md (about)

     1  # Integration Framework
     2  
     3  ## Introduction
     4  The **Integration Framework** is designed to provide a flexible way for contributors to write integration tests for new sinks or MQ protocols. The core of the framework is stored in `{ticdc_root}/integration/framework`, and test cases should be stored in `{ticdc_root}/integration`. Currently, although the Framework is still under active development, it is capable of helping test Avro support and it is the only officially supported way for developers to run integration tests with Kafka connect.
     5  
     6  ## Quick Start
     7  To create a test case, you need to:
     8  - create a struct that implements the `Task` interface,
     9  - and ask the Environment to run the task in the `main` function in `integration.go`.
    10  Note that the second step will be automated soon.
    11  
    12  ```go
    13  // Task represents a single test case
    14  type Task interface {
    15  	Name() string
    16  	GetCDCProfile() *CDCProfile
    17  	Prepare(taskContext *TaskContext) error
    18  	Run(taskContext *TaskContext) error
    19  }
    20  ```
    21  For the time being, if you would like to write a test case for Avro and Canal, it is recommended to write a base case which define the common operations of test, and write construct function, pass `canal.SingleTableTask` or `canal.SingleTableTask` as parameters, which execute the necessary setup steps, including creating the Kafka Connect sink and creating the changefeed with appropriate configurations. 
    22  
    23  
    24  Example:
    25  ```go
    26  // tests/base_mycase.go
    27  type MyCase struct {
    28  	framework.Task
    29  }
    30  
    31  func NewMyCase(task framework) *MyCase{
    32  	return &MyCase{
    33          Task: task,  
    34      }   
    35  }
    36  
    37  func (c *MyCase) Name() string {
    38  	return "My Case"
    39  }
    40  
    41  func (c *MyCase) Run(ctx *framework.TaskContext) error {
    42  	_, err := ctx.Upstream.ExecContext(ctx.Ctx, "create table test (id int primary key, value int)")
    43  	if err != nil {
    44  		return err
    45  	}
    46  
    47  	// Get a handle of an existing table
    48  	table := ctx.SQLHelper().GetTable("test")
    49  	// Create an SQL request, send it to the upstream, wait for completion and check the correctness of replication
    50  	err = table.Insert(map[string]interface{}{
    51  		"id":    0,
    52  		"value": 0,
    53  	}).Send().Wait().Check()
    54  	if err != nil {
    55  		return errors.AddStack(err)
    56  	}
    57  
    58  	// To wait on a batch of SQL requests, create a slice of Awaitables
    59  	reqs := make([]framework.Awaitable, 0)
    60  	for i := 1; i < 1000; i++ {
    61  		// Only send, do not wait
    62  		req := table.Insert(map[string]interface{}{
    63  			"id":    i,
    64  			"value": i,
    65  		}).Send()
    66  		reqs = append(reqs, req)
    67  	}
    68  
    69  	// Wait on SQL requests in batch and check the correctness
    70  	return framework.All(ctx.SQLHelper(), reqs).Wait().Check()
    71  }
    72  
    73  
    74  // tests/integration.go
    75  func main() {
    76      task := &canal.SingleTableTask{TableName: "test"}
    77      testCases := []framework.Task{
    78          tests.NewMyCase(task),
    79      }
    80      task := &avro.SingleTableTask{TableName: "test"}
    81      testCases := []framework.Task{
    82          tests.NewMyCase(task),
    83      }
    84      //run
    85  }
    86  ```