github.com/pingcap/tiflow@v0.0.0-20240520035814-5bf52d54e205/README_TiCDC.md (about)

     1  # TiCDC
     2  
     3  **TiCDC** is [TiDB](https://docs.pingcap.com/tidb/stable)'s change data capture framework. It replicates change data to various downstream systems, such as MySQL protocol-compatible databases and [Kafka](https://kafka.apache.org/).
     4  
     5  ## Architecture
     6  
     7  <img src="docs/media/cdc_architecture.svg?sanitize=true" alt="architecture" width="600"/>
     8  
     9  See a detailed introduction to [the TiCDC architecture](https://docs.pingcap.com/tidb/stable/ticdc-overview#ticdc-architecture).
    10  
    11  ## Documentation
    12  
    13  - [English](https://docs.pingcap.com/tidb/stable/ticdc-overview)
    14  - [简体中文](https://docs.pingcap.com/zh/tidb/stable/ticdc-overview)
    15  
    16  ## Blog
    17  
    18  - [English](https://pingcap.com/blog/)
    19  - [简体中文](https://cn.pingcap.com/blog/)
    20  
    21  ## Build
    22  
    23  To check the source code, run test cases and build binaries, you can simply run:
    24  
    25  ```bash
    26  $ make cdc
    27  $ make test
    28  ```
    29  
    30  Note that TiCDC supports building with the Go version `Go >= 1.21`.
    31  
    32  When TiCDC is built successfully, you can find binary in the `bin` directory. Instructions for unit test and integration test can be found in [Running tests](./tests/integration_tests/README.md).
    33  
    34  ## Deploy
    35  
    36  You can set up a CDC cluster for replication test manually as following:
    37  
    38  1. Set up a TiDB cluster.
    39  2. Start a CDC cluster, which contains one or more CDC servers. The command to start on CDC server
    40     is `cdc server --pd http://10.0.10.25:2379`, where `http://10.0.10.25:2379` is the client-url of pd-server.
    41  3. Start a replication changefeed by `cdc cli changefeed create --pd http://10.0.10.25:2379 --start-ts 413105904441098240 --sink-uri mysql://root:123456@127.0.0.1:3306/`. The TSO is TiDB `timestamp oracle`. If it is not provided or set to zero, the TSO of start time will be used. Currently, TiCDC supports MySQL protocol-compatible databases as downstream sinks only, and we will add more sink types in the future.
    42  
    43  For details, see [Deploy TiCDC](https://docs.pingcap.com/tidb/stable/deploy-ticdc).
    44  
    45  ## Quick start
    46  
    47  ```sh
    48  # Start TiDB cluster
    49  $ docker-compose -f ./deployments/ticdc/docker-compose/docker-compose-mysql.yml up -d
    50  
    51  # Attach to control container to run TiCDC
    52  $ docker exec -it ticdc_controller sh
    53  
    54  # Start to feed the changes on the upstream tidb, and sink to the downstream tidb
    55  $ ./cdc cli changefeed create --pd http://upstream-pd:2379 --sink-uri mysql://root@downstream-tidb:4000/
    56  
    57  # Exit the control container
    58  $ exit
    59  
    60  # Load data to the upstream tidb
    61  $ sysbench --mysql-host=127.0.0.1 --mysql-user=root --mysql-port=4000 --mysql-db=test oltp_insert --tables=1 --table-size=100000 prepare
    62  
    63  # Check sync progress
    64  $ mysql -h 127.0.0.1 -P 5000 -u root -e "SELECT COUNT(*) FROM test.sbtest1"
    65  ```
    66  
    67  ## Contributing
    68  
    69  We welcome and greatly appreciate contributions. See [CONTRIBUTING.md](./CONTRIBUTING.md)
    70  for details on submitting patches and the contribution workflow.