tidb 数据导入流程

tidb 批量数据导入命令

tidb 数据导出命令

1.设置mysql的binlog格式为row

show variables like 'binlog_format'

2.将mysql数据导出

./bin/mydumper -h mysqlip -P 3306 -u root -p  password -t 16 -F 64 -B database  --skip-tz-utc -o ./var/database

3.导入数据到tidb

./bin/loader -h tidbip -u root -P 4000 -p password -t 32 -d ./var/database

4.配置syncer.meta

binlog-name = "mysql-bin.000032"

binlog-pos = 3222

binlog-gtid = ""

binlog-name和binlog-pos可以在var/database/metadata 中找到

5.配置syncer.toml

log-level = "info"

 

server-id = 101

 

meta = "./syncer.meta"

 

worker-count = 16

 

batch = 10

 

replicate-do-db = ["database"]

[from]

host = "mysqlip"

user = "root"

password = "password"

port = 3306

 

[to]

host = "127.0.0.1"

user = "root"

password = "password"

port = 4000

6.启动syncer

./bin/syncer -config config.toml

 

数据同步中遇到的问题

1.当binlog中有修改字段类型时，tidb不支持int 转varchar，也不支持datetime转varchar

解决方案：需要手动删除新建需要转换的字段，并且保证转换的字段顺序和同步的表的字段顺序一致