1.准备阶段
1.1 环境
虚拟机A :192.168.0.130
虚拟机B :192.168.0.131
系统:Ubuntu 16.04 LTS
JRE:OpenJDK 1.8.0_151(A、B都要安装)
DB:MySQL 5.7.20(A、B都要安装)
ZooKeeper:3.4.11(只装A机)
Otter Manager:4.2.14 (只装A机)(4.2.15启动Node时感觉有Bug,没有使用)
Otter Node:4.2.14(A、B都要安装)
1.2 安装环境
1.2.1 JRE安装
sudo apt-get install default-jre
1.2.2 MySQL安装
sudo apt-get install mysql-server
sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf
character-set-server=utf8 log-bin=mysql-bin binlog-format=ROW server-id=1 #A,B需要设置成不一样的id #bind-address=127.0.0.1 #注释该行
service mysql restart
进入mysql,设置单独账号密码
grant all on *.* to 'root'@'%' identified by 'password';
1.2.3 ZooKeeper安装
安装包地址 http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.11/zookeeper-3.4.11.tar.gz
解压可见conf文件夹下有一个zoo_sample.cfg的文件,重命名为zoo.cfg并修改以下配置
dataDir=../datas dataLogDir=../logs
1.2.4 Otter Manager安装
安装包地址 https://github.com/alibaba/otter/releases 请自己选择版本
解压可见conf文件夹下otter.properties文件,修改以下配置
#以下配置为最基本需要改的配置,其他配置可根据实际要用的功能进行修改 otter.domainName = 192.168.0.130 #一定要改,不要用127.0.0.1 otter.port = 9000 #manager 站点端口号 otter.database.driver.url = jdbc:mysql://192.168.0.130:3306/otter otter.database.driver.username = root otter.database.driver.password = password otter.zookeeper.cluster.default = 192.168.0.130:2181
1.2.5 Otter Node 安装
安装包地址 https://github.com/alibaba/otter/releases 请自己选择版本
解压可见conf文件夹下otter.properties文件,修改以下配置
#以下配置为最基本需要改的配置,其他配置可根据实际要用的功能进行修改 otter.nodeHome = ../ otter.manager.address = 192.168.0.130:1099
conf文件夹下创建一个新的配置文件,命名为nid,写入node id,A机为1,B机为2。该id后面会介绍。
1
2.启动
2.1 启动ZooKeeper
进入ZooKeeper目录下的bin文件夹,执行以下命令(不要使用sh,一定要用bash)
bash zkServer.sh start
2.2 启动Otter Manager
在A机中执行以下数据库语句,创建Otter库
CREATE DATABASE /*!32312 IF NOT EXISTS*/ `otter` /*!40100 DEFAULT CHARACTER SET utf8 COLLATE utf8_bin */; USE `otter`; CREATE TABLE `ALARM_RULE` ( `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT, `MONITOR_NAME` varchar(1024) DEFAULT NULL, `RECEIVER_KEY` varchar(1024) DEFAULT NULL, `STATUS` varchar(32) DEFAULT NULL, `PIPELINE_ID` bigint(20) NOT NULL, `DESCRIPTION` varchar(256) DEFAULT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, `MATCH_VALUE` varchar(1024) DEFAULT NULL, `PARAMETERS` text DEFAULT NULL, PRIMARY KEY (`ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `AUTOKEEPER_CLUSTER` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `CLUSTER_NAME` varchar(200) NOT NULL, `SERVER_LIST` varchar(1024) NOT NULL, `DESCRIPTION` varchar(200) DEFAULT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `CANAL` ( `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT, `NAME` varchar(200) DEFAULT NULL, `DESCRIPTION` varchar(200) DEFAULT NULL, `PARAMETERS` text DEFAULT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), UNIQUE KEY `CANALUNIQUE` (`NAME`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `CHANNEL` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `NAME` varchar(200) NOT NULL, `DESCRIPTION` varchar(200) DEFAULT NULL, `PARAMETERS` text DEFAULT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), UNIQUE KEY `CHANNELUNIQUE` (`NAME`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `COLUMN_PAIR` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `SOURCE_COLUMN` varchar(200) DEFAULT NULL, `TARGET_COLUMN` varchar(200) DEFAULT NULL, `DATA_MEDIA_PAIR_ID` bigint(20) NOT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), KEY `idx_DATA_MEDIA_PAIR_ID` (`DATA_MEDIA_PAIR_ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `COLUMN_PAIR_GROUP` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `DATA_MEDIA_PAIR_ID` bigint(20) NOT NULL, `COLUMN_PAIR_CONTENT` text DEFAULT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), KEY `idx_DATA_MEDIA_PAIR_ID` (`DATA_MEDIA_PAIR_ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `DATA_MEDIA` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `NAME` varchar(200) NOT NULL, `NAMESPACE` varchar(200) NOT NULL, `PROPERTIES` varchar(1000) NOT NULL, `DATA_MEDIA_SOURCE_ID` bigint(20) NOT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), UNIQUE KEY `DATAMEDIAUNIQUE` (`NAME`,`NAMESPACE`,`DATA_MEDIA_SOURCE_ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `DATA_MEDIA_PAIR` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `PULLWEIGHT` bigint(20) DEFAULT NULL, `PUSHWEIGHT` bigint(20) DEFAULT NULL, `RESOLVER` text DEFAULT NULL, `FILTER` text DEFAULT NULL, `SOURCE_DATA_MEDIA_ID` bigint(20) DEFAULT NULL, `TARGET_DATA_MEDIA_ID` bigint(20) DEFAULT NULL, `PIPELINE_ID` bigint(20) NOT NULL, `COLUMN_PAIR_MODE` varchar(20) DEFAULT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), KEY `idx_PipelineID` (`PIPELINE_ID`,`ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `DATA_MEDIA_SOURCE` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `NAME` varchar(200) NOT NULL, `TYPE` varchar(20) NOT NULL, `PROPERTIES` varchar(1000) NOT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), UNIQUE KEY `DATAMEDIASOURCEUNIQUE` (`NAME`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `DELAY_STAT` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `DELAY_TIME` int(21) NOT NULL, `DELAY_NUMBER` bigint(20) NOT NULL, `PIPELINE_ID` bigint(20) NOT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), KEY `idx_PipelineID_GmtModified_ID` (`PIPELINE_ID`,`GMT_MODIFIED`,`ID`), KEY `idx_Pipeline_GmtCreate` (`PIPELINE_ID`,`GMT_CREATE`), KEY `idx_GmtCreate_id` (`GMT_CREATE`,`ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `LOG_RECORD` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `NID` varchar(200) DEFAULT NULL, `CHANNEL_ID` varchar(200) NOT NULL, `PIPELINE_ID` varchar(200) NOT NULL, `TITLE` varchar(1000) DEFAULT NULL, `MESSAGE` text DEFAULT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), KEY `logRecord_pipelineId` (`PIPELINE_ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `NODE` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `NAME` varchar(200) NOT NULL, `IP` varchar(200) NOT NULL, `PORT` bigint(20) NOT NULL, `DESCRIPTION` varchar(200) DEFAULT NULL, `PARAMETERS` text DEFAULT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), UNIQUE KEY `NODEUNIQUE` (`NAME`,`IP`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `PIPELINE` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `NAME` varchar(200) NOT NULL, `DESCRIPTION` varchar(200) DEFAULT NULL, `PARAMETERS` text DEFAULT NULL, `CHANNEL_ID` bigint(20) NOT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), UNIQUE KEY `PIPELINEUNIQUE` (`NAME`,`CHANNEL_ID`), KEY `idx_ChannelID` (`CHANNEL_ID`,`ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `PIPELINE_NODE_RELATION` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `NODE_ID` bigint(20) NOT NULL, `PIPELINE_ID` bigint(20) NOT NULL, `LOCATION` varchar(20) NOT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), KEY `idx_PipelineID` (`PIPELINE_ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `SYSTEM_PARAMETER` ( `ID` bigint(20) unsigned NOT NULL, `VALUE` text DEFAULT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; CREATE TABLE `TABLE_HISTORY_STAT` ( `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT, `FILE_SIZE` bigint(20) DEFAULT NULL, `FILE_COUNT` bigint(20) DEFAULT NULL, `INSERT_COUNT` bigint(20) DEFAULT NULL, `UPDATE_COUNT` bigint(20) DEFAULT NULL, `DELETE_COUNT` bigint(20) DEFAULT NULL, `DATA_MEDIA_PAIR_ID` bigint(20) DEFAULT NULL, `PIPELINE_ID` bigint(20) DEFAULT NULL, `START_TIME` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `END_TIME` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), KEY `idx_DATA_MEDIA_PAIR_ID_END_TIME` (`DATA_MEDIA_PAIR_ID`,`END_TIME`), KEY `idx_GmtCreate_id` (`GMT_CREATE`,`ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `TABLE_STAT` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `FILE_SIZE` bigint(20) NOT NULL, `FILE_COUNT` bigint(20) NOT NULL, `INSERT_COUNT` bigint(20) NOT NULL, `UPDATE_COUNT` bigint(20) NOT NULL, `DELETE_COUNT` bigint(20) NOT NULL, `DATA_MEDIA_PAIR_ID` bigint(20) NOT NULL, `PIPELINE_ID` bigint(20) NOT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), KEY `idx_PipelineID_DataMediaPairID` (`PIPELINE_ID`,`DATA_MEDIA_PAIR_ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `THROUGHPUT_STAT` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `TYPE` varchar(20) NOT NULL, `NUMBER` bigint(20) NOT NULL, `SIZE` bigint(20) NOT NULL, `PIPELINE_ID` bigint(20) NOT NULL, `START_TIME` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `END_TIME` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), KEY `idx_PipelineID_Type_GmtCreate_ID` (`PIPELINE_ID`,`TYPE`,`GMT_CREATE`,`ID`), KEY `idx_PipelineID_Type_EndTime_ID` (`PIPELINE_ID`,`TYPE`,`END_TIME`,`ID`), KEY `idx_GmtCreate_id` (`GMT_CREATE`,`ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `USER` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `USERNAME` varchar(20) NOT NULL, `PASSWORD` varchar(20) NOT NULL, `AUTHORIZETYPE` varchar(20) NOT NULL, `DEPARTMENT` varchar(20) NOT NULL, `REALNAME` varchar(20) NOT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), UNIQUE KEY `USERUNIQUE` (`USERNAME`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE `DATA_MATRIX` ( `ID` bigint(20) NOT NULL AUTO_INCREMENT, `GROUP_KEY` varchar(200) DEFAULT NULL, `MASTER` varchar(200) DEFAULT NULL, `SLAVE` varchar(200) DEFAULT NULL, `DESCRIPTION` varchar(200) DEFAULT NULL, `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`ID`), KEY `GROUPKEY` (`GROUP_KEY`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; insert into USER(ID,USERNAME,PASSWORD,AUTHORIZETYPE,DEPARTMENT,REALNAME,GMT_CREATE,GMT_MODIFIED) values(null,'admin','801fc357a5a74743894a','ADMIN','admin','admin',now(),now()); insert into USER(ID,USERNAME,PASSWORD,AUTHORIZETYPE,DEPARTMENT,REALNAME,GMT_CREATE,GMT_MODIFIED) values(null,'guest','471e02a154a2121dc577','OPERATOR','guest','guest',now(),now());
在A,B机中都要执行以下数据库语句,创建retl库(若只是单向同步,则不需要执行该语句。该语句用作双A同步时的算法)
/* 供 otter 使用, otter 需要对 retl.* 的读写权限,以及对业务表的读写权限 1. 创建database retl */ CREATE DATABASE retl; /* 2. 用户授权 给同步用户授权 */ CREATE USER retl@'%' IDENTIFIED BY 'retl'; GRANT USAGE ON *.* TO `retl`@'%'; GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO `retl`@'%'; GRANT SELECT, INSERT, UPDATE, DELETE, EXECUTE ON `retl`.* TO `retl`@'%'; /* 业务表授权,这里可以限定只授权同步业务的表 */ GRANT SELECT, INSERT, UPDATE, DELETE ON *.* TO `retl`@'%'; /* 3. 创建系统表 */ USE retl; DROP TABLE IF EXISTS retl.retl_buffer; DROP TABLE IF EXISTS retl.retl_mark; DROP TABLE IF EXISTS retl.xdual; CREATE TABLE retl_buffer ( ID BIGINT(20) AUTO_INCREMENT, TABLE_ID INT(11) NOT NULL, FULL_NAME varchar(512), TYPE CHAR(1) NOT NULL, PK_DATA VARCHAR(256) NOT NULL, GMT_CREATE TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, GMT_MODIFIED TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, CONSTRAINT RETL_BUFFER_ID PRIMARY KEY (ID) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; CREATE TABLE retl_mark ( ID BIGINT AUTO_INCREMENT, CHANNEL_ID INT(11), CHANNEL_INFO varchar(128), CONSTRAINT RETL_MARK_ID PRIMARY KEY (ID) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; CREATE TABLE xdual ( ID BIGINT(20) NOT NULL AUTO_INCREMENT, X timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (ID) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8; /* 4. 插入初始化数据 */ INSERT INTO retl.xdual(id, x) VALUES (1,now()) ON DUPLICATE KEY UPDATE x = now();
进入Otter Manager 下的bin目录
bash startup.sh
这时等待数秒,Otter Manager站点已启动,可以访问http://192.168.0.130:9000配置Otter Node 的信息了。
2.3 启动Otter Node
如果你未在Otter Manager站点配置Otter Node的信息,则请暂时跳过该节,直接查看下面配置章节,配置完成后再回来该节启动Otter Node
进入Otter Node下的bin目录
bash startup.sh
3.配置
进入Otter Manager站点后,使用账号:admin,密码:admin(默认),获得超级管理员权限。
3.1 ZooKeeper配置
机器管理 --> ZooKeeper管理 ,添加,保存,如下图
3.2 Node配置
机器管理 --> Node管理 ,添加,保存(2次,A机一个Node,B机一个Node),如下图(名称和IP需要根据不同机器修改,这里只列出A机图片)
最后可在Node管理列表中看见,其中序号即我们上面安装过程中写入的nid文件的id号,配置好之后,即可启动Otter Node
启动后可见配置列表
3.3 数据源配置
配置管理 --> 数据源配置,配置A机、B机两个MySQL 数据源
配置完成后可见配置列表
3.4 数据表配置
配置管理 --> 数据表配置,配置A机、B机两个MySQL 数据源所需要同步的数据表(自己的数据库,我这里新建test库),其中table name配置如果想同步所有表则配置为.*,否则则配置对应表名即可
配置完成后可见配置列表
3.5 Canal配置
将canal看作是A、B两机的从库即可,github上解释道,canal模拟mysql slave的交互协议,伪装自己为mysql slave,向mysql master发送dump协议。
配置管理 --> canal配置,配置A机、B机两个canal。实质上两个canal分别运行在A、B两机上的Node节点上。配置如下,其他使用默认配置。
位点信息分别在A、B两个MySQL执行以下语句获取
show master status; select unix_timestamp(now());
配置完成后可见配置列表
3.6 同步任务配置
同步管理 --> Channel管理 ,添加一个Channel
添加后,点击进入Pipeline管理
添加两个Pipeline,如图是其中一个方向,另外一个方向Select、Load机器相反。canal选择与Node机器选择一致方向,即要与Select机器一样。选择其中一个Pipeline作为主站点,并在高级设置中勾选支持ddl同步,另外一个主站点勾选否,支持ddl同步选择否。
添加后,点击进入映射关系列表,选择好同步表的方向
至此。配置完成。到Channel管理中开启同步任务,Otter将为我们自动双A同步数据啦!
4. 参考资料
https://github.com/alibaba/otter
https://github.com/alibaba/canal
https://junnan.org/2017/01/mysql-two-way-synchronization-use-otter.html
https://yq.aliyun.com/articles/58388?spm=5176.100239.blogcont58420.15.ZyRocX
http://download.csdn.net/download/yjx19930417/10170913