使用canal通过mysql复制协议从binlog实现热数据nosql缓存(2)

开启mysql binlog功能

以5.7版本为例,找到/etc/mysql/mysql.conf.d/mysqld.cnf

[mysqld]
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
datadir         = /var/lib/mysql
#log-error      = /var/log/mysql/error.log
# By default we only accept connections from localhost
#bind-address   = 127.0.0.1
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

# 开启binlog
log-bin=mysql-bin 
binlog-format=ROW
server_id=1	
# 开启binlog

配置完毕重启mysql服务,重启完成后,查看是否已经成功开启

show variables like 'log_%';

 

canal-server服务

点击下载ali-canal 配置canal-server,修改\canal.deployer-1.0.22\conf\example\instance.properties

## mysql serverId 务必与master数据库不同
canal.instance.mysql.slaveId = 1234

需要追踪复制的master数据库信息
canal.instance.master.address = 127.0.0.1:3307
canal.instance.master.journal.name = 
canal.instance.master.position = 
canal.instance.master.timestamp = 

#canal.instance.standby.address = 
#canal.instance.standby.journal.name =
#canal.instance.standby.position = 
#canal.instance.standby.timestamp = 

username/password
canal.instance.dbUsername = root
canal.instance.dbPassword = root
canal.instance.defaultDatabaseName = test
canal.instance.connectionCharset = UTF-8

启动项目
\canal.deployer-1.0.22\bin\

 

canal-client服务

https://github.com/wyfteam123/mysqlbinlog-canal-nosql/tree/master/canal-client
修改对应配置,使用maven打包启动项目。

 

mq-consummer服务

java实现
https://github.com/wyfteam123/mysqlbinlog-canal-nosql/tree/master/sync-server/java_sync_nosql

python实现
https://github.com/wyfteam123/mysqlbinlog-canal-nosql/tree/master/sync-server/python_sync_nosql

 

rabbit/redis/mysql

推荐docker容器安装

posted @ 2019-01-21 16:57  马达狮子  阅读(174)  评论(0编辑  收藏  举报