不用看了,问题仍然存在----使用canal-deploy读取MySQL的binlog日志报错:java.net.SocketTimeoutException: Timeout occurred, failed to read total 4 bytes in 25000 milliseconds, actual read only 0 bytes

背景介绍

使用canal-deploy读取MySQL的binlog日志,检测到mall_test数据库中的数据表有数据更新的话,则会自动获取更新的数据往doris中写

详细报错信息:

2023-06-19 13:33:08.539 [destination = mall_test , address = /172.31.115.14:3306 , EventParser] ERROR com.alibaba.otter.canal.common.alarm.LogAlarmHandler - destination:mall_test[java.net.SocketTimeoutException: Timeout occurred, failed to read total 4 bytes in 25000 milliseconds, actual read only 0 bytes
        at com.alibaba.otter.canal.parse.driver.mysql.socket.BioSocketChannel.read(BioSocketChannel.java:124)
        at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.DirectLogFetcher.fetch0(DirectLogFetcher.java:170)
        at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.DirectLogFetcher.fetch(DirectLogFetcher.java:77)
        at com.alibaba.otter.canal.parse.inbound.mysql.MysqlConnection.dump(MysqlConnection.java:237)
        at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$1.run(AbstractEventParser.java:262)
        at java.lang.Thread.run(Thread.java:748)

参考网址:https://www.modb.pro/doc/22590

canal官网查询之前的issue有提到过这个问题的根源是mysql在25秒内都没有任何binlog事件,canal等了25秒没有任何消息就会出现类似的错误。一般就是没有开启心跳检查



下面操作开启心跳检测。开启心跳mysql,让mysql定时产生binlog

修改canal的配置

# vim conf/canal.properties

## detecing config
canal.instance.detecting.enable = true # 开启
#canal.instance.detecting.sql = select 1
canal.instance.detecting.sql = insert into mall_test.canal_health_check values(1,now()) on duplicate key update update_time=now()  # 写入心跳检查标的sql语句,mall_test.canal_health_check为指定的库表,表示往这个表里写心跳检测数据
canal.instance.detecting.interval.time = 20 # mysql定时事件,每隔20秒更新一次数据,时间长的话,mysql错误日志中会报错:Got an error reading communication packets
canal.instance.detecting.retry.threshold = 3
canal.instance.detecting.heartbeatHaEnable = false

数据库方面操作

登录mall_test数据库中,创建canal_health_check数据表

CREATE TABLE `canal_health_check` (
  `id` bigint(16) unsigned NOT NULL AUTO_INCREMENT,
  `update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4;

insert into canal_health_check(update_time) values(now());

开启insert的sql心跳后,canal用户需要新增权限

grant insert,update on mall_test.canal_health_check to 'xxxxx'@'%';

重启canal

posted @ 2023-06-19 13:46  哈喽哈喽111111  阅读(514)  评论(1编辑  收藏  举报