Icident event 分析
现象
备库中断,显示如下错误
Connect_Retry: 60 Master_Log_File: mysql-bin.000185 Read_Master_Log_Pos: 308647804 Relay_Log_File: slave-relay.000002 Relay_Log_Pos: 1456 Relay_Master_Log_File: mysql-bin.000119 Slave_IO_Running: Yes Slave_SQL_Running: No Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 1590 Last_Error: The incident LOST_EVENTS occured on the master. Message: error writing to the binary log Skip_Counter: 0 Exec_Master_Log_Pos: 525863559 Relay_Log_Space: 35663403551
error log日志如下
2016-06-08 15:40:27 22134 [ERROR] The incident LOST_EVENTS occured on the master. Message: error writing to the binary log, Error_code: 1590
查看binlog
mysqlbinlog -vv slave-relay.000002 --start-position=1456|more # at 1456 #160604 19:18:57 server id 1129472893 end_log_pos 525863616 CRC32 0xece8422b # Incident: LOST_EVENTS RELOAD DATABASE; # Shall generate syntax error
Incident: LOST_EVENTS 是从主库拉取来
分析
主库写INCIDENT_EVENT的时机
- 权限变更操作可能只处理了一部分并发生错误时,会写一条INCIDENT_EVENT
- 事务中包含非事务表操作,但中途写binlog到cache出错会写一条INCIDENT_EVENT
以上两种情况都会造成binlog复制出现不一致,因此主库选择记录INCIDENT_EVENT,备库在解析到 Incident event就直接报错
Incident_log_event::do_apply_event(Relay_log_info const *rli) { DBUG_ENTER("Incident_log_event::do_apply_event"); if (ignored_error_code(ER_SLAVE_INCIDENT)) { DBUG_PRINT("info", ("Ignoring Incident")); DBUG_RETURN(0); } rli->report(ERROR_LEVEL, ER_SLAVE_INCIDENT, ER(ER_SLAVE_INCIDENT), description(), m_message.length > 0 ? m_message.str : "<none>"); DBUG_RETURN(1); }
修复
此种情况一般建议备库重搭
相关bug https://bugs.mysql.com/bug.php?id=68892 已修复