mysqld: Normal shutdown

情况：

今早发现，昨天下午安装的4台mysql服务器，突然出现，由于在shell窗口

(root@localhost:mysql.sock) [(none)]> 190102 18:12:16 mysqld_safe mysqld from pid file /home/data/mysqldata/3306/data/mysql3.pid ended

什么情况，mysql意外关闭，4个shell窗口都出现了同样的错误，还是有点紧张，于是赶紧找error日志

[mysql@mysql3 scripts]$ tail -f -n 200 /home/data/mysqldata/3306/log/mysql-error.log

2019-01-02 17:24:21 3654 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.15-log'  socket: '/home/data/mysqldata/3306/mysql.sock'  port: 3306  MySQL Community Server - (GPL)
2019-01-02 17:24:21 3654 [Note] Event Scheduler: scheduler thread started with id 1
2019-01-02 17:27:46 3654 [Warning] 'proxies_priv' entry '@ root@mysql1' ignored in --skip-name-resolve mode.
2019-01-02 18:12:05 3654 [Note] /usr/sbin/mysqld: Normal shutdown

2019-01-02 18:12:05 3654 [Note] Giving 3 client threads a chance to die gracefully
2019-01-02 18:12:05 3654 [Note] Event Scheduler: Killing the scheduler thread, thread id 1
2019-01-02 18:12:05 3654 [Note] Event Scheduler: Waiting for the scheduler thread to reply
2019-01-02 18:12:05 3654 [Note] Event Scheduler: Stopped
2019-01-02 18:12:05 3654 [Note] Event Scheduler: Purging the queue. 0 events
2019-01-02 18:12:05 3654 [Note] Shutting down slave threads
2019-01-02 18:12:15 3654 [Note] Forcefully disconnecting 1 remaining clients
2019-01-02 18:12:15 3654 [Warning] /usr/sbin/mysqld: Forcing close of thread 6  user: 'root'

2019-01-02 18:12:15 3654 [Note] Binlog end
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'partition'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'BLACKHOLE'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_DELETED'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_METRICS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMPMEM'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMP_RESET'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_CMP'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_LOCKS'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'INNODB_TRX'
2019-01-02 18:12:15 3654 [Note] Shutting down plugin 'InnoDB'
2019-01-02 18:12:15 3654 [Note] InnoDB: FTS optimize thread exiting.
2019-01-02 18:12:15 3654 [Note] InnoDB: Starting shutdown...
2019-01-02 18:12:16 3654 [Note] InnoDB: Shutdown completed; log sequence number 7120303
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'ARCHIVE'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'MRG_MYISAM'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'MyISAM'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'MEMORY'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'CSV'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'sha256_password'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'mysql_old_password'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'mysql_native_password'
2019-01-02 18:12:16 3654 [Note] Shutting down plugin 'binlog'
2019-01-02 18:12:16 3654 [Note] /usr/sbin/mysqld: Shutdown complete

190102 18:12:16 mysqld_safe mysqld from pid file /home/data/mysqldata/3306/data/mysql3.pid ended
190102 18:13:32 mysqld_safe Starting mysqld daemon with databases from /home/data/mysqldata/3306/data
2019-01-02 18:13:34 4770 [Note] Plugin 'FEDERATED' is disabled.

如上面的日志标记所示，4台都是一样，很奇怪为什么会发现这样的问题

系统：cpu 4核，内存18g，磁盘hdd 200g

[root@mysql3 ~]# cat /etc/redhat-release
Kylin Linux release 3.3.1707 (Core)

mysql version：

Server version: 5.6.15-log MySQL Community Server - (GPL)

刚安装好的机器，不应该出现这样的问题，但是除了我，又没有谁来搞这个mysql，看了 linux的history

[root@mysql3 ~]# export HISTTIMEFORMAT='%F %T '
  105  2019-01-02 15:55:27 whereis mysql
  106  2019-01-02 15:55:31 which mysql
  107  2019-01-02 15:57:08 vim /home/data/mysqldata/3306/my.cnf
  108  2019-01-02 16:00:39 chown -R mysql:mysql /home/data/mysqldata/
  109  2019-01-02 16:00:43 su - mysql
  110  2019-01-03 09:00:19 history
  111  2019-01-03 09:00:44 su - mysql
  112  2019-01-03 10:06:49 cd /home/so
  113  2019-01-03 10:06:54 cd /home/

没有相应时间的history

[root@mysql1 3306]# su - mysql
[mysql@mysql1 ~]$ history
   66  2019-01-02 17:41:10 cd ../scripts/
   67  2019-01-02 17:41:11 ll
   68  2019-01-02 17:44:00 mysql -uroot -p -S /home/data/mysqldata/3306/mysql.sock
   69  2019-01-03 08:38:13 ll
   70  2019-01-03 08:38:15 cd ..
   71  2019-01-03 08:38:17 cd 3306/log/

找不到问题，这时候开始网上找找，发现有一些人遇到这个问题

2019-01-02 18:09:56 3549 [Note] /usr/sbin/mysqld: Normal shutdown

但是回答的大部分都是，找找是否有history |grep mysqladmin 或者service mysqld stop等命令，看看是否手动或者其他的关闭了mysql

因为从error上看，是正常的关闭mysql服务

查看slow log也没有什么发现

还看到有一个帖子

https://dba.stackexchange.com/questions/40581/usr-libexec-mysqld-normal-shutdown-but-my-team-doesnt-do-that

有提到bug的帖子

https://bugs.mysql.com/bug.php?id=51081

由于还是找不到原因，寻思许久，不得解

这是问了一下开发同学是否有重启过数据库，一问，才发现，他们悄悄的修改了my.cnf，然后同时重启了这4台机器。。。。。

哎，搞了半天，。。被开发的重启了，