MySQL5.7 利用keepalived来实现mysql双主高可用方案的详细过程
Reference: http://blog.csdn.net/mchdba/article/details/51377989
服务器准备
Keepalived:192.168.13.15
Keepalived:192.168.13.16
Mysql-m1: 192.168.13.15
Mysql-m2: 192.168.13.16
1，在m1、m2上准备mysql5.7环境
在两台服务器上安装mysql5.7，安装mysql参考如下：http://blog.csdn.net/mchdba/article/details/51138063，然后关闭selinux。
2，在m1上部署keepalived
（1）yum源安装：yuminstall keepalived –y，只是版本低一些是1.2.13
（2）源码安装：
|
# 下载最新版本：1.2.20 wget http://www.keepalived.org/software/keepalived-1.2.20.tar.gz
# 解压缩安装 tar -xvf keepalived-1.2.20.tar.gz cd keepalived-1.2.20 yum install openssl* -y ./configure make make install
|
设置开机启动项：
|
cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/ cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/ ln -s /usr/local/sbin/keepalived /usr/sbin/ chmod +x /etc/init.d/keepalived chkconfig --add keepalived chkconfig keepalived on |
添加keepalived.conf配置文件：
|
mkdir /etc/keepalived vim /etc/keepalived/keepalived.conf global_defs { notification_email { haohailuo@163.com } notification_email_from mchdba@139.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id MySQL-ha }
vrrp_instance VI_1 { state BACKUP # 2 severs keep the same value. interface eth0 virtual_router_id 51 priority 100 # priority, the another set to 90 advert_int 1 nopreempt #don't race to control, set the highst priorty mysql servers only. authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.13.14 } }
virtual_server 192.168.13.14 3317 { delay_loop 2 # check the real_server status for every 2 seconds. lb_algo wrr #LVS arithmetic lb_kind DR #LVS model persistence_timeout 60 #k protocol TCP real_server 192.168.13.15 3317 { weight 3 notify_down /usr/local/mysql/bin/mysql.sh # run the scripts if mysql is down. TCP_CHECK { connect_timeout 10 #timeout nb_get_retry 3 #conect times to try to connect delay_before_retry 3 #interval of retry connect_port 3317 # check mysql port } } } |
blog来源地址：http://blog.csdn.net/mchdba/article/details/51377989，谢绝转载。
Keepalived默认的日志在/var/log/messages里面，如果要设置单独的日志路径，然后通过如下命令启动/usr/local/keepalived/sbin/keepalived -d -D -S 0 的方式来启动keepalived，或者修改/etc/sysconfig/keepalived，然后用service来启动。
service方式设置日志路径：
|
# 1 修改keepalived启动方式 [root@azure_lvdi_dbm1_13_15 mysql]# vim /etc/sysconfig/keepalived KEEPALIVED_OPTIONS="-D -d -S 0"
# 2 设置路径 /etc/syslog.conf [root@azure_lvdi_dbm1_13_15 mysql]# vim /etc/syslog.conf
# Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console
# Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access. authpriv.* /var/log/secure
# Log all the mail messages in one place. mail.* -/var/log/maillog
# Log cron stuff cron.* /var/log/cron
# Everybody gets emergency messages *.emerg *
# Save news errors of level crit and higher in a special file. uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log local7.* /var/log/boot.log
# keepalived -S 0 local0.* /var/log/keepalived.log |
3，在m2上部署keepalived
步骤和在m1上一模一样，只是keepalived.conf有所不同，如下黄色背景的部分配置信息：
|
[root@azure_lvdi_dbm1_13_16 ~]# more /etc/keepalived/keepalived.conf #vim /etc/keepalived/keepalived.conf global_defs { notification_email { haohailuo@163.com } notification_email_from mchdba@139.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id MySQL-ha }
vrrp_instance VI_1 { state BACKUP # 2 severs keep the same value. interface eth0 virtual_router_id 51 priority 90 # priority, m2 is set to 90 advert_int 1 #nopreempt #don't race to control, set the highst priorty mysql servers only. authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.13.14 } }
virtual_server 192.168.13.14 3317 { delay_loop 2 # check the real_server status for every 2 seconds. lb_algo wrr #LVS arithmetic lb_kind DR #LVS model persistence_timeout 60 #k protocol TCP real_server 192.168.13.16 3317 { weight 3 notify_down /usr/local/mysql/bin/mysql.sh # run the scripts if mysql is down. TCP_CHECK { connect_timeout 10 # nb_get_retry 3 #conect times to try to connect delay_before_retry 3 #interval of retry connect_port 3317 # check mysql port } } } |
4，远程登录验证数据库的vip
使用远程登录验证，能通过vip来连接数据库，对数据进行操作处理：
|
# 1，建立个测试帐号： GRANT SELECT, INSERT, UPDATE, DELETE ON `test`.* TO 'kt'@'1%' IDENTIFIED BY 'zhita26ywx18';
# 2，远程通过vip登录 [root@azure_lvdi_dbm1_13_16 keepalived]# mysql -h192.168.13.14 -P3317 -ukt --password="zhita26ywx18" mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1315 Server version: 5.7.11-log MySQL Community Server (GPL)
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use business_db Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A
Database changed mysql> insert into t2 select 10; Query OK, 1 row affected (0.00 sec) Records: 1 Duplicates: 0 Warnings: 0
mysql> select * from t2; +----+ | a | +----+ | 1 | | 3 | | 4 | | 3 | | 10 | +----+ 5 rows in set (0.00 sec)
mysql> |
5，准备验证vip自动切换的思路
为了测试验证的准确和可观性，暂时停止m1、m2的slave功能，并且在m1、m2上建立特殊的表来标示区分m1和m2，这样在通过vip进去的时候，能及时准确的知道vip指向哪个mysql实例。
|
# 1，m1上 mysql> stop slave; Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> create table test.m select "m1-db" as m1; # 准备m1表标识m1库 Query OK, 1 row affected (0.00 sec) Records: 1 Duplicates: 0 Warnings: 0
mysql> select * from business_db.m1; +-------+ | m1 | +-------+ | m1-db | +-------+ 1 row in set (0.00 sec)
mysql>
#2，m2上 mysql> stop slave; Query OK, 0 rows affected (0.00 sec)
mysql> create table test.m select "m2-db" as m2; # 准备m2表标识m2库 Query OK, 1 row affected (0.00 sec) Records: 1 Duplicates: 0 Warnings: 0
mysql> select * from business_db.m2; +-------+ | m2 | +-------+ | m2-db | +-------+ 1 row in set (0.00 sec)
mysql> |
6，开始验证
按照顺序，依次测试，可以测试出vip的迁移规律，可以通过如下来判断mysql服务和keepalived服务是否已经关闭：
|
# 关闭mysql服务 [root@azure_lvdi_dbm1_13_16 ~]# service mysqld stop Shutting down MySQL.... [ OK ] [root@azure_lvdi_dbm1_13_16 ~]# # 启动mysql服务 [root@azure_lvdi_dbm1_13_16 ~]# service mysqld start Starting MySQL.. [ OK ] [root@azure_lvdi_dbm1_13_16 ~]# # 判断mysql服务是否关闭，为0关闭 [root@azure_lvdi_dbm1_13_16 ~]# ps -eaf|grep mysqld |grep -v grep |wc |awk '{print $1}' 0 # 判断keepalived是否关闭，为0关闭 [root@azure_lvdi_dbm1_13_16 ~]# ps -eaf|grep keepalived |grep -v grep |wc |awk '{print $1}' 0 [root@azure_lvdi_dbm1_13_16 ~]#
|
6.1 ，m1、m2都在启动着keepalived，那么此时vip指向默认的m1实例
|
# 通过vip查询test.m表的标识数据来判断vip绑定在哪个mysql实例所在的服务器上 [root@localhost ~]# mysql –h10.254.13.14 -P3317 -ukt --password="zhita26ywx18" -e "select * from test.m"; Warning: Using a password on the command line interface can be insecure. +-------+ | m1 | +-------+ | m1-db | +-------+ [root@localhost ~]#
|
结论：启动m1、m2上mysql实例keepalived实例，此时，vip指向m1。
6.2，停止m1上的mysql实例，m2上的mysql实例和keepalived都启动着
|
# 通过vip查询test.m表的标识数据来判断vip绑定在哪个mysql实例所在的服务器上 [root@localhost ~]# mysql –h10.254.13.14 -P3317 -ukt --password="zhita26ywx18" -e "select * from test.m"; Warning: Using a password on the command line interface can be insecure. +-------+ | m2 | +-------+ | m2-db | +-------+ [root@localhost ~]#
|
结论：停止m1，原绑定在m1上的vip被释放了，通过ip addr也可以看到已经释放；然后vip启动切换到m2，通过vip自动访问上mysql实例，可以看到指向的是m2上的mysql实例，所以此时，vip指向m2。
6.3，再次启动已经停止的m1上的实例和keepalived
|
# 通过vip查询test.m表的标识数据来判断vip绑定在哪个mysql实例所在的服务器上 [root@localhost ~]# mysql –h10.254.13.14 -P3317 -ukt --password="zhita26ywx18" -e "select * from test.m"; Warning: Using a password on the command line interface can be insecure. +-------+ | m2 | +-------+ | m2-db | +-------+ [root@localhost ~]#
|
结论：再次启动m1后，发现vip还是绑定在m2上，表明如果当前vip所在的mysql实例没有down，则vip不会自动切换到别的mysql实例上，哪怕你启动了别的优先级高的keepalived服务绑定的mysql实例，主要原因是因为我们设定的不抢占的规则。
6.4，停止m2上的mysql实例
|
# 通过vip查询test.m表的标识数据来判断vip绑定在哪个mysql实例所在的服务器上 [root@localhost ~]# mysql –h10.254.13.14 -P3317 -ukt --password="zhita26ywx18" -e "select * from test.m"; Warning: Using a password on the command line interface can be insecure. +-------+ | m1 | +-------+ | m1-db | +-------+ [root@localhost ~]#
|
结论：停止m2上的mysql实例，则绑定在m2上的vip自动释放了，然后vip会切换到m1上的mysql实例服务器。
6.5 总结
从6.1到6.4的测试来看，keepalived基本满足了我们的ha服务，自动切换功能基本满足了，mysql down后能释放vip，切换vip到另外一台备用的mysql实例上面。
7，查看版本号
|
[root@azure_lvdi_dbm1_13_15 ~]# keepalived -v Keepalived v1.2.20 (05/08,2016)
Copyright (C) 2001-2016 Alexandre Cassen, <acassen@gmail.com>
Build options: KRNL_2_6 WITH_LVS HAVE_IPVS_SYNCD WITH_VRRP HAVE_VRRP_VMAC WITHOUT_ADDR_GEN_MODE WITHOUT_SNMP WITHOUT_SNMP_KEEPALIVED WITHOUT_SNMP_CHECKER WITHOUT_SNMP_RFC WITHOUT_SNMP_RFCV2 WITHOUT_SNMP_RFCV3 WITHOUT_LIBNL WITH_VRRP_AUTH WITH_SO_MARK WITHOUT_LIBIPTC WITHOUT_LIBIPSET WITHOUT_IPV4_DEVCONF WITHOUT_IF_H_LINK_H_COLLISION [root@azure_lvdi_dbm1_13_15 ~]# |
8，问题记录1
to the PKG_CONFIG_PATH environment variable
No package 'libiptc' found
checking for iptc_init in -liptc... no
checking for kernel version... 2.6.18
checking for IPVS syncd support... yes
checking for kernel macvlan support... yes
checking for kernel inet6_addr_gen_modesupport... no
checking whether SO_MARK is declared... no
configure: error: No SO_MARK declaration inheaders
解决方法2：
能ping通，但是telnet不通vip的3306端口
./configure --prefix=/usr/local/keepalived--disable-fwmark
9，问题记录2
[root@localhost ~]# mysql -h192.168.121.181-P3306 -ukt --password="zhita26ywx18" -e "select * fromtest.m";
Warning: Using a password on the commandline interface can be insecure.
ERROR 2003 (HY000): Can't connect to MySQLserver on '192.168.121.181' (111)
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# ping 192.168.121.181
PING 192.168.121.181 (192.168.121.181)56(84) bytes of data.
64 bytes from 192.168.121.181: icmp_seq=1ttl=64 time=0.316 ms
64 bytes from 192.168.121.181: icmp_seq=2ttl=64 time=0.330 ms
^C
--- 192.168.121.181 ping statistics ---
2 packets transmitted, 2 received, 0%packet loss, time 1713ms
rtt min/avg/max/mdev = 0.316/0.323/0.330/0.007ms
[root@localhost ~]# telnet 192.168.121.1813306
Trying 192.168.121.181...
telnet: connect to address 192.168.121.181:Connection refused
[root@localhost ~]# telnet 192.168.121.1813307
Trying 192.168.121.181...
Connected to 192.168.121.181.
Escape character is '^]'.
N
5.6.12-log[¦~jv5A)Fg-TPqkDOB<F);.mysql_native_passwordXshell
!
#08S01Got packets out of orderConnection closed by foreignhost.
[root@localhost ~]#
看到m2上面的mysql端口是3307，但是统一的vip的数据库端口是3306，所以telnet不通3306，可以telnet3307，这里有一个折中方案，采用iptables端口转发下，将m2的3306端口转到3307。
加一个端口映射：
[root@zabbix_serv_121_12 keepalived]#iptables -t nat -A PREROUTING -p tcp --dport 3306 -j REDIRECT --to-port 3307
[root@zabbix_serv_121_12 keepalived]#
再验证，可以连接上m2了：
[root@localhost ~]# telnet 192.168.121.1813306
Trying 192.168.121.181...
Connected to 192.168.121.181.
Escape character is '^]'.
N
5.6.12-log꧞)P6\PXK-P1A2uz'P<xl7mysql_native_passwordXshell
!
#08S01Got packets out of orderConnection closed by foreignhost.
[root@localhost ~]#
[root@localhost ~]# mysql -h192.168.121.181-P3306 -ukt --password="zhita26ywx18" -e 'select * from test.m';
Warning: Using a password on the commandline interface can be insecure.
+-------+
| m2 |
+-------+
| m2-db |
+-------+
[root@localhost ~]#
