keepalive配置kingbaseES主备流复制集群VIP
案例说明：
在KingbaseES主备流复制架构基础上，通过keepalive提供vip及vip切换管理。在本案例中，keepalive只是对vip及vip切换管理，没有涉及到其他数据资源切换的管理。
测试环境：
操作系统：
[kingbase@srv1 bin]$ cat /etc/centos-release
CentOS Linux release 7.7.1908 (Core)
数据库：
prod=# select version();
version
-------------------------------------------------------------------------------------------------------------------
Kingbase V008R003C002B0061 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46), 64-bit
(1 row)
系统架构信息：
一、下载安装keepalived
1、下载keepalived
https://www.keepalived.org/download.html
2、安装依赖包
[root@srv2 network-scripts]# yum install -y openssl-devel libnl-devel libnl3-devel libnfnetlink-devel
3、安装keepalive（源码）
[root@srv1 soft]# scp keepalived-2.0.8.tar.gz srv2:/soft
keepalived-2.0.8.tar.gz 100% 888KB 24.3MB/s 00:00
[root@srv1 soft]# tar zxvf keepalived-2.0.8.tar.gz
keepalived-2.0.8/
keepalived-2.0.8/install-sh
keepalived-2.0.8/missing
keepalived-2.0.8/compile
keepalived-2.0.8/Makefile.in
keepalived-2.0.8/bin_install/
keepalived-2.0.8/bin_install/Makefile.in
keepalived-2.0.8/bin_install/Makefile.am
keepalived-2.0.8/keepalived.spec.in
......
[root@srv2 keepalived-2.0.8]# ./configure --prefix=/usr/local/keepalive
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for gcc... gcc
......
[root@srv2 keepalived-2.0.8]# make
[root@srv2 keepalived-2.0.8]# make
Making all in lib
make[1]: 进入目录“/soft/keepalived-2.0.8/lib”
make all-am
make[2]: 进入目录“/soft/keepalived-2.0.8/lib”
CC memory.o
CC utils.o
CC notify.o
CC timer.o
CC scheduler.o
CC vector.o
CC list.o
CC html.o
CC parser.o
......
[root@srv2 keepalived-2.0.8]# make install
Making install in lib
make[1]: 进入目录“/soft/keepalived-2.0.8/lib”
make install-am
make[2]: 进入目录“/soft/keepalived-2.0.8/lib”
make[3]: 进入目录“/soft/keepalived-2.0.8/lib”
make[3]: 对“install-exec-am”无需做任何事。
make[3]: 对“install-data-am”无需做任何事。
make[3]: 离开目录“/soft/keepalived-2.0.8/lib”
make[2]: 离开目录“/soft/keepalived-2.0.8/lib”
make[1]: 离开目录“/soft/keepalived-2.0.8/lib”
Making install in keepalived
......
二、配置keeepalived
1、配置keepalived有systemctl启动管理
[root@srv2 keepalive]# cp /usr/local/keepalive/sbin/* /sbin
[root@srv2 keepalive]# cp /usr/local/keepalive/bin/* /bin
[root@srv1 keepalive]# ls sbin
keepalived
[root@srv2 keepalive]# cp /usr/local/keepalive/etc/keepalived/ /etc/ -rf
[root@srv1 keepalive]# ls /etc/keepalived/
keepalived.conf samples
2、主库配置keepalive配置文件
[root@srv2 keepalived]# cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_script chk_kes_port { #检测mysql服务是否在运行。有很多方式，比如进程，用脚本检测等等
script "/opt/chk_kes.sh" #这里通过脚本监测
interval 2 #脚本执行间隔，每2s检测一次
weight -5 #脚本结果导致的优先级变更，检测失败（脚本返回非0）则优先级 -5
fall 2 #检测连续2次失败才算确定是真失败。会用weight减少优先级（1-255之间）
rise 1 #检测1次成功就算成功。但不修改优先级
}
vrrp_instance VI_1 {
state BACKUP
nopreempt
interface enp0s8
virtual_router_id 51
priority 101 #主库优先级高于备库
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.254
}
track_script { #监控脚本
chk_kes_port
}
}
3、 配置keepalive配置文件（备库）
[root@srv1 keepalived]# cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_script chk_kes_port { #检测mysql服务是否在运行。有很多方式，比如进程，用脚本检测等等
script "/opt/chk_kes.sh" #这里通过脚本监测
interval 2 #脚本执行间隔，每2s检测一次
weight -5 #脚本结果导致的优先级变更，检测失败（脚本返回非0）则优先级 -5
fall 2 #检测连续2次失败才算确定是真失败。会用weight减少优先级（1-255之间）
rise 1 #检测1次成功就算成功。但不修改优先级
}
vrrp_instance VI_1 {
state BACKUP #主备库状态都是BACKUP
nopreempt #非抢占模式
interface enp0s8 #绑定指定网卡
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.254 #指定VIP
}
track_script { #监控脚本
chk_kes_port
}
}
4、 检测数据库状态的脚本（主库）
[root@srv2 keepalived]# ls -lh /opt/chk_kes.sh
-rwxr-xr-x 1 root root 318 1月 8 15:19 /opt/chk_kes.sh
[root@srv2 keepalived]# cat /opt/chk_kes.sh
#!/bin/bash
#This scripts is check for kingbaseES DB status
counter=$(netstat -na|grep "LISTEN"|grep "54321"|wc -l)
if [ "${counter}" -eq 0 ]; then
systemctl stop keepalived
killall keepalived
fi
ping 192.168.1.3 -w1 -c1 &>/dev/null
if [ $? -ne 0 ]
then
systemctl stop keepalived
killall keepalived
fi
5、 备库检测数据库状态脚本
[root@srv1 opt]# cat chk_kes.sh
#!/bin/bash
#This scripts is check for kingbaseES DB status
counter=$(netstat -na|grep "LISTEN"|grep "54321"|wc -l)
if [ "${counter}" -eq 0 ]; then
systemctl stop keepalived
killall keepalived
fi
ping 192.168.1.2 -w1 -c1 &>/dev/null
if [ $? -ne 0 ]
then
systemctl stop keepalived
killall keepalived
fi
[root@srv1 opt]# chmod +x chk_kes.sh
三、启动数据库和keepalived服务
注意：首先要启动数据库服务，否则keepalived服务启动失败！！！
1、查看数据库服务状态（主库）
[kingbase@srv2 bin]$ netstat -an |grep 54321
tcp 0 0 0.0.0.0:54321 0.0.0.0:* LISTEN
tcp6 0 0 :::54321 :::* LISTEN
unix 2 [ ACC ] STREAM LISTENING 92169 /tmp/.s.KINGBASE.54321
2、启动keepalived服务
[root@srv2 sbin]# systemctl start keepalived
[root@srv2 sbin]# systemctl status keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
Active: active (running) since 五 2021-01-08 15:38:56 CST; 4s ago
Process: 31879 ExecStart=/usr/local/keepalive/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 31881 (keepalived)
Tasks: 2
CGroup: /system.slice/keepalived.service
├─31881 /usr/local/keepalive/sbin/keepalived -D
└─31882 /usr/local/keepalive/sbin/keepalived -D
1月 08 15:39:00 srv2 Keepalived_vrrp[31882]: (VI_1) Receive advertisement timeout
1月 08 15:39:00 srv2 Keepalived_vrrp[31882]: (VI_1) Entering MASTER STATE
1月 08 15:39:00 srv2 Keepalived_vrrp[31882]: (VI_1) setting iptable drop rule
1月 08 15:39:00 srv2 Keepalived_vrrp[31882]: (VI_1) setting VIPs.
1月 08 15:39:00 srv2 Keepalived_vrrp[31882]: Sending gratuitous ARP on enp0s8 for 192.168.1.254
1月 08 15:39:00 srv2 Keepalived_vrrp[31882]: (VI_1) Sending/queueing gratuitous ARPs on enp0s8 for 192.168.1.254
1月 08 15:39:00 srv2 Keepalived_vrrp[31882]: Sending gratuitous ARP on enp0s8 for 192.168.1.254
1月 08 15:39:00 srv2 Keepalived_vrrp[31882]: Sending gratuitous ARP on enp0s8 for 192.168.1.254
1月 08 15:39:00 srv2 Keepalived_vrrp[31882]: Sending gratuitous ARP on enp0s8 for 192.168.1.254
1月 08 15:39:00 srv2 Keepalived_vrrp[31882]: Sending gratuitous ARP on enp0s8 for 192.168.1.254
# 查看keepalived进程
[root@srv2 sbin]# ps -ef |grep keepalived
root 31881 1 0 15:38 ? 00:00:00 /usr/local/keepalive/sbin/keepalived -D
root 31882 31881 0 15:38 ? 00:00:00 /usr/local/keepalive/sbin/keepalived -D
3、查看主库IP信息
[root@srv2 sbin]# ip add sh
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:6d:e6:1c brd ff:ff:ff:ff:ff:ff
inet 192.168.1.3/24 brd 192.168.1.255 scope global enp0s8
valid_lft forever preferred_lft forever
### 在主库网卡上已经绑定了vip（192.168.1.254）
4、启动备库数据库服务
[kingbase@srv1 ~]$ netstat -an |grep 54321
tcp 0 0 0.0.0.0:54321 0.0.0.0:* LISTEN
tcp 0 1 10.0.2.15:52367 192.168.2.3:54321 SYN_SENT
tcp6 0 0 :::54321 :::* LISTEN
unix 2 [ ACC ] STREAM LISTENING 116032 /tmp/.s.KINGBASE.54321
5、启动备库keepalived服务
[root@srv2 sbin]# systemctl enable keepalived
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to
[root@srv1 keepalived]# systemctl start keepalived
[root@srv1 keepalived]# systemctl status keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)
Active: active (running) since 五 2021-01-08 15:40:38 CST; 3s ago
Process: 16799 ExecStart=/usr/local/keepalive/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 16801 (keepalived)
Tasks: 2
CGroup: /system.slice/keepalived.service
├─16801 /usr/local/keepalive/sbin/keepalived -D
└─16802 /usr/local/keepalive/sbin/keepalived -D
1月 08 15:40:38 srv1 Keepalived_vrrp[16802]: SECURITY VIOLATION - scripts are being executed but script_sec...led.
1月 08 15:40:38 srv1 Keepalived_vrrp[16802]: (VI_1) Strict mode does not support authentication. Ignoring.
1月 08 15:40:38 srv1 Keepalived_vrrp[16802]: Assigned address 192.168.1.2 for interface enp0s8
1月 08 15:40:38 srv1 Keepalived_vrrp[16802]: Assigned address fe80::a00:27ff:fe9b:c439 for interface enp0s8
1月 08 15:40:38 srv1 Keepalived_vrrp[16802]: Registering gratuitous ARP shared channel
1月 08 15:40:38 srv1 Keepalived_vrrp[16802]: (VI_1) removing VIPs.
1月 08 15:40:38 srv1 Keepalived_vrrp[16802]: (VI_1) removing iptable drop rule
1月 08 15:40:38 srv1 Keepalived_vrrp[16802]: (VI_1) Entering BACKUP STATE (init)
1月 08 15:40:38 srv1 Keepalived_vrrp[16802]: VRRP sockpool: [ifindex(3), family(IPv4), proto(112), unicast(...11)]
1月 08 15:40:39 srv1 Keepalived_vrrp[16802]: VRRP_Script(chk_kes_port) succeeded
Hint: Some lines were ellipsized, use -l to show in full.
# 查看keepalived进程
[root@srv1 keepalived]# ps -ef |grep keepalived
root 13462 1 0 15:29 ? 00:00:00 /usr/local/keepalive/sbin/keepalived -D
root 13463 13462 0 15:29 ? 00:00:00 /usr/local/keepalive/sbin/keepalived -D
6、查看备库ip状态
[root@srv1 keepalived]# ip add sh
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:6d:e6:1c brd ff:ff:ff:ff:ff:ff
inet 192.168.1.2/24 brd 192.168.1.255 scope global enp0s8
valid_lft forever preferred_lft forever
四、测试主库数据库服务宕机vip切换
1、将主库数据库服务宕机
[root@srv2 sbin]# ps -ef |grep kingbase
kingbase 28843 1 0 15:32 pts/0 00:00:00 /home/kingbase/cluster/kdb/db/bin/kingbase -D /home/kingbase/cluster/kdb/db/data
kingbase 28888 28843 0 15:32 ? 00:00:00 kingbase: logger process
kingbase 28890 28843 0 15:32 ? 00:00:00 kingbase: checkpointer process
kingbase 28891 28843 0 15:32 ? 00:00:00 kingbase: writer process
kingbase 28892 28843 0 15:32 ? 00:00:00 kingbase: wal writer process
kingbase 28893 28843 0 15:32 ? 00:00:00 kingbase: autovacuum launcher process
kingbase 28894 28843 0 15:32 ? 00:00:00 kingbase: archiver process
kingbase 28895 28843 0 15:32 ? 00:00:00 kingbase: stats collector process
kingbase 28896 28843 0 15:32 ? 00:00:00 kingbase: bgworker: syslogical supervisor
[root@srv2 sbin]# kill -9 28843
# 数据库服务已经关闭
[root@srv2 sbin]# ps -ef |grep kingbase
2、查看主库IP状态信息（VIP已经漂移）
[root@srv2 sbin]# ip add sh
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:6d:e6:1c brd ff:ff:ff:ff:ff:ff
inet 192.168.1.3/24 brd 192.168.1.255 scope global enp0s8
valid_lft forever preferred_lft forever
3、keepalive服务被kill
[root@srv2 sbin]# ps -ef |grep keepalive
4、备库ip信息（vip漂移到备库：192.168.1.254）
[root@srv1 keepalived]# ip add sh
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:9b:c4:39 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.2/24 brd 192.168.1.255 scope global enp0s8
valid_lft forever preferred_lft forever
inet 192.168.1.254/32 scope global enp0s8
valid_lft forever preferred_lft forever
5、启动主库数据库服务和keepalive进程
1、主库数据库服务
[kingbase@srv2 bin]$ netstat -an |grep 54321
tcp 0 0 0.0.0.0:54321 0.0.0.0:* LISTEN
tcp6 0 0 :::54321 :::* LISTEN
unix 2 [ ACC ] STREAM LISTENING 92169 /tmp/.s.KINGBASE.54321
2、启动keepalived服务
[root@srv2 sbin]# systemctl enable keepalived
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.
[root@srv2 sbin]# systemctl start keepalived
[root@srv2 sbin]# systemctl status keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)
Active: active (running) since 五 2021-01-08 15:45:41 CST; 5s ago
Process: 2659 ExecStart=/usr/local/keepalive/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 2666 (keepalived)
Tasks: 2
CGroup: /system.slice/keepalived.service
├─2666 /usr/local/keepalive/sbin/keepalived -D
└─2667 /usr/local/keepalive/sbin/keepalived -D
1月 08 15:45:40 srv2 Keepalived_vrrp[2667]: (VI_1) Strict mode does not support authentication. Ignoring.
1月 08 15:45:40 srv2 Keepalived_vrrp[2667]: Assigned address 192.168.1.3 for interface enp0s8
1月 08 15:45:40 srv2 Keepalived_vrrp[2667]: Assigned address fe80::a00:27ff:fe6d:e61c for interface enp0s8
1月 08 15:45:40 srv2 Keepalived_vrrp[2667]: Registering gratuitous ARP shared channel
1月 08 15:45:40 srv2 Keepalived_vrrp[2667]: (VI_1) removing VIPs.
1月 08 15:45:40 srv2 Keepalived_vrrp[2667]: (VI_1) removing iptable drop rule
1月 08 15:45:40 srv2 Keepalived_vrrp[2667]: (VI_1) Entering BACKUP STATE (init)
1月 08 15:45:40 srv2 Keepalived_vrrp[2667]: VRRP sockpool: [ifindex(3), family(IPv4), proto(112), unicast(0...11)]
1月 08 15:45:40 srv2 Keepalived_vrrp[2667]: VRRP_Script(chk_kes_port) succeeded
1月 08 15:45:41 srv2 systemd[1]: Started LVS and VRRP High Availability Monitor.
Hint: Some lines were ellipsized, use -l to show in full.
3、查看主库IP状态信息
[root@srv2 sbin]# ip add sh
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:6d:e6:1c brd ff:ff:ff:ff:ff:ff
inet 192.168.1.3/24 brd 192.168.1.255 scope global enp0s8
valid_lft forever preferred_lft forever
---由以上来看，主库启动数据库服务和keepalived进程后，并没有将VIP抢占过来，因为在keepalived.conf配置为nopreempt模式。
五、总结
对于主备流复制集群，如果不支持failover切换机制，在原主库数据库服务down后，需要手工将备库提升（promote）新的主库，否则在vip漂移到备库后，将导致业务访问数据库（read-only）失败。在本案例中，keepalive只是对vip及vip切换管理，没有涉及到其他数据资源切换的管理。
【推荐】国内首个AI IDE，深度理解中文开发场景，立即下载体验Trae
【推荐】编程新体验，更懂你的AI，立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包，你的智能百科全书，全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell：AI 加持，快人一步
· 全程不用写代码，我用AI程序员写了一个飞机大战
· MongoDB 8.0这个新功能碉堡了，比商业数据库还牛
· 记一次.NET内存居高不下排查解决与启示
· 白话解读 Dapr 1.15：你的「微服务管家」又秀新绝活了
· DeepSeek 开源周回顾「GitHub 热点速览」
2022-01-12 KingbaseES V8R3集群运维案例之---在线删除数据节点案例
2022-01-12 KingbaseES V8R6集群运维案例之---sys_rman备份异机恢复案例