keepalived+LVS-DR模型高可用配置
1、LVS DR模型高可用
LVS 可以实现负载均衡功能,但是没有健康检查机制,如果一台 RS 节点故障, LVS 任然会将请求调度至该故障 RS 节点服务器;
那么我们可以使用 Keepalived 来实现解决: 1、使用 Keepalived 可以实现 LVS 的健康检查机制, RS 节点故障,则自动剔除该故障的 RS 节点,如果 RS 节点恢复则自动加入集群。 2、使用 Keeplaived 可以解决 LVS 单点故障,以此实现 LVS 的高可用。 3、可以理解 Keepalived 就是为 LVS 而诞生的。
2、环境准备
2.1、主机准备
keepalived-01+lvs 192.168.10.13 [vip 192.168.10.200] keepalived-02+lvs 192.168.10.14 [vip 192.168.10.200] rs-01 192.168.10.16 rs-02 192.168.10.15
2.2、安装LVS
这里不再重复记录,请参考文章:https://www.cnblogs.com/ygbh/p/17370813.html
2.3、安装keepalived
这里不再重复记录,请参考文章:https://www.cnblogs.com/ygbh/p/17373985.html
2.4、rs主机安装与配置
2.4.1、安装httpd
这里不再重复记录,请参考文章:https://www.cnblogs.com/ygbh/p/17371925.html#_lab2_1_4
2.4.2、将rs-01主机的网关指向vip地址
rs-01 ~]# vi /etc/sysconfig/network-scripts/ifcfg-ens36 TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=static DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=ens36 DEVICE=ens36 ONBOOT=yes IPADDR=192.168.10.16 NETMASK=255.255.255.0 GATEWAY=192.168.10.200
2.4.3、将rs-02主机的网关指向vip地址
rs-02 ~]# vi /etc/sysconfig/network-scripts/ifcfg-ens36 TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=static DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=ens36 DEVICE=ens36 ONBOOT=yes IPADDR=192.168.10.15 PREFIX=24 GATEWAY=192.168.10.200
2.4.4、rs-01、rs-02配置【ifcfg-lo:0】
cat >/etc/sysconfig/network-scripts/ifcfg-lo:0<<'EOF'
DEVICE=lo:0
IPADDR=192.168.87.200
NETMASK=255.255.255.255
ONBOOT=yes
NAME=loopback
EOF
systemctl restart network
2.4.5、禁止本机的ARP请求【rs-01、rs-02配置】
# 配置 arp ,不对外宣告本机 VIP 地址,也不响应其他节点发起 ARP 请求 本机的VIP
# 【临时设置,重启将会失效】
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
# 【重启后也生效的配置方法】
cat >> /etc/sysctl.conf << 'EOF'
net.ipv4.conf.all.arp_announce=2
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.lo.arp_announce=2
net.ipv4.conf.lo.arp_ignore=1
EOF
sysctl -p
3、keepalived配置
3.1、Master节点配置
cat << 'CAT_END' >/usr/local/keepalived/etc/keepalived/keepalived.conf global_defs { router_id kp01 } vrrp_instance VI_1 { state MASTER interface ens33 virtual_router_id 50 priority 150 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.10.200 } } virtual_server 192.168.10.200 80 { delay_loop 6 lb_algo wlc lb_kind DR perssisstence_timeout 5 protocol TCP real_server 192.168.10.16 80 { weight 1 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 2 delay_beefore_retry 3 } } real_server 192.168.10.15 80 { weight 1 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 2 delay_beefore_retry 3 } } } CAT_END
3.2、Backup节点配置
cat << 'CAT_END' >/usr/local/keepalived/etc/keepalived/keepalived.conf global_defs { router_id kp02 } vrrp_instance VI_1 { state BACKUP interface ens33 virtual_router_id 50 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.10.200 } } virtual_server 192.168.10.200 80 { delay_loop 6 lb_algo wlc lb_kind DR perssisstence_timeout 5 protocol TCP real_server 192.168.10.16 80 { weight 1 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 2 delay_beefore_retry 3 } } real_server 192.168.10.15 80 { weight 1 TCP_CHECK { connect_port 80 connect_timeout 3 nb_get_retry 2 delay_beefore_retry 3 } } } CAT_END
3.3、virtual_server属性解析
virtual_server 192.168.10.200 80 { # 配置集群地址访问的IP+Port delay_loop 6 # 健康检查的时间,单位:秒 lb_algo wlc # 配置负载均衡的算法 lb_kind DR # 设置LVS的模式 NAT|TUN|DR perssisstence_timeout 5 # 设置会话持久化的时间 protocol TCP # 设置协议 real_server 192.168.10.16 80 { # 负载均衡后端的真实服务节点RS-1 weight 1 # 权重配比设置为1 TCP_CHECK { # 设置健康检查 connect_port 80 # 检测后端80端口 connect_timeout 3 # 超时时间 nb_get_retry 2 # 重试次数2次 delay_beefore_retry 3 # 间隔时间3s } } real_server 192.168.10.15 80 { # 负载均衡后端的真实服务节点RS-2 weight 1 # 权重配比设置为1 TCP_CHECK { # 设置健康检查 connect_port 80 # 检测后端80端口 connect_timeout 3 # 超时时间 nb_get_retry 2 # 重试次数2次 delay_beefore_retry 3 # 间隔时间3s } } }
4、重启keepalived并且测试
4.1、重启keepalived
systemctl restart keepalived
4.2、虚拟IP地址是否开启
keepalived-01 ~]# ip addr | grep 200 inet 192.168.10.200/32 scope global ens33
4.3、查询ipvsadm规则
4.3.1、keepalived-01
keepalived-01 ~]# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.10.200:80 wlc -> 192.168.10.15:80 Route 1 0 0 -> 192.168.10.16:80 Route 1 0 0
4.3.2、keepalived-02
keepalived-02 ~]# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.10.200:80 wlc -> 192.168.10.15:80 Route 1 0 0 -> 192.168.10.16:80 Route 1 0 0
4.4、访问VIP测试
~]# curl 192.168.10.200:80 rs-02 ~]# curl 192.168.10.200:80 rs-01
5、高可用架构测试
5.1、关闭master检查VIP是否漂移到Backup【验证VIP漂移】
keepalived-01 ~]# systemctl stop keepalived keepalived-02 ~]# ip addr | grep 200 inet 192.168.10.200/32 scope global ens33 # VIP漂移成功至backup
5.2、 模拟一台Real Server 故障, keeplaived 检测后会自动将节点移除【验证rs健康检查】
5.2.1、关闭httpd服务
rs-01 ~]# systemctl stop httpd
5.2.2、查询ipvsadm规则是否自动踢除故障的rs
keepalived-01 ~]# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.10.200:80 wlc -> 192.168.10.15:80 Route 1 0 0 # 发现已经踢除掉故障的rs
5.2.3、恢复故障的rs
rs-01 ~]# systemctl start httpd keepalived-01 ~]# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.10.200:80 wlc -> 192.168.10.15:80 Route 1 0 0 -> 192.168.10.16:80 Route 1 0 0 # 自动增加rs节点