基于LVS负载均衡群集来实现Keepalived的部署
摘要
Keepalived的作用是检测服务器的状态，如果有一台web服务器宕机，或工作出现故障，Keepalived将检测到，并将有故障的服务器从系统中剔除，同时使用其他服务器代替该服务器的工作，当服务器工作正常后Keepalived自动将服务器加入到服务器群中，这些工作全部自动完成，不需要人工干涉，需要人工做的只是修复故障的服务器
一、Keepalived案例分析
1.1、企业应用中，单台服务器承担应用存在单点故障的危险
1.2、单点故障一旦发生，企业服务将发生中断，造成极大的危害
二、Keepalived工具介绍
专为LVS和HA设计的一款健康检查工具
①支持故障自动切换（Failover）
②支持节点健康状态检查（Health Checking）
③官方网站：http://www.keepalived.org/
三、Keepalived实现原理剖析
3.1、Keepalived采用VRRP热备份协议实现Linux服务器的多机热备功能
3.2、VRRP（虚拟路由冗余协议）是针对路由器的一种备份解决方案
①由多台路由器组成一个热备组，通过共用的虚拟IP地址对外提供服务
②每个热备组内同时只有一台主路由器提供服务，其他路由器处于冗余状态
③若当前在线的路由器失效，则其他路由器会根据设置的优先级自动接替虚拟IP地址，继续提供服务
四、Keepalived案例讲解
4.1、双机热备的故障切换是由虚拟IP地址来实现，适用于各种应用服务器
4.2、实现基于Web服务的双机热备
①漂移地址：192.168.100.100
②主、备服务器：192.168.100.10、192.168.100.50
③提供的应用服务：Web
4.3、环境(基于LVS-DR进行搭建)
|
虚拟IP |
192.168.100.100/24 |
|
主调度器 |
192.168.100.10/24 |
|
备调度器 |
192.168.100.40/24 |
|
Web1服务器 |
192.168.100.20/24 |
|
Web2服务器 |
192.168.100.30/24 |
|
NFS共享服务器 |
192.168.100.50/24 |
|
客户机一台 |
用于测试验证 |
4.4、配置主调度器（192.168.100.10）
1 加载ip_vs模块
2 [root@lvs-zhu ~]# modprobe ip_vs
3 [root@lvs-zhu ~]# cat /proc/net/ip_vs
4 IP Virtual Server version 1.2.1 (size=4096)
5 Prot LocalAddress:Port Scheduler Flags
6 -> RemoteAddress:Port Forward Weight ActiveConn InActConn
7
8 安装管理软件ipvsadm
9 [root@lvs-zhu ~]# rpm -ivh /mnt/Packages/ipvsadm-1.27-7.el7.x86_64.rpm
10
11 安装编译工具
12 [root@lvs-zhu ~]# yum -y install gcc gcc-c++ make popt-devel openssl-devel kernel-devel
13
14 解压缩，编译安装
15 [root@lvs-zhu ~]# tar zxf keepalived-2.0.13.tar.gz
16 [root@lvs-zhu ~]# cd keepalived-2.0.13/
17 [root@lvs-zhu keepalived-2.0.13]# ./configure --prefix=/
18 [root@lvs-zhu keepalived-2.0.13]# make && make install
19
20 加入系统管理服务，设置开机自启
21 [root@lvs-zhu keepalived-2.0.13]# cp keepalived/etc/init.d/keepalived /etc/init.d/
22 [root@lvs-zhu keepalived-2.0.13]# systemctl enable keepalived.service
23
24 编辑配置文件
25 [root@lvs-zhu keepalived-2.0.13]# vi /etc/keepalived/keepalived.conf
26 ! Configuration File for keepalived
27 global_defs {
28 router_id LVS_01 #本服务器的名称
29 }
30 vrrp_instance VI_1 { #定义VRRP热备实例
31 state MASTER #热备状态，MASTER表示主服务器，BACKUP表示从服务器
32 interface ens33 #承载VIP地址的物理接口
33 virtual_router_id 51 #虚拟路由器的ID号，每个热备组保持一致
34 priority 110 #优先级，数值越大优先级越高
35 advert_int 1 #通告间隔秒数（心跳频率）
36 authentication { #热备认证信息，每个热备组保持一致
37 auth_type PASS #认证类型
38 auth_pass 6666 #密码字符串
39 }
40 virtual_ipaddress { #指定飘逸地址（VIP），可以有多个
41 192.168.100.100
42 }
43 }
44 virtual_server 192.168.100.100 80 { #虚拟服务器地址（VIP）、端口
45 delay_loop 6 #健康检查的间隔时间（秒）
46 lb_algo rr #轮询（rr）调度算法
47 lb_kind DR #直接路由（DR）群集工作模式
48 persistence_timeout 6 #连接保持时间（秒）
49 protocol TCP #应用服务器采用的是TCP协议
50 real_server 192.168.100.20 80 { #第一个web服务器节点的地址、端口
51 weight 1 #节点的权重
52 TCP_CHECK { #健康检查方式
53 connect_port 80 #检查的目标端口
54 connect_timeout 3 #连接超时（秒）
55 nb_get_retry 3 #重试次数
56 delay_before_retry 3 #重试间隔
57 }
58 }
59 real_server 192.168.100.30 80 {
60 weight 1
61 TCP_CHECK {
62 connect_port 80
63 connect_timeout 3
64 nb_get_retry 3
65 delay_before_retry 3
66 }
67 }
68 }
69
70 开启服务
71 [root@lvs-zhu keepalived-2.0.13]# systemctl start keepalived.service
72 [root@lvs-zhu keepalived-2.0.13]# tail -f /var/log/messages
73 [root@lvs-zhu keepalived-2.0.13]# ipvsadm -Ln
74 IP Virtual Server version 1.2.1 (size=4096)
75 Prot LocalAddress:Port Scheduler Flags
76 -> RemoteAddress:Port Forward Weight ActiveConn InActConn
77 TCP 192.168.100.100:80 rr persistent 6
78 -> 192.168.100.20:80 Route 1 0 0
79 -> 192.168.100.30:80 Route 1 0 0
80 [root@lvs-zhu keepalived-2.0.13]# ip addr show dev ens33
81 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
82 link/ether 00:0c:29:9a:cd:27 brd ff:ff:ff:ff:ff:ff
83 inet 192.168.100.10/24 brd 192.168.100.255 scope global ens33
84 valid_lft forever preferred_lft forever
85 inet 192.168.100.100/32 scope global ens33
86 valid_lft forever preferred_lft forever
87 inet6 fe80::26b5:ebd3:a0d2:db12/64 scope link
88 valid_lft forever preferred_lft forever
4.5、配置从调度器（192.168.100.40）
1 加载ip_vs模块
2 [root@lvs-bei ~]# modprobe ip_vs
3 [root@lvs-bei ~]# cat /proc/net/ip_vs
4 IP Virtual Server version 1.2.1 (size=4096)
5 Prot LocalAddress:Port Scheduler Flags
6 -> RemoteAddress:Port Forward Weight ActiveConn InActConn
7
8 安装管理软件ipvsadm
9 [root@lvs-bei ~]# rpm -ivh /mnt/Packages/ipvsadm-1.27-7.el7.x86_64.rpm
10
11 安装编译工具
12 [root@lvs-bei ~]# yum -y install gcc gcc-c++ make popt-devel openssl-devel kernel-devel
13
14 解压缩，编译安装
15 [root@lvs-bei ~]# tar zxf keepalived-2.0.13.tar.gz
16 [root@lvs-bei ~]# cd keepalived-2.0.13/
17 [root@lvs-bei keepalived-2.0.13]# ./configure --prefix=/
18 [root@lvs-bei keepalived-2.0.13]# make && make install
19
20 加入系统管理服务，设置开机自启
21 [root@lvs-bei keepalived-2.0.13]# cp keepalived/etc/init.d/keepalived /etc/init.d/
22 [root@lvs-bei keepalived-2.0.13]# systemctl enable keepalived.service
23
24 编辑配置文件
25 [root@lvs-bei keepalived-2.0.13]# vi /etc/keepalived/keepalived.conf
26 ! Configuration File for keepalived
27 global_defs {
28 router_id LVS_02 #本服务器的名称
29 }
30 vrrp_instance VI_1 { #定义VRRP热备实例
31 state BACKUP #热备状态，MASTER表示主服务器，BACKUP表示从服务器
32 interface ens33 #承载VIP地址的物理接口
33 virtual_router_id 51 #虚拟路由器的ID号，每个热备组保持一致
34 priority 105 #优先级，数值越大优先级越高
35 advert_int 1 #通告间隔秒数（心跳频率）
36 authentication { #热备认证信息，每个热备组保持一致
37 auth_type PASS #认证类型
38 auth_pass 6666 #密码字符串
39 }
40 virtual_ipaddress { #指定飘逸地址（VIP），可以有多个
41 192.168.100.100
42 }
43 }
44 virtual_server 192.168.100.100 80 { ＃虚拟服务器地址（VIP）、端口
45 delay_loop 6 #健康检查的间隔时间（秒）
46 lb_algo rr #轮询（rr）调度算法
47 lb_kind DR #直接路由（DR）群集工作模式
48 persistence_timeout 6 #连接保持时间（秒）
49 protocol TCP #应用服务器采用的是TCP协议
50 real_server 192.168.100.20 80 { #第一个web服务器节点的地址、端口
51 weight 1 #节点的权重
52 TCP_CHECK { #健康检查方式
53 connect_port 80 #检查的目标端口
54 connect_timeout 3 #连接超时（秒）
55 nb_get_retry 3 #重试次数
56 delay_before_retry 3 #重试间隔
57 }
58 }
59 real_server 192.168.100.30 80 {
60 weight 1
61 TCP_CHECK {
62 connect_port 80
63 connect_timeout 3
64 nb_get_retry 3
65 delay_before_retry 3
66 }
67 }
68 }
69
70 开启服务
71 [root@lvs-bei keepalived-2.0.13]# systemctl start keepalived.service
72 [root@lvs-bei keepalived-2.0.13]# tail -f /var/log/messages
73 [root@lvs-bei keepalived-2.0.13]# ipvsadm -Ln
74 IP Virtual Server version 1.2.1 (size=4096)
75 Prot LocalAddress:Port Scheduler Flags
76 -> RemoteAddress:Port Forward Weight ActiveConn InActConn
77 TCP 192.168.100.100:80 rr
78 -> 192.168.100.20:80 Route 1 0 0
79 -> 192.168.100.30:80 Route 1 0 0
80 [root@lvs-bei keepalived-2.0.13]# ip addr show dev ens33
81 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
82 link/ether 00:0c:29:dc:10:18 brd ff:ff:ff:ff:ff:ff
83 inet 192.168.100.40/24 brd 192.168.100.255 scope global ens33
84 valid_lft forever preferred_lft forever
85 inet6 fe80::c1f0:d588:3477:d684/64 scope link
86 valid_lft forever preferred_lft forever
87
88 ####由于设置了连接保持时间为60秒，一分钟后重新再访问该地址自动轮询交给另一台Web服务器
4.6、配置NFS服务器（192.168.100.50）
1 安装NFS服务
2 [root@nfs-server ~]# yum -y install rpcbind nfs-utils
3
4 创建测试目录并新建网页
5 [root@nfs-server ~]# mkdir -p /opt/web1
6 [root@nfs-server ~]# mkdir -p /opt/web2
7 [root@nfs-server ~]# echo '<h1>this is web 1!</h1>' > /opt/web1/index.html
8 [root@nfs-server ~]# echo '<h1>this is web 2!</h1>' > /opt/web2/index.html
9
10 设置访问权限
11 [root@nfs-server ~]# vi /etc/exports
12 /opt/web1 192.168.100.20(ro)
13 /opt/web2 192.168.100.30(ro)
14
15 开启服务并开机自启
16 [root@nfs-server ~]# systemctl start nfs
17 [root@nfs-server ~]# systemctl start rpcbind
18 [root@nfs-server ~]# systemctl enable nfs
19 [root@nfs-server ~]# systemctl enable rpcbind
20
21 查看访问权限
22 [root@nfs-server ~]# showmount -e
23 Export list for nfs-server:
24 /opt/web2 192.168.100.30
25 /opt/web1 192.168.100.20
4.7、配置Web1服务器（192.168.100.20）
1 安装httpd服务
2 [root@web1 ~]# yum -y install httpd
3
4 挂载NFS服务
5 [root@web1 ~]# mount 192.168.100.50:/opt/web1 /var/www/html
6
7 查看权限
8 [root@web1 ~]# showmount -e 192.168.100.50
9 Export list for 192.168.100.50:
10 /opt/web2 192.168.100.30
11 /opt/web1 192.168.100.20
12
13 开启httpd服务
14 [root@web1 ~]# systemctl start httpd
15
16 本机访问
17 [root@web1 ~]# curl http://localhost
18 <h1>this is web1!</h1>
19
20 配置DR模式
21 [root@web1 ~]# vi web1.sh
22 #!/bin/bash
23 # web1
24 ifconfig lo:0 192.168.100.100 broadcast 192.168.100.100 netmask 255.255.255.255 up
25 route add -host 192.168.100.100 dev lo:0
26 echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore
27 echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore
28 echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce
29 echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce
30 sysctl -p &> /dev/null
31
32 执行脚本
33 [root@web1 ~]# sh web1.sh
34
35 查看端口信息
36 [root@web1 ~]# ifconfig
37 ......
38 lo:0: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
39 inet 192.168.100.100 netmask 255.255.255.255
40 loop txqueuelen 1 (Local Loopback)
41 ......
4.8、配置web2服务器（192.168.100.30）
1 安装httpd服务
2 [root@web2 ~]# yum -y install httpd
3
4 挂载NFS服务
5 [root@web2 ~]# mount 192.168.100.50:/opt/web2 /var/www/html
6
7 查看权限
8 [root@web2 ~]# showmount -e 192.168.100.50
9 Export list for 192.168.100.50:
10 /opt/web2 192.168.100.30
11 /opt/web1 192.168.100.20
12
13 开启httpd服务
14 [root@web2 ~]# systemctl start httpd
15
16 本机访问
17 [root@web2 ~]# curl http://localhost
18 <h1>this is web2!</h1>
19
20 配置DR模式
21 [root@web2 ~]# vi web2.sh
22 #!/bin/bash
23 # web2
24 ifconfig lo:0 192.168.100.100 broadcast 192.168.100.100 netmask 255.255.255.255 up
25 route add -host 192.168.100.100 dev lo:0
26 echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore
27 echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore
28 echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce
29 echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce
30 sysctl -p &> /dev/null
31
32 执行脚本
33 [root@web2 ~]# sh web2.sh
34
35 查看端口信息
36 [root@web2 ~]# ifconfig
37 ......
38 lo:0: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
39 inet 192.168.100.100 netmask 255.255.255.255
40 loop txqueuelen 1 (Local Loopback)
41 ......
4.9、测试web1和web2的IP访问
4.10、测试虚拟IP地址级查看状态
1 [root@lvs-zhu keepalived-2.0.13]# ipvsadm -Ln
2 IP Virtual Server version 1.2.1 (size=4096)
3 Prot LocalAddress:Port Scheduler Flags
4 -> RemoteAddress:Port Forward Weight ActiveConn InActConn
5 TCP 192.168.100.100:80 rr
6 -> 192.168.100.20:80 Route 1 1 1
7 -> 192.168.100.30:80 Route 1 1 0
4.10、模拟主调度器故障，验证结果
1 关闭主调度器keepalived
2 [root@lvs-zhu keepalived-2.0.13]# systemctl stop keepalived.service
3
4 查看从调度器状态
5 [root@lvs-bei keepalived-2.0.13]# tail -f /var/log/messages
6 [root@lvs-bei keepalived-2.0.13]# ipvsadm -Ln
7 IP Virtual Server version 1.2.1 (size=4096)
8 Prot LocalAddress:Port Scheduler Flags
9 -> RemoteAddress:Port Forward Weight ActiveConn InActConn
10 TCP 192.168.100.100:80 rr
11 -> 192.168.100.20:80 Route 1 0 0
12 -> 192.168.100.30:80 Route 1 0 0
13 [root@lvs-bei keepalived-2.0.13]# ip addr show dev ens33
14 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
15 link/ether 00:0c:29:dc:10:18 brd ff:ff:ff:ff:ff:ff
16 inet 192.168.100.40/24 brd 192.168.100.255 scope global ens33
17 valid_lft forever preferred_lft forever
18 inet 192.168.100.100/32 scope global ens33
19 valid_lft forever preferred_lft forever #虚拟地址漂移到备调度器上
20 inet6 fe80::c1f0:d588:3477:d684/64 scope link
21 valid_lft forever preferred_lft forever
查看备调度器连接状态
1 [root@lvs-bei keepalived-2.0.13]# ipvsadm -Ln
2 IP Virtual Server version 1.2.1 (size=4096)
3 Prot LocalAddress:Port Scheduler Flags
4 -> RemoteAddress:Port Forward Weight ActiveConn InActConn
5 TCP 192.168.100.100:80 rr
6 -> 192.168.100.20:80 Route 1 0 1
7 -> 192.168.100.30:80 Route 1 2 0
4.11、开启主调度器并查看状态
1 [root@lvs-zhu keepalived-2.0.13]# systemctl start keepalived.service
2 [root@lvs-zhu keepalived-2.0.13]# tail -f /var/log/messages
3 [root@lvs-zhu keepalived-2.0.13]# ipvsadm -Ln
4 IP Virtual Server version 1.2.1 (size=4096)
5 Prot LocalAddress:Port Scheduler Flags
6 -> RemoteAddress:Port Forward Weight ActiveConn InActConn
7 TCP 192.168.100.100:80 rr
8 -> 192.168.100.20:80 Route 1 0 0
9 -> 192.168.100.30:80 Route 1 0 0
10 [root@lvs-zhu keepalived-2.0.13]# ip addr show dev ens33
11 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
12 link/ether 00:0c:29:9a:cd:27 brd ff:ff:ff:ff:ff:ff
13 inet 192.168.100.10/24 brd 192.168.100.255 scope global ens33
14 valid_lft forever preferred_lft forever
15 inet 192.168.100.100/32 scope global ens33
16 valid_lft forever preferred_lft forever #虚拟地址又到主调度器上
17 inet6 fe80::26b5:ebd3:a0d2:db12/64 scope link
18 valid_lft forever preferred_lft forever
4.12、模拟web服务器故障，查看状态
①关闭web1的httpd服务
1 [root@web1 ~]# systemctl stop httpd
②测试网页，只能查看web2服务器的网页
③查看调度器节点状态
1 [root@lvs-zhu keepalived-2.0.13]# ipvsadm -Ln
2 IP Virtual Server version 1.2.1 (size=4096)
3 Prot LocalAddress:Port Scheduler Flags
4 -> RemoteAddress:Port Forward Weight ActiveConn InActConn
5 TCP 192.168.100.100:80 rr
6 -> 192.168.100.30:80 Route 1 1 2
④开启Web1服务又可以轮询了
1 [root@lvs-zhu keepalived-2.0.13]# ipvsadm -Ln
2 IP Virtual Server version 1.2.1 (size=4096)
3 Prot LocalAddress:Port Scheduler Flags
4 -> RemoteAddress:Port Forward Weight ActiveConn InActConn
5 TCP 192.168.100.100:80 rr
6 -> 192.168.100.20:80 Route 1 2 0
7 -> 192.168.100.30:80 Route 1 3 1
总结
通过本篇文章学习了Keepalived的群集部署，故障时是如何自动切换以及节点健康检查的方式
