LVS粗讲
该文章不对 LVS 和 keepalived 的理论展开介绍，本次做的是拓扑中红色虚线中的部分
主机配置
lvs/172.16.186.111/1G/NAT/1U/80G/VIP:172.16.186.109/CentOS7.9.2009
web1/172.16.186.112/1G/NAT/1U/80G/VIP:172.16.186.109/CentOS7.9.2009
web2/172.16.186.113/1G/NAT/1U/80G/VIP:172.16.186.109/CentOS7.9.2009
keepalived/172.16.186.114/NAT/1G/1U/80G/VIP:172.16.186.109/CentOS7.9.2009
lvs端设置
lvs端网络设置
[root@lvs ~]# systemctl stop firewalld && systemctl disable firewalld
[root@lvs ~]# setenforce 0
[root@lvs ~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
[root@lvs ~]# systemctl stop NetworkManager && systemctl disable NetworkManager
[root@lvs ~]# cd /etc/sysconfig/network-scripts/
[root@lvs network-scripts]# cp ifcfg-ens33{,:0} ===> 在物理网卡上开一个子接口，0可自定义
[root@lvs network-scripts]# vim ifcfg-ens33:0 ===> 配置子接口
TYPE="Ethernet"
BOOTPROTO="static"
NAME="ens33:0"
DEVICE="ens33:0" ===> 这里和子接口文件名匹配
ONBOOT="yes"
IPADDR="172.16.186.109" ===> 这里写vip
PREFIX="24" ===> 掩码必须写
[root@lvs network-scripts]# cd
[root@lvs ~]# systemctl restart network
[root@lvs ~]# ifconfig ===> 使用ip a查看网卡时186.109的地址会在ens33网卡配置中
安装lvs集群管理工具
[root@lvs ~]# mount /dev/sr0 /mnt/usb1 ===>挂载光盘
[root@lvs ~]# yum -y install /mnt/usb1/Packages/ipvsadm-1.27-8.el7.x86_64.rpm ===> 未有互联网时使用该方式，有互联网可直接yum安装
配置web端
所有web端网络配置相同
web1端网络设置
[root@web* ~]# systemctl stop firewalld && systemctl disable firewalld
[root@web* ~]# setenforce 0
[root@web* ~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
[root@web* ~]# systemctl stop NetworkManager && systemctl disable NetworkManager
[root@web* ~]# cd /etc/sysconfig/network-scripts/
[root@web* network-scripts]# cp ifcfg-lo{,:0} ===> 在回环口上开一个子接口
[root@web* network-scripts]# vim ifcfg-lo:0 ===> 配置回环子接口
DEVICE=lo:0 ===> 名字要改
IPADDR=172.16.186.109 ===> vip地址
NETMASK=255.255.255.255 ===> 全部是255,表示自己是一个网段
NETWORK=127.0.0.0
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback
启动lo:0网卡
[root@web* network-scripts]# ifup lo:0
[root@web* network-scripts]# ifconfig ===> 使用ip a查看时，子接口地址在lo网卡中
所有web端设置arp相应级别
arp-ignore：arp响应级别(是否接受外来的请求)
0：只要本机配置了ip就能响应请求
1：请求的目标地址到达对应的网络接口才会响应请求
arp-announce：arp通过行为(是否主动向外宣告自己)
0：本机上任何网络接口都向外通告，所有的网卡都能接受到通告
1：尽可能避免网卡与不匹配的目标进行通过
2：只在本网卡通告
开始设置(所有web端设置相同)
[root@web* ~]# vim /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
[root@web* ~]# sysctl -p
添加临时路由
作用：会接收到lvs集群IP(186.109)的数据报文，接收到后会交给lo:0网卡处理
[root@web* ~]# route add -host 172.16.186.109 dev lo:0
注：如没有route命令可用yum安装net-tools工具
[root@web* ~]# route -n | grep "186.109"
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 172.16.186.2 0.0.0.0 UG 100 0 0 ens33
172.16.186.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
172.16.186.109 0.0.0.0 255.255.255.255 UH 0 0 0 lo ===> 所有来自186.109的请求都送到lo:0网卡处理
永久添加路由
[root@web* ~]# echo "route add -host 172.16.186.109 dev lo:0">>/etc/rc.local
安装httpd
[root@web* ~]# yum -y install httpd
[root@web1 ~]# echo "<h1>This is web1</h1>" >> /var/www/html/index.html
[root@web2 ~]# echo "<h1>This is web2</h1>" >> /var/www/html/index.html
[root@web* ~]# systemctl start httpd && systemctl enable httpd
配置IPVS集群
创建ipvs集群
[root@lvs ~]# ipvsadm -A -t 172.16.186.109:80 -s rr ===> rr 为轮询模式
往集群中添加节点
[root@lvs ~]# ipvsadm -a -t 172.16.186.109:80 -r 172.16.186.112:80 -g ===> -g是dr模式，-m是NAT模式，-i是tun模式
[root@lvs ~]# ipvsadm -a -t 172.16.186.109:80 -r 172.16.186.113:80 -g
查看集群情况
[root@lvs ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.186.109:80 rr
-> 172.16.186.112:80 Route 1 0 0
-> 172.16.186.113:80 Route 1 0 0
查看集群状态
[root@lvs ~]# ipvsadm -ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 172.16.186.109:80 4 33 0 6617 0
-> 172.16.186.112:80 2 17 0 3188 0
-> 172.16.186.113:80 2 16 0 3429 0
注：OutPkts、OutBytes这两列为数据包出去时的记录，这里看到出去时没有经过lvs，lvs集群模式为dr
非集群内的且和集群中同一个网段的IP使访问186.109进行测试
http://172.16.186.109
注：多次访问集群地址，但一直是访问的是后台的一台机器，并没有对后端的2台机器进行轮询，这是因为lvs的持久化机制，默认持久化时间是300秒，可使用下面命令进行查询详情
[root@lvs ~]# man ipvsadm | grep -w "\-\-persistent"
可修改持久的时间(该操作仅限于看效果，生产中不用设置)
[root@lvs ~]# ipvsadm -E -t 172.16.186.109:80 -s rr -p 3 ===>设置持久时间为3秒
再次查看集群情况
[root@lvs ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.186.109:80 rr persistent 3 ====>这里已经将持久时间进行了改变
-> 172.16.186.112:80 Route 1 0 1
-> 172.16.186.113:80 Route 1 0 0
再次访问集群地址发现还是不行，这是因为还有一个tcp/udp的超时时间未设置
[root@lvs ~]# man ipvsadm
--set tcp tcpfin udp
Change the timeout values used for IPVS connections. This command always takes 3 parameters, representing the timeout val‐
ues (in seconds) for TCP sessions, TCP sessions after receiving a FIN packet, and UDP packets, respectively. A timeout
value 0 means that the current timeout value of the corresponding entry is preserved.
设置tcp/udp的超时时间
[root@lvs ~]# ipvsadm --set 1 1 1 ===> 分别表示TCP会话时间、接收FIN数据包后的TCP会话时间、UDP数据包的ues时间
查看超时时间
[root@lvs ~]# ipvsadm -lnc
IPVS connection entries
pro expire state source virtual destination
注：这里还没有请求，这时刷新一次web页面(是刷新一次)，再查看超时的情况会得到如下回显
[root@lvs ~]# ipvsadm -lnc
IPVS connection entries
pro expire state source virtual destination
TCP 00:00 NONE 172.16.186.1:0 172.16.186.109:80 172.16.186.112:80
注：expire列是超时时间，因为设置的TCP会话、接收FIN数据包后的TCP会话、UDP数据包的ues都是1秒所以这里会很快超时，进而expire显示的为0，再次ipvsadm -lnc查看即显示为空
如在web页面上刷新了n次则expire显示的时间依然是1秒后失效，如执行下面命令时且不断刷新页面会得到类似下面的回显
[root@lvs ~]# for (( i = 0; i < 20; i=(i+1) ));do ipvsadm -lnc && sleep 1;done
IPVS connection entries
pro expire state source virtual destination
TCP 00:59 NONE 172.16.186.1:0 172.16.186.109:80 172.16.186.112:80
IPVS connection entries ===> 第一个周期
pro expire state source virtual destination
TCP 00:58 NONE 172.16.186.1:0 172.16.186.109:80 172.16.186.112:80
IPVS connection entries ===> 第二个周期
pro expire state source virtual destination
TCP 00:59 NONE 172.16.186.1:0 172.16.186.109:80 172.16.186.112:80
TCP 00:00 ESTABLISHED 172.16.186.1:44234 172.16.186.109:80 172.16.186.112:80 ===> 1秒内新建了2个连接
TCP 00:00 ESTABLISHED 172.16.186.1:44236 172.16.186.109:80 172.16.186.112:80
IPVS connection entries ===> 第三个周期
pro expire state source virtual destination
TCP 00:58 NONE 172.16.186.1:0 172.16.186.109:80 172.16.186.112:80
TCP 00:00 ESTABLISHED 172.16.186.1:44234 172.16.186.109:80 172.16.186.112:80
IPVS connection entries ===> 第四个周期...
pro expire state source virtual destination
TCP 00:57 NONE 172.16.186.1:0 172.16.186.109:80 172.16.186.112:80
TCP 00:00 ESTABLISHED 172.16.186.1:44234 172.16.186.109:80 172.16.186.112:80
IPVS connection entries
pro expire state source virtual destination
TCP 00:56 NONE 172.16.186.1:0 172.16.186.109:80 172.16.186.112:80
TCP 00:00 ESTABLISHED 172.16.186.1:44234 172.16.186.109:80 172.16.186.112:80
IPVS connection entries
pro expire state source virtual destination
TCP 00:55 NONE 172.16.186.1:0 172.16.186.109:80 172.16.186.112:80
TCP 00:00 ESTABLISHED 172.16.186.1:44234 172.16.186.109:80 172.16.186.112:80
IPVS connection entries
pro expire state source virtual destination
TCP 00:54 NONE 172.16.186.1:0 172.16.186.109:80 172.16.186.112:80
TCP 00:00 ESTABLISHED 172.16.186.1:44234 172.16.186.109:80 172.16.186.112:80
IPVS connection entries
pro expire state source virtual destination
TCP 00:53 NONE 172.16.186.1:0 172.16.186.109:80 172.16.186.112:80
TCP 00:00 ESTABLISHED 172.16.186.1:44234 172.16.186.109:80 172.16.186.112:80
IPVS connection entries
pro expire state source virtual destination
TCP 00:52 NONE 172.16.186.1:0 172.16.186.109:80 172.16.186.112:80
注：因为设置了TCP会话、接收FIN数据包后的TCP会话、UDP数据包的ues都为1秒,也就是说一个连接中TCP会话、接收FIN数据包后的TCP会话、UDP数据包的ues都处理完毕后才会进行重新调度，即约为4秒一轮的重新调度
keepalived安装
lvs端(keepalived主)配置keepalived
[root@lvs ~]# mkdir /usr/local/keepalived
[root@lvs ~]# yum -y install gcc gcc-c++ openssl-devel libnl libnl-devel
[root@lvs ~]# wget https://www.keepalived.org/software/keepalived-2.2.2.tar.gz
[root@lvs ~]# tar -zxvf keepalived-2.2.2.tar.gz
[root@lvs ~]# cd keepalived-2.2.2
[root@keepalived keepalived-2.2.2]# ./configure --prefix=/usr/local/keepalived --with-init=SYSV
[root@keepalived keepalived-2.2.2]# make && make install
# 以下都使用的绝对路径就不带主机名了
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig
cp /usr/local/keepalived/sbin/keepalived /usr/sbin
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived.conf
chmod +x /etc/init.d/keepalived
chkconfig --level 2345 keepalived on
vim /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-f /etc/keepalived.conf -D -S 0"
配置keepalived
[root@keepalived keepalived-2.2.2]# cp /etc/keepalived.conf{,.bak}
[root@keepalived keepalived-2.2.2]# vim /etc/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS_186.110
}
# 一个vrrp_instance就是定义一个虚拟路由器，实例名称
vrrp_instance VI_1 {
state MASTER ===> 是主还是备
interface ens33 ===> 监听的哪个网卡，通告选举使用哪个接口进行
virtual_router_id 41 ===> 路由标识，一般不改，也可写成当前主机的主机名或IP
priority 100 ===> ID还是虚拟MAC最后一段地址的信息，取值范围0-255
advert_int 1 ===> 通告频率(秒)
authentication { ===> 通信认证机制，这里可选择是明文认证还有一种是加密认证
auth_type PASS
auth_pass 1111
}
virtual_ipaddress { ===> 设置虚拟VIP地址
172.16.186.109
}
}
#========================= 此处可配置检测脚本，该文档未配置 ==================================
vrrp_script check_nginx {
script "/etc/keepalived/check_nginx.sh"
}
在 /etc/keepalived/目录中编写好check_nginx.sh脚本后需要给该脚本添加 x 权限
如需使用邮件则global_defs段中的一切不要删除并且完整配置，该文档未配置且未使用邮件功能
脚本示例
cat > /etc/keepalived/check_nginx.sh << EOF
#!/bin/bash
count=\$(ps -ef | grep nginx | egrep -cv "grep | \$\$")
if [ "\$count" -eq 0 ];then
xxxxx
else
xxxxx
fi
EOF
#========================================================================================
# 配置集群地址访问的IP+port，端口和nginx保持一致
virtual_server 172.16.186.109 80 {
delay_loop 6 ===> 健康检查的时间,秒
lb_algo rr ===> 负载均衡的算法,默认是轮询(rr|wrr|lc|wlc|lblc|sh|dh)
lb_kind DR ===> LVS的模式，NAT|TUN|DR
persistence_timeout 5 ===> 持久连接超时时间,默认是50
protocol TCP ===> 使用的协议 -t
# 负载均衡的后端真实服务器,也就是nginx节点的具体IP地址
real_server 172.16.186.111 80 { ===> web1端的
weight 1 ==> 轮询的默认权重
TCP_CHECK { ===> 如果是应用服务器不是WEB服务器，就用TCP_CHECK检查MSIC_CHECK|SMTP_CHEKC|TCP_CHECK|SSL_GET|HTTP_GET
connect_port 80 ===> 检测的80端口
connect_timeout 2 ===> 超时时间2秒
nb_get_retry 2 ===> 重试次数2次
delay_before_retry 3 ===> 间隔时间3秒
}
}
real_server 172.16.186.112 80 { ===> web2 端的
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 2
nb_get_retry 2
delay_before_retry 3
}
}
}
# 清空当前ipvs的规则
[root@lvs keepalived-2.2.2]# ipvsadm -C
[root@lvs keepalived-2.2.2]# ipvsadm -ln
注：清空后将无规则
# 启动keepalived
[root@lvs keepalived-2.2.2]# systemctl restart keepalived
[root@lvs keepalived-2.2.2]# systemctl status keepalived
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since 四 2021-07-22 03:51:17 EDT; 6s ago
Docs: man:systemd-sysv-generator(8)
Process: 11560 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 11567 (keepalived)
CGroup: /system.slice/keepalived.service
├─11567 keepalived -f /etc/keepalived.conf -D -S 0
├─11569 keepalived -f /etc/keepalived.conf -D -S 0
└─11570 keepalived -f /etc/keepalived.conf -D -S 0
7月 22 03:51:20 lvs Keepalived_vrrp[11570]: (VI_1) Receive advertisement timeout
7月 22 03:51:20 lvs Keepalived_vrrp[11570]: (VI_1) Entering MASTER STATE
7月 22 03:51:20 lvs Keepalived_vrrp[11570]: (VI_1) setting VIPs.
7月 22 03:51:20 lvs Keepalived_vrrp[11570]: (VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.16.186.109
7月 22 03:51:20 lvs Keepalived_vrrp[11570]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 03:51:20 lvs Keepalived_vrrp[11570]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 03:51:20 lvs Keepalived_vrrp[11570]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 03:51:20 lvs Keepalived_vrrp[11570]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 03:51:20 lvs Keepalived_vrrp[11570]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 03:51:20 lvs Keepalived_healthcheckers[11569]: TCP connection to [172.16.186.112]:tcp:80 success.
[root@lvs keepalived-2.2.2]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.186.109:80 rr persistent 5
-> 172.16.186.111:80 Route 1 0 0
-> 172.16.186.112:80 Route 1 0 0
以上是主keepalived配置完毕,接下来配置从的keepalived
从keepalived配置
从主keepalived上发送包和配置文件到从的上面
[root@lvs ~]# scp -r keepalived-2.2.2 root@172.16.186.114:~
[root@lvs ~]# scp /etc/keepalived.conf root@172.16.186.114:/etc
回到从的keepalived节点上开始配置LVS 和 keepalived
安装lvs (只需安装不用配置规则)
[root@keepalived ~]# mount /dev/sr0 /mnt/usb1 ===> 挂载光盘
[root@keepalived ~]# yum -y install /mnt/usb1/Packages/ipvsadm-1.27-8.el7.x86_64.rpm ===> 如有互联网可不挂载直接yum安装ipvsadm
配置keepalived
[root@keepalived ~]# mkdir /usr/local/keepalived
[root@keepalived ~]# yum -y install gcc gcc-c++ openssl-devel libnl libnl-devel
[root@keepalived ~]# cd keepalived-2.2.2
[root@keepalived keepalived-2.2.2]# ./configure --prefix=/usr/local/keepalived --with-init=SYSV
[root@keepalived keepalived-2.2.2]# make && make install
以下都使用的绝对路径，主机名和当前路径加到一块太长，这里就不带了
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig
cp /usr/local/keepalived/sbin/keepalived /usr/sbin
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived.conf
chmod +x /etc/init.d/keepalived
chkconfig --level 2345 keepalived on
修改keepalived启动选项
vim /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-f /etc/keepalived.conf -D -S 0"
keepalived配置文件主的和从的不是一模一样，所以这里要改
vim /etc/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS_186.114 ===> 需要修改，可自定义
}
vrrp_instance VI_1 {
state BACKUP ===> 改成备机
interface ens33
virtual_router_id 41
priority 90 ===> 权重值,应比主的权重值小
....
....
其他所有都不用改也不用动
# 清空当前ipvs的规则
[root@lvs keepalived-2.2.2]# ipvsadm -C
[root@lvs keepalived-2.2.2]# ipvsadm -ln
注：清空后将无规则
# 启动keepalived
[root@lvs keepalived-2.2.2]# systemctl restart keepalived
[root@lvs keepalived-2.2.2]# systemctl status keepalived
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since 四 2021-07-22 16:36:45 CST; 4s ago
Docs: man:systemd-sysv-generator(8)
Process: 16580 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 16587 (keepalived)
CGroup: /system.slice/keepalived.service
├─16587 keepalived -f /etc/keepalived.conf -D -S 0
├─16589 keepalived -f /etc/keepalived.conf -D -S 0
└─16590 keepalived -f /etc/keepalived.conf -D -S 0
7月 22 16:36:45 keepalived Keepalived_vrrp[16590]: Registering Kernel netlink reflector
7月 22 16:36:45 keepalived Keepalived_vrrp[16590]: Registering Kernel netlink command channel
7月 22 16:36:45 keepalived Keepalived_vrrp[16590]: Assigned address 172.16.186.114 for interface ens33
7月 22 16:36:45 keepalived Keepalived_vrrp[16590]: Assigned address fe80::b77e:df59:787b:674b for interface ens33
7月 22 16:36:45 keepalived Keepalived_vrrp[16590]: Registering gratuitous ARP shared channel
7月 22 16:36:45 keepalived Keepalived_vrrp[16590]: (VI_1) removing VIPs.
7月 22 16:36:45 keepalived Keepalived_vrrp[16590]: (VI_1) Entering BACKUP STATE (init)
7月 22 16:36:45 keepalived Keepalived_vrrp[16590]: VRRP sockpool: [ifindex( 2), family(IPv4), proto(112), fd(13,14)]
7月 22 16:36:45 keepalived Keepalived[16587]: Startup complete
7月 22 16:36:48 keepalived Keepalived_healthcheckers[16589]: TCP connection to [172.16.186.111]:tcp:80 success.
查看ipvs
[root@keepalived ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.186.109:80 rr persistent 5
-> 172.16.186.111:80 Route 1 0 0
-> 172.16.186.112:80 Route 1 0 0
测试keepalived主备
在未关闭主keepalive端的时先看下状态
[root@lvs ~]# systemctl status keepalived
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since 四 2021-07-22 04:55:11 EDT; 1h 1min ago
Docs: man:systemd-sysv-generator(8)
Process: 11617 ExecStop=/etc/rc.d/init.d/keepalived stop (code=exited, status=0/SUCCESS)
Process: 11676 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 11683 (keepalived)
CGroup: /system.slice/keepalived.service
├─11683 keepalived -f /etc/keepalived.conf -D -S 0
├─11685 keepalived -f /etc/keepalived.conf -D -S 0
└─11686 keepalived -f /etc/keepalived.conf -D -S 0
7月 22 05:32:17 lvs Keepalived_vrrp[11686]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 05:32:17 lvs Keepalived_vrrp[11686]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 05:32:17 lvs Keepalived_vrrp[11686]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 05:32:17 lvs Keepalived_vrrp[11686]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 05:32:22 lvs Keepalived_vrrp[11686]: (VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.16.186.109
7月 22 05:32:22 lvs Keepalived_vrrp[11686]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 05:32:22 lvs Keepalived_vrrp[11686]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 05:32:22 lvs Keepalived_vrrp[11686]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 05:32:22 lvs Keepalived_vrrp[11686]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 05:32:22 lvs Keepalived_vrrp[11686]: Sending gratuitous ARP on ens33 for 172.16.186.109
将lvs端的keepalived服务关闭
[root@lvs ~]# systemctl stop keepalived
再来看Keepalived的状态，确定该服务已停止
[root@lvs ~]# systemctl status keepalived
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: inactive (dead) since 四 2021-07-22 04:41:10 EDT; 2min 47s ago
Docs: man:systemd-sysv-generator(8)
Process: 11617 ExecStop=/etc/rc.d/init.d/keepalived stop (code=exited, status=0/SUCCESS)
Process: 11560 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 11567 (code=exited, status=0/SUCCESS)
7月 22 04:07:42 lvs Keepalived_vrrp[11570]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 04:07:42 lvs Keepalived_vrrp[11570]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 04:07:42 lvs Keepalived_vrrp[11570]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 04:07:42 lvs Keepalived_vrrp[11570]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 04:41:08 lvs systemd[1]: Stopping SYSV: Start and stop Keepalived...
7月 22 04:41:08 lvs Keepalived[11567]: Stopping
7月 22 04:41:08 lvs Keepalived_vrrp[11570]: (VI_1) sent 0 priority
7月 22 04:41:08 lvs Keepalived_vrrp[11570]: (VI_1) removing VIPs.
7月 22 04:41:10 lvs keepalived[11617]: Stopping keepalived: [ OK ]
7月 22 04:41:10 lvs systemd[1]: Stopped SYSV: Start and stop Keepalived. ===> 已停止
# 查看ipvs规则也为空
[root@lvs ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
从keepalived节点上查看
[root@keepalived ~]# systemctl status keepalived
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since 四 2021-07-22 16:36:45 CST; 11min ago
Docs: man:systemd-sysv-generator(8)
Process: 16580 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 16587 (keepalived)
CGroup: /system.slice/keepalived.service
├─16587 keepalived -f /etc/keepalived.conf -D -S 0
├─16589 keepalived -f /etc/keepalived.conf -D -S 0
└─16590 keepalived -f /etc/keepalived.conf -D -S 0
7月 22 16:41:09 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:09 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:09 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:09 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: (VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109 ===> 已正常接管所有请求
修改持久的时间(该操作仅限于看效果，生产中不用设置)
[root@lvs ~]# ipvsadm -E -t 172.16.186.109:80 -s rr -p 3 ===>设置持久时间为3秒
设置tcp/udp的超时时间
[root@lvs ~]# ipvsadm --set 1 1 1
浏览器访问：http://172.16.186.109 #每隔4秒刷新一下就能看到效果
再将lvs节点上的主keepalived恢复
[root@lvs ~]# systemctl start keepalived
[root@lvs ~]# systemctl status keepalived ====> 会重新接管所有请求
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since 四 2021-07-22 06:02:05 EDT; 2s ago
Docs: man:systemd-sysv-generator(8)
Process: 11744 ExecStop=/etc/rc.d/init.d/keepalived stop (code=exited, status=0/SUCCESS)
Process: 11764 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 11771 (keepalived)
CGroup: /system.slice/keepalived.service
├─11771 keepalived -f /etc/keepalived.conf -D -S 0
├─11773 keepalived -f /etc/keepalived.conf -D -S 0
└─11774 keepalived -f /etc/keepalived.conf -D -S 0
7月 22 06:02:05 lvs Keepalived_vrrp[11774]: Assigned address 172.16.186.110 for interface ens33
7月 22 06:02:05 lvs Keepalived_vrrp[11774]: Assigned address fe80::20c:29ff:fef1:e2f9 for interface ens33
7月 22 06:02:05 lvs Keepalived_vrrp[11774]: Registering gratuitous ARP shared channel
7月 22 06:02:05 lvs Keepalived_vrrp[11774]: (VI_1) removing VIPs.
7月 22 06:02:05 lvs Keepalived_vrrp[11774]: (VI_1) Entering BACKUP STATE (init)
7月 22 06:02:05 lvs Keepalived_vrrp[11774]: VRRP sockpool: [ifindex( 2), family(IPv4), proto(112), fd(13,14)]
7月 22 06:02:05 lvs Keepalived[11771]: Startup complete
7月 22 06:02:06 lvs Keepalived_vrrp[11774]: (VI_1) received lower priority (90) advert from 172.16.186.114 - discarding
7月 22 06:02:07 lvs Keepalived_vrrp[11774]: (VI_1) received lower priority (90) advert from 172.16.186.114 - discarding
7月 22 06:02:08 lvs Keepalived_healthcheckers[11773]: TCP connection to [172.16.186.111]:tcp:80 success.
查看从keepalived状态
[root@keepalived ~]# systemctl status keepalived ====> 从keepalived会将ipvs移除
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since 四 2021-07-22 16:36:45 CST; 18min ago
Docs: man:systemd-sysv-generator(8)
Process: 16580 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 16587 (keepalived)
CGroup: /system.slice/keepalived.service
├─16587 keepalived -f /etc/keepalived.conf -D -S 0
├─16589 keepalived -f /etc/keepalived.conf -D -S 0
└─16590 keepalived -f /etc/keepalived.conf -D -S 0
7月 22 16:41:09 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: (VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:41:14 keepalived Keepalived_vrrp[16590]: Sending gratuitous ARP on ens33 for 172.16.186.109
7月 22 16:55:14 keepalived Keepalived_vrrp[16590]: (VI_1) Master received advert from 172.16.186.110 with higher priority 100, ours 90
7月 22 16:55:14 keepalived Keepalived_vrrp[16590]: (VI_1) Entering BACKUP STATE
7月 22 16:55:14 keepalived Keepalived_vrrp[16590]: (VI_1) removing VIPs.
当后端的httpd/nginx异常后ipvsadm会自动将该节点移除，当恢复好后ipvsadm会自动再将该节点添加到ipvs集群中
Linux学习QQ群
