LVS跨网段DR模型和FWM多服务绑定
一、实验环境
系统：CentOS7.6
主机：5台 (虚拟机)
客户端1台：172.16.236.134/24 (NAT网卡)，网关指向 172.16.236.185/24（路由服务器）
路由服务器1台：172.16.236.185/24 (NAT)，192.168.214.17/16 (仅主机)，附加绑定IP (10.0.0.200/8)
LVS服务器1台：VIP (10.0.0.100/32，绑定在lo网卡上)，DIP (192.168.214.27/16，仅主机)，网关指向 192.168.214.17/16（路由服务器）
RS1服务器1台：VIP (10.0.0.100/32，绑定在lo网卡上)，RIP (192.168.214.37/16，仅主机)，网关指向 192.168.214.17/16（路由服务器）
RS2服务器1台：VIP (10.0.0.100/32，绑定在lo网卡上)，RIP (192.168.214.47/16，仅主机)，网关指向 192.168.214.17/16（路由服务器）
软件包：ipvsadm，httpd，mod_ssl（光盘yum源）
二、相关实验
1、实现LVS跨网段DR模型
(1) 按网络规划，配置好各主机的IP
客户端服务器：eth0：172.16.236.134/24，网关 172.16.236.185
路由器服务器：eth0：172.16.236.185/24，eth1：192.168.214.17/16，网关不需配
LVS服务器：eth0：192.168.214.27/16，网关 192.168.214.17
RS1服务器：eth0：192.168.214.37/16，网关 192.168.214.17
RS2服务器：eth0：192.168.214.47/16，网关 192.168.214.17
(2) 在路由服务器上开启网络转发功能
[root@centos7-17 ~]# vim /etc/sysctl.conf net.ipv4.ip_forward=1 [root@centos7-17 ~]# sysctl -p net.ipv4.ip_forward = 1
(3) 在LVS服务器上安装ipvsadm包
[root@centos7-27 ~]# yum install -y ipvsadm
(4) 在LVS服务器上配置LVS，此处用脚本实现，脚本如下
[root@centos7-27 ~]# vim lvs_dr_vs.sh #!/bin/bash vip='10.0.0.100' #VIP iface='lo:1' #VIP绑定接口 mask='255.255.255.255' #VIP子网掩码 port='80' #端口 rs1='192.168.214.37' #RS1服务器IP rs2='192.168.214.47' #RS2服务器IP scheduler='wrr' #调度算法 type='-g' #LVS类型，-m为nat模式,-g为dr模式，-i为tun模式 case $1 in start) ifconfig $iface $vip netmask $mask #broadcast $vip up iptables -F ipvsadm -A -t ${vip}:${port} -s $scheduler ipvsadm -a -t ${vip}:${port} -r ${rs1} $type -w 1 ipvsadm -a -t ${vip}:${port} -r ${rs2} $type -w 1 ;; stop) ipvsadm -C ifconfig $iface down ;; *) echo "Usage $(basename $0) start|stop" exit 1 ;; esac
#!/bin/bash vip='10.0.0.100' iface='lo:1' mask='255.255.255.255' port='80' rs1='192.168.214.37' rs2='192.168.214.47' scheduler='wrr' type='-g' case $1 in start) ifconfig $iface $vip netmask $mask #broadcast $vip up iptables -F ipvsadm -A -t ${vip}:${port} -s $scheduler ipvsadm -a -t ${vip}:${port} -r ${rs1} $type -w 1 ipvsadm -a -t ${vip}:${port} -r ${rs2} $type -w 1 ;; stop) ipvsadm -C ifconfig $iface down ;; *) echo "Usage $(basename $0) start|stop" exit 1 ;; esac
(5) 在R1服务器配置httpd服务，并配置VIP绑定与相关内核参数，配置RIP(192.168.214.37)时，记得网关指向 192.168.214.27
[root@centos7-37 ~]# yum install -y httpd [root@centos7-37 ~]# echo 192.168.214.37 RS1 > /var/www/html/index.html [root@centos7-37 ~]# systemctl start httpd #配置VIP绑定与相关内核参数，配置脚本如下 [root@centos7-37 ~]# vim lvs_dr_rs.sh #!/bin/bash vip='10.0.0.100' mask='255.255.255.255' dev='lo:1' case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig $dev $vip netmask $mask #broadcast $vip up #route add -host $vip dev $dev ;; stop) ifconfig $dev down echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce ;; *) echo "Usage:$(basename $0) start|stop" exit 1 ;; esac [root@centos7-37 ~]# bash lvs_dr_rs.sh start
(6) 在R2服务器配置httpd服务，并配置VIP绑定与相关内核参数，配置RIP(192.168.214.37)时，记得网关指向 192.168.214.27
[root@centos7-47 ~]# yum install -y httpd [root@centos7-47 ~]# systemctl start httpd [root@centos7-47 ~]# echo 192.168.214.47 RS2 > /var/www/html/index.html #配置VIP绑定相关内核参数，脚本与R1上一样 [root@centos7-47 ~]# bash lvs_dr_rs.sh start
#!/bin/bash vip='10.0.0.100' mask='255.255.255.255' dev='lo:1' case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig $dev $vip netmask $mask #broadcast $vip up #route add -host $vip dev $dev ;; stop) ifconfig $dev down echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce ;; *) echo "Usage:$(basename $0) start|stop" exit 1 ;; esac
(7) 在LVS服务器上启动配置脚本
[root@centos7-27 ~]# bash lvs_dr_vs.sh start [root@centos7-27 ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.0.0.100:80 wrr -> 192.168.214.37:80 Route 1 0 0 -> 192.168.214.47:80 Route 1 0 0
(8) 在客户端上进行测试
[root@centos7 ~]# while true;do curl 10.0.0.100 ;sleep 1;done
#可以看到成功了 192.168.214.37 RS1 192.168.214.47 RS2 192.168.214.37 RS1 192.168.214.47 RS2 192.168.214.37 RS1 192.168.214.47 RS2
...
2、实现LVS的FWM多服务绑定
当http服务同时用到80与443端口时，用DR模式会有点繁琐，此时用FWM (FireWall Mark)是一种不错的选择。
(1) 只需要LVS服务器稍稍修改即可，其它服务器还是按以上配置
在LVS主机上打标记
iptables -t mangle -A PREROUTING -d $vip -p $proto –m multiport --dports $port1,$port2,... -j MARK --set-mark NUMBER
在LVS主机基于标记定义集群服务
ipvsadm -A -f NUMBER [options]
[root@centos7-27 ~]# bash lvs_dr_vs.sh stop [root@centos7-27 ~]# ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn [root@centos7-27 ~]# ifconfig lo:1 10.0.0.100 netmask 255.255.255.255 #绑定VIP #打标记 [root@centos7-27 ~]# iptables -t mangle -A PREROUTING -d 10.0.0.100 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10 #配置集群 [root@centos7-27 ~]# ipvsadm -A -f 10 -s rr [root@centos7-27 ~]# ipvsadm -a -f 10 -r 192.168.214.37 -g [root@centos7-27 ~]# ipvsadm -a -f 10 -r 192.168.214.47 -g [root@centos7-27 ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn FWM 10 rr -> 192.168.214.37:0 Route 1 0 0 -> 192.168.214.47:0 Route 1 0 0
(2) 在两台RS服务器上安装mod_ssl模块，实现https的访问
[root@centos7-37 ~]# yum install -y mod_ssl [root@centos7-37 ~]# systemctl restart httpd [root@centos7-47 ~]# yum install -y mod_ssl [root@centos7-47 ~]# systemctl restart httpd
(3) 在客户端上测试80与443端口的调度情况
[root@centos7 ~]# while true;do curl 10.0.0.100 ;curl -k https://10.0.0.100;sleep 1;done #可以看到成功了，且不分端口进行了统一调度 192.168.214.47 RS2 192.168.214.37 RS1 192.168.214.47 RS2 192.168.214.37 RS1 192.168.214.47 RS2 192.168.214.37 RS1 192.168.214.47 RS2 192.168.214.37 RS1 192.168.214.47 RS2 192.168.214.37 RS1
