Centos7+LVS-DR+Apache负载均衡web实验
一、简介
1、理论已经在上一篇博客简述，不了解得可以看看
https://www.cnblogs.com/zhangxingeng/p/10497279.html
2、LVS-DR优缺点复习
关于这种模式：
1）确保前端路由器将目标ip为vip的请求报文发往director
a、在前端网关做静态绑定；
b、在RS上使用arptables；
c、在RS上修改内核参数以限制arp通告即应答级别；
arp_announce
arp_ignore
2）、RS的RIP可以使用私网或公网地址；
3）、RS跟director在同一物理网络；
4）、请求报文经由director，响应报文直接发往client；
5）、此模式不支持端口映射；
6）、RS支持大多数的OS；
7）、RIP的网关不能指向DIP，以确保响应报文不经由director；
缺点：LVS调度器及应用服务器在同一个网段中，因此不能实现集群的跨网段应用。
优点：直接路由转发，通过修改请求报文的目标mac地址进行转发，效率提升明显
3、实验拓扑
实验说明：测试环境为虚拟机，准备三台即可，这里是四台外加本机物理机
| 节点 | ip | 应用 | ip2 | 备注 |
| web1 | 192.168.13.11（DIP） | director | VIP:ens37:0,192.168.13.100 | |
| web2 | 192.168.13.12（RIP） | rs | VIP:Lo：0，192.168.13.100 | |
| web3 | 192.168.13.13(RIP) | rs | VIP:Lo：0，192.168.13.100 | |
| web4 | 192.168.13.14 | client | curl | |
| 本地物理机测试 | 172网段 | client | web访问 |
二、开始部署
1、网络配置
修改虚拟机网卡rip、vip、dip都在同一个网段，这里使用仅主机模式
配置网卡基础参数这里就不演示，配置静态，网关指向仅主机虚拟网卡网关即可
director一块网卡即可，后续配置添加vip使用别名即可，别着急，在后面具体配置上
1 [root@web1 ~]# ifconfig 2 ens37: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 3 inet 192.168.13.11 netmask 255.255.255.0 broadcast 192.168.13.255 4 inet6 fe80::20c:29ff:fe1c:8b43 prefixlen 64 scopeid 0x20<link> 5 ether 00:0c:29:1c:8b:43 txqueuelen 1000 (Ethernet) 6 RX packets 271095 bytes 48234818 (46.0 MiB) 7 RX errors 0 dropped 0 overruns 0 frame 0 8 TX packets 129716 bytes 9264277 (8.8 MiB) 9 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
rs也是一块网卡，后续vip配置在lo：0上即可但是需要先修改arp_ignore,arp_announce才能添加lo:0
1 [root@web2 ~]# ifconfig 2 ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 3 inet 192.168.13.12 netmask 255.255.255.0 broadcast 192.168.13.255 4 inet6 fe80::3409:e73d:1ef:2e1 prefixlen 64 scopeid 0x20<link> 5 ether 00:0c:29:75:de:46 txqueuelen 1000 (Ethernet) 6 RX packets 291171 bytes 43500727 (41.4 MiB) 7 RX errors 0 dropped 0 overruns 0 frame 0 8 TX packets 84267 bytes 12274101 (11.7 MiB) 9 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@web3 ~]# ifconfig ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.13.13 netmask 255.255.255.0 broadcast 192.168.13.255 inet6 fe80::3409:e73d:1ef:2e1 prefixlen 64 scopeid 0x20<link> inet6 fe80::9416:80e8:f210:1e24 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:79:23:62 txqueuelen 1000 (Ethernet) RX packets 281305 bytes 43019942 (41.0 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 88839 bytes 12307738 (11.7 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
2、关闭防火墙
1 systemctl stop firewalld 2 3 systemctl disable firewalld 4 5 sed -i '/SELINUX/s/enforcing/disabled/' /etc/selinux/config
3、安装软件
1 11上安装ipvsadm 2 yum install ipvsadm -y 3 12、13安装httpd 4 yum install httpd -y
4、配置web
注意：arp_ignore 改为1的意义是，响应报文，请求报文从哪个地址进来的，就只能这个接口地址响应
arp_announce 改为2的意义是，通知，不通告不同网段
1 #修改web页面 2 web2 3 4 echo "welcom to web2" >/var/www/html/index.html 5 6 web3 7 echo "welcom to web3" >/var/www/html/index.html 8 #启动与开机自启 9 systemctl restart httpd 10 systemctl enable httpd 11 #修改内核参数，两台rs都修改 12 echo 1 > /porc/sys/net/ipv4/conf/all/arp_ignore 13 echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore 14 echo 1 > /proc/sys/net/ipv4/conf/ens33/arp_ignore 15 echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce 16 echo 2 > /proc/sys/net/ipv4/conf/ens33/arp_announce
#添加vip
ifconfig lo:0 192.168.13.100 broadcast 192.168.13.100 netmask 255.255.255.255 up 17 #添加默认路由，两台rs都修改，
18 route add -host 192.168.13.100 dev lo:0
19
#ping 192.168.13.100
20
ping 192.168.13.100
21 #物理机查看mac地址没有变化，说明rs没有响应ok ，11和100的地址保持一致就是对的
物理机是windows10 cmd 命令行 arp -a 查看mac表
22 arp -a
23 接口: 192.168.13.1 --- 0x16
24 Internet 地址 物理地址 类型
25 192.168.13.11 00-0c-29-1c-8b-43 动态
26 192.168.13.12 00-0c-29-75-de-46 动态
27 192.168.13.13 00-0c-29-79-23-62 动态
28 192.168.13.14 00-0c-29-0f-ea-b4 动态
29 192.168.13.100 00-0c-29-1c-8b-43 动态
检查网络信息，web2举例，web3差不多就不贴上来了
1 [root@web2 ~]# ifconfig 2 ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 3 inet 192.168.13.12 netmask 255.255.255.0 broadcast 192.168.13.255 4 inet6 fe80::3409:e73d:1ef:2e1 prefixlen 64 scopeid 0x20<link> 5 ether 00:0c:29:75:de:46 txqueuelen 1000 (Ethernet) 6 RX packets 291171 bytes 43500727 (41.4 MiB) 7 RX errors 0 dropped 0 overruns 0 frame 0 8 TX packets 84267 bytes 12274101 (11.7 MiB) 9 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 10 11 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 12 inet 127.0.0.1 netmask 255.0.0.0 13 inet6 ::1 prefixlen 128 scopeid 0x10<host> 14 loop txqueuelen 1 (Local Loopback) 15 RX packets 1181 bytes 99767 (97.4 KiB) 16 RX errors 0 dropped 0 overruns 0 frame 0 17 TX packets 1181 bytes 99767 (97.4 KiB) 18 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 19 20 lo:0: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 21 inet 192.168.13.100 netmask 255.255.255.255 22 loop txqueuelen 1 (Local Loopback) 23 24 virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 25 inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255 26 ether 52:54:00:23:a5:7c txqueuelen 1000 (Ethernet) 27 RX packets 0 bytes 0 (0.0 B) 28 RX errors 0 dropped 0 overruns 0 frame 0 29 TX packets 0 bytes 0 (0.0 B) 30 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 31 32 [root@web2 ~]#
5、配置director
lvs-dr模式不需要开启地址转发，ip_forward功能，因为director发给rs的数据包是修改过的包，不是原包，不经过ip_forward转发，而是经过修改后的mac根据l路由决策直接路由到rip
1 #ens33是另外一块网卡down掉，配置网卡别名，添加vip 2 ifconfig ens33 down 3 ifconfig ens37:0 192.168.13.100/24 broadcast 192.168.13.100 up 4 #配置ipvsadm 5 ipvsadm -C 清空 6 #添加规则 7 ipvsadm -A -t 192.168.13.100:80 -s rr 8 ipvsadm -a -t 192.168.13.100:80 -r 192.168.13.12 -g 9 ipvsadm -a -t 192.168.13.100:80 -r 192.168.13.13 -g 10 #查看规则 11 [root@web1 zhangxingeng]# ipvsadm -ln 12 IP Virtual Server version 1.2.1 (size=4096) 13 Prot LocalAddress:Port Scheduler Flags 14 -> RemoteAddress:Port Forward Weight ActiveConn InActConn 15 TCP 192.168.13.100:80 rr 16 -> 192.168.13.12:80 Route 1 0 0 17 -> 192.168.13.13:80 Route 1 0 0
检查网络信息
[root@web1 ~]# ifconfig ens37: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.13.11 netmask 255.255.255.0 broadcast 192.168.13.255 inet6 fe80::20c:29ff:fe1c:8b43 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:1c:8b:43 txqueuelen 1000 (Ethernet) RX packets 271095 bytes 48234818 (46.0 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 129716 bytes 9264277 (8.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens37:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.13.100 netmask 255.255.255.255 broadcast 192.168.13.100 ether 00:0c:29:1c:8b:43 txqueuelen 1000 (Ethernet) lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1 (Local Loopback) RX packets 123187 bytes 11609729 (11.0 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 123187 bytes 11609729 (11.0 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255 ether 52:54:00:23:a5:7c txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@web1 ~]#
6、测试结果
1 [root@web4 ~]# curl 192.168.13.100 2 welcom to web3 3 [root@web4 ~]# curl 192.168.13.100 4 welcom to web2 5 [root@web4 ~]# curl 192.168.13.100 6 welcom to web3 7 [root@web4 ~]# curl 192.168.13.100 8 welcom to web2 9 [root@web4 ~]# curl 192.168.13.100 10 welcom to web3 11 [root@web4 ~]# curl 192.168.13.100 12 welcom to web2 13 [root@web4 ~]# curl 192.168.13.100 14 welcom to web3
#统计信息
[root@web1 ~]# ipvsadm -ln --stats IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes -> RemoteAddress:Port TCP 192.168.13.100:80 46 500 0 93970 0 -> 192.168.13.12:80 23 250 0 47616 0 -> 192.168.13.13:80 23 250 0 46354 0 [root@web1 ~]#
#连接条目
1 [root@web1 ~]# ipvsadm -lnc 2 IPVS connection entries 3 pro expire state source virtual destination 4 TCP 01:53 FIN_WAIT 192.168.13.163:48482 192.168.13.100:80 192.168.13.12:80 5 TCP 01:54 FIN_WAIT 192.168.13.163:48488 192.168.13.100:80 192.168.13.13:80 6 TCP 01:53 FIN_WAIT 192.168.13.163:48480 192.168.13.100:80 192.168.13.13:80 7 TCP 14:56 ESTABLISHED 192.168.13.1:59343 192.168.13.100:80 192.168.13.13:80 8 TCP 01:52 FIN_WAIT 192.168.13.163:48478 192.168.13.100:80 192.168.13.12:80 9 TCP 01:54 FIN_WAIT 192.168.13.163:48490 192.168.13.100:80 192.168.13.12:80 10 TCP 01:53 FIN_WAIT 192.168.13.163:48484 192.168.13.100:80 192.168.13.13:80 11 TCP 01:53 FIN_WAIT 192.168.13.163:48486 192.168.13.100:80 192.168.13.12:80 12 TCP 15:01 ESTABLISHED 192.168.13.1:59344 192.168.13.100:80 192.168.13.12:80
