LVS+keepalived DR模式配置高可用负载均衡集群
实验环境
| LVS-Master | 10.0.100.201 |
VIP：10.0.100.203 |
| LVS-Slave |
10.0.100.204 |
| WEB1-Tomcat | 10.0.2.29 | gateway:10.0.2.253 |
| WEB2-Tomcat | 10.0.100.202 |
(注意：应保持所有服务器的时间一致)
Install LVS
# ln -s /usr/src/kernels/2.6.32-431.el6.i686/ /usr/src/linux //若找不到这个目录则先安装kernel-devel # yum -y install kernel-devel # tar xf ipvsadm-1.24.tar.gz # cd /soft/ipvsadm-1.24 # make;make install # ipvsadm -L IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn
Install keepalived
# tar –xvf keepalived-1.2.12.tar.gz # ./configure --sysconf=/etc/ --with-kernel-dir=/usr/src/kernels/2.6.32-431.el6.i686/ Keepalived version : 1.2.12 Compiler : gcc Compiler flags : -g -O2 Extra Lib : -lssl -lcrypto -lcrypt Use IPVS Framework : Yes //表示使用ipvs框架也代表启动ipvs功能 IPVS sync daemon support : Yes //表示启动ipvs的同步功能，--disabled-lvs-sysncd此参数可以关闭 IPVS use libnl : No //表示使用新的libn1版本，如果想使用新的版本，需要重新安装libn1的包 Use VRRP Framework : Yes //表示使用VRRP框架，这个实现keepalived高可用的必须功能 Use VRRP VMAC : Yes //表示使用基础VMAC接口的xmit VRRP包 SNMP support : No SHA1 support : No Use Debug flags : No # make;make install # cp /soft/keepalived-1.2.12/bin/keepalived /usr/bin/
配置
1. LVS-Master的Keepalived配置
! Configuration File for keepalived //全局定义 global_defs { notification_email //定义邮件 { xxxxxxxxx@163.com //定义邮件地址 } notification_email_from xxxxxxxxx@163.com smtp_server mail.163.com //定义邮件服务器 smtp_connect_timeout 30 //邮件超时时间 router_id LVS_DEVEL //route_id标识 } //定义VRR组 vrrp_instance VI_1 { state MASTER //定义为MASTER interface eth0 //对外访问的网络接口 virtual_router_id 100 //虚拟路由标识(注意要主从一致) priority 100 //优先级(主肯定比从高) nopreempt //不抢占模式 advert_int 1 //广播周期秒数 authentication { auth_type PASS auth_pass 2209 } virtual_ipaddress { 10.0.100.203 //VIP地址 } } //VIP地址与端口等设置-->注意DR模式的VIP端口和所有WEB端口应该保持一致 virtual_server 10.0.100.203 80 { delay_loop 6 //健康检查时间间隔，单位秒 lb_algo rr //调度算法为rr lb_kind DR //模式为DR persistence_timeout 0 //设置同一IP多少秒内的请求都发送到同一个realserver protocol TCP //使用TCP协议 //realserver的IP与端口等设置 real_server 10.0.2.29 80{ weight 1 TCP_CHECK { connect_timeout 10 //连接超时时间，单位s nb_get_retry 3 delay_before_retry 3 connect_port 80 //连接端口 } } real_server 10.0.100.202 80 { weight 1 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } }
2. LVS-Slave的keepalived配置
! Configuration File for keepalived global_defs { notification_email { xxxxxxxxx@163.com } notification_email_from xxxxxxxxx@163.com smtp_server mail.163.com smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_instance VI_1 { state BACKUP //这里的角色是BACKUP interface eth0 virtual_router_id 100 //这里注意和MASTER保持一致 priority 90 //优先级设置应比MASTER低 advert_int 1 authentication { auth_type PASS auth_pass 2209 } virtual_ipaddress { 10.0.100.203 } } virtual_server 10.0.100.203 80 { delay_loop 6 lb_algo rr lb_kind DR persistence_timeout 0 protocol TCP real_server 10.0.2.29 80{ weight 1 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 10.0.100.202 80 { weight 1 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } }
3. 利用脚本配置所有的WEB服务器
#!/bin/bash # Written by NetSeek # description: Config realserver lo and apply noarp WEB_VIP="10.0.100.203" //这里是VIP地址 . /etc/rc.d/init.d/functions case "$1" in start) ifconfig lo:0 $WEB_VIP netmask 255.255.255.255 broadcast $WEB_VIP /sbin/route add -host $WEB_VIP dev lo:0 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl -p >/dev/null 2>&1 echo "RealServer Start OK" ;; stop) ifconfig lo:0 down route del $WEB_VIP >/dev/null 2>&1 echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce echo "RealServer Stoped" ;; status) # Status of LVS-DR real server. islothere=`/sbin/ifconfig lo:0 | grep $WEB_VIP` isrothere=`netstat -rn | grep "lo:0" | grep $web_VIP` if [ ! "$islothere" -o ! "isrothere" ] then echo "LVS-DR real server Stopped." else echo "LVS-DR Running." fi ;; *) # Invalid entry. echo "$0: Usage: $0 {start|status|stop}" exit 1 ;; esac exit 0
启动与验证
1. 在所有WEB服务器运行脚本，并查看是否都有了VIP？
WEB1:10.0.2.29 # ./realserver.sh start RealServer Start OK # ip add show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet 10.0.100.203/32 brd 10.0.100.203 scope global lo:0 inet6 ::1/128 scope host valid_lft forever preferred_lft forever WEB2：10.0.100.202 # ./realserver.sh start RealServer Start OK # ip add show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet 10.0.100.203/32 brd 10.0.100.203 scope global lo:0 inet6 ::1/128 scope host valid_lft forever preferred_lft forever
2. 在MASTER和SLAVE都启动LVS，查看VIP位置
LVS-Master:10.0.100.201 # /etc/init.d/keepalived start 正在启动 keepalived： [确定] # ip add show 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 link/ether 00:0c:29:0f:bf:4a brd ff:ff:ff:ff:ff:ff inet 10.0.100.201/16 brd 10.0.255.255 scope global eth0 inet 10.0.100.203/32 scope global eth0 inet6 fe80::20c:29ff:fe0f:bf4a/64 scope link valid_lft forever preferred_lft forever LVS-Slave:10.0.100.204 # /etc/init.d/keepalived start 正在启动 keepalived： [确定] # ip add show 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 link/ether 00:0c:29:2e:1d:0d brd ff:ff:ff:ff:ff:ff inet 10.0.100.204/16 brd 10.0.255.255 scope global eth0 inet6 fe80::20c:29ff:fe2e:1d0d/64 scope link valid_lft forever preferred_lft forever
3. 访问测试(可通过浏览器访问VIP，我们测试是否在两个实验WEB服务器访问中实现了负载均衡)
LVS-Master:10.0.100.201
# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.100.203:http rr
-> 10.0.100.202:http Route 1 0 0
-> 10.0.2.29:http Route 1 0 0
LVS-Slave:10.0.100.204
# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.100.203:http rr
-> 10.0.100.202:http Route 1 0 0
-> 10.0.2.29:http Route 1 0 0
