lvs+keepalive实现主从效果，以及RS健康监测和tcp，udp实现非web的负载均衡（DR模式）
前面文章讲到了tcp和udp负载均衡，但是没有健康监测，这几天我优化了一下上次的操作。当然，我也是用的跨网段的通讯，因为线上业务主要是海外业务，所以做了iptables流量转发
IP：
lvs-master：10.60.196.184 vip10.60.196.186
lvs-buckup：10.60.196.183 vip10.60.196.186
rs1：10.60.196.164
rs2：10.60.196.165
rs3：10.60.196.168
route：10.10.10.1 内网ip：10.60.196.180 （主要做内外网转发）
lvs配置：
vim lvs.sh:
#!/bin/bash
GW=10.60.196.254
# website director vip.
SNS_VIP=10.60.196.186
SNS_RIP1=10.60.196.165
SNS_RIP2=10.60.196.168
SNS_RIP3=10.60.196.164
logger $0 called with $1
case "$1" in
start)
# set squid vip
/sbin/ipvsadm -C
/sbin/ipvsadm --set 30 5 30
ipvsadm -A -t $SNS_VIP:60000 -s rr
ipvsadm -a -t $SNS_VIP:60000 -r $SNS_RIP1:60000 -g -w 1
ipvsadm -a -t $SNS_VIP:60000 -r $SNS_RIP2:60000 -g -w 1
ipvsadm -a -t $SNS_VIP:60000 -r $SNS_RIP3:60000 -g -w 1
ipvsadm -A -u $SNS_VIP:60000 -s rr
ipvsadm -a -u $SNS_VIP:60000 -r $SNS_RIP1:60000 -g -w 1
ipvsadm -a -u $SNS_VIP:60000 -r $SNS_RIP2:60000 -g -w 1
ipvsadm -a -u $SNS_VIP:60000 -r $SNS_RIP3:60000 -g -w 1
touch /var/lock/subsys/ipvsadm >/dev/null 2>&1
;;
stop)
/sbin/ipvsadm -C
/sbin/ipvsadm -Z
# ifconfig eth3:0 down
# route del $SNS_VIP
# route del $SNS_VIP
rm -rf /var/lock/subsys/ipvsadm >/dev/null 2>&1
echo "ipvsadm stoped"
;;
status)
if [ ! -e /var/lock/subsys/ipvsadm ];then
echo "ipvsadm stoped"
exit 1
else
echo "ipvsadm OK"
fi
;;
*)
echo "Usage: $0 {start|stop|status}"
exit 1
esac
exit 0
因为lvs规则这次我是直接写到ipvsadm里面，所以没有健康监测机制，因此需要写脚本来实现：
vim rs_health.sh
#!/bin/bash
#real health monitor
SNS_VIP=10.60.196.186
SNS_RIP1=10.60.196.165
SNS_RIP2=10.60.196.168
SNS_RIP3=10.60.196.164
/usr/bin/nc -v -w 1 $SNS_RIP1 -n 60000 > /dev/null
if [[ $? -ne 0 ]]; then
/sbin/ipvsadm -d -t $SNS_VIP -r $SNS_RIP1
/sbin/ipvsadm -d -u $SNS_VIP -r $SNS_RIP1
sed -i '19s/^ /#/' /root/lvs.sh (这里是编辑lvs脚本，如果端口通不了就注释对应的rs条目，由于线上需求我的lvs脚本需要死循环运行，所以这里不适合用for循环写)
sed -i '26s/^ /#/' /root/lvs.sh
else
sed -i '19s/#/ /' /root/lvs.sh
sed -i '26s/#/ /' /root/lvs.sh
fi
/usr/bin/nc -v -w 1 $SNS_RIP2 -n 60000 > /dev/null
if [[ $? -ne 0 ]]; then
/sbin/ipvsadm -d -t $SNS_VIP -r $SNS_RIP2
/sbin/ipvsadm -d -u $SNS_VIP -r $SNS_RIP2
sed -i '20s/^ /#/' /root/lvs.sh
sed -i '27s/^ /#/' /root/lvs.sh
else
sed -i '20s/#/ /' /root/lvs.sh
sed -i '27s/#/ /' /root/lvs.sh
fi
/usr/bin/nc -v -w 1 $SNS_RIP3 -n 60000 > /dev/null
if [[ $? -ne 0 ]]; then
/sbin/ipvsadm -d -t $SNS_VIP -r $SNS_RIP3
/sbin/ipvsadm -d -u $SNS_VIP -r $SNS_RIP3
sed -i '21s/^ /#/' /root/lvs.sh
sed -i '28s/^ /#/' /root/lvs.sh
else
sed -i '21s/#/ /' /root/lvs.sh
sed -i '28s/#/ /' /root/lvs.sh
fi
lvs上面keepalive的配置：
vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from admin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER （备机主要修改为buckup）
interface eth0
virtual_router_id 51
priority 100 （备机改98）
advert_int 1
authentication {
auth_type PASS
auth_pass 11112222
}
virtual_ipaddress {
10.60.196.186/32 label eth0:0
}
}
由于lvs的dr模式用的是内网做的vip，所以不需要开启ip转发功能
以上关于lvs主备配置
-------------------------------------------------------------------------- 主从配置一样注意修改一下彼此的master，backup和优先级
rs配置：
#!/bin/bash
#chkconfig: 2345 79 20
#description:realserver
SNS_VIP2=10.60.196.186
. /etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $SNS_VIP1 netmask 255.255.255.255 broadcast $SNS_VIP1
/sbin/route add -host $SNS_VIP1 dev lo:0
ifconfig lo:1 $SNS_VIP2 netmask 255.255.255.255 broadcast $SNS_VIP2
/sbin/route add -host $SNS_VIP2 dev lo:1
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
route del $SNS_VIP1 >/dev/null 2>&1
ifconfig lo:1 down
route del $SNS_VIP2 >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0
route上面配置iptables转发：
-A PREROUTING -d 10.10.10.1/32 -p tcp -m tcp --dport 60000 -j DNAT --to-destination 10.60.196.186:60000
-A PREROUTING -d 10.10.10.1/32 -p udp -m udp --dport 60000 -j DNAT --to-destination 10.60.196.186:60000
-A POSTROUTING -d 10.60.196.186/32 -p tcp -m tcp --dport 60000 -j SNAT --to-source 10.60.196.180
-A POSTROUTING -d 10.60.196.186/32 -p udp -m udp --dport 60000 -j SNAT --to-source 10.60.196.180
