LVS+keepalived+nginx
LVS是Linux Virtual Server的简写，基于4层协议不处理，不响应，只转发，速度更快
wget -c http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
ln -s /usr/src/kernels/2.6.* /usr/src/linux //IPVS模块编译进内核里，需要做软连接
tar xzvf ipvsadm-1.24.tar.gz &&cd ipvsadm-1.24 && make && make install
wget -c http://www.keepalived.org/software/keepalived-1.1.15.tar.gz
tar -xzvf keepalived-1.1.15.tar.gz &&cd keepalived-1.1.15
./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-431.el6.x86_64/;make&&make install
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp -r /usr/local/keepalived/etc/keepalived/ /etc
192.168.10.130，192.168.10.128为均衡的后端机器 ，192.168.10.131 192.168.10.132 为LVS主备
MASTER配置文件
! Configuration File for keepalived
global_defs {
notification_email {
liu6630@qq.com
}
notification_email_from liu6630@qq.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
lvs_sync_daemon_inteface eth0
virtual_router_id 152
priority 100
advert_int 5
nopreempt
authentication {
auth_type PASS
auth_pass 2222
}
virtual_ipaddress {
192.168.10.141
}
}
virtual_server 192.168.10.141 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
#persistence_timeout 60
protocol TCP
real_server 192.168.10.128 80 {
weight 100
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.10.130 80 {
weight 100
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
BACKUP配置文件 将priority 改小些就行
real_server 真实服务器起执行脚本lvs_realserver.sh
#!/bin/sh
#LVS Client Server
VIP=192.168.10.141
case $i in
start)
ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
/sbin/route add -host $VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
exit 0
;;
stop)
ifconfig lo:0 down
route del $VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped OK"
exit 1
;;
*)
echo "Usage: $0 {start|stop}"
;;
esac
LVS不会检测你后端502超时错误，只关心80端口是否开启对于应用来说，这样检测明显不足，可以增加LVS对后端Nginx URL的检测，能访问URL则表示服务正常，代码：real_server 192.168.1.4 80 {
weight 100
HTTP_GET {
url {
path /monitor/warn.jsp
status_code 200
}
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
}
}
与mysql+keepalived去区别，real_server 不能为本机 IP
lvs的设置：
查看是ipvsadm --list --timeout, 比如我的机器就会返回如下结果：
# ipvsadm --list --timeout
Timeout (tcp tcpfin udp): 7200 5 60
这就表明我的tcp session的timeout时间是7200秒。
设置timeout：
ipvsadm --set 7200 5 60
这个值如果设置太小，你的client将会收到 connection reset by peer此类的错误提示。
keepalived的配置：
就是virtual_server的persistence_timeout，意思就是在这个一定时间内会讲来自同一用户（根据ip来判断的）route到同一个real
server。对于长连接类的应用，配置值最好跟lvs的配置的timeout一致。
排错
1.编译是否加了内核 没加内核不会启动检测 加了
2配置文件是否在 /etc/keepalivde 下 cp -r /usr/local/keepalived/etc/keepalived/ /etc
3是否加载modprobe ip_vs 未加载虚拟IP模块 有VIP其他服务也不能访问VIP
4是否VIP与真实IP同一网段且无IP冲突
5VIP是否起来 可以用 ip addr list 查看
6 virtual_router_id 同一集群的keepalived的主、备机的virtual_router_id 必须相同,取值0-255 但是同一内网中不应有相同virtual_router_id的集群
7 ipvsadm -ln 察看IPVS规则 是否添加真实服务器
8tcpdump -nn port 80 and host 192.168.10.141
9ab -c 100 -n 1000 http://192.168.10.141/ 压测
