LVS+Keepalived+Nginx+Tomcat实现负载均衡高可用
LVS+Keepalived+Nginx+Tomcat实现负载均衡高可用
1、nginx+tomcat已经实现web服务，为什么还要用lvs+keepalived？
目前系统整体设计是采用Nginx做负载均衡，若出现Nginx单机故障，则导致整个系统无法正常运行。针对系统架构设计的高可用要求，我们需要解决Nginx负载均衡出现单机故障时，系统正常运行的需求。所以系统架构引入Lvs+Keepalived组件，实现系统高可用。
2、LVS和Keepalived是什么？
LVS(Linux Virtual Server)是章文嵩博士发起和领导的优秀的集群解决方案，我们使用LVS的目的：通过LVS提供的负载均衡技术和Linux操作系统实现一个高性能、高可用的服务器群集，它具有良好可靠性、可扩展性和可操作性，从而以低廉的成本实现最优的服务性能。
Keepalived是集群管理中保证集群高可用的一个服务软件，其功能类似于heartbeat，主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现，用来防止单点故障。
LVS+Keepalived的简单理论介绍可参考我前面写的：LVS+Keepalived，优点是部分网友中文博客贡献+我个人的理解和排版；
LVS 官方网站：http://www.linuxvirtualserver.org/
Keepalived 官方网站：http://www.keepalived.org/
这里我主要想描述LVS+Keepalived的负载均衡高可用，tomcat部分介绍就不在这里叙述了，后面会补一篇tomcat的文章。
1、网站的架构图
[root@LVSDR1-211 ~]# service iptables stop #关闭iptables：
iptables: Setting chains to policy ACCEPT: filter [ OK ]iptables: Flushing firewall rules: [ OK ]iptables: Unloading modules: [ OK ][root@LVSDR1-211 ~]# chkconfig iptables off #关闭iptables开机启动[root@LVSDR1-211 ~]# chkconfig --list |grep iptables #查看iptables开机启动情况iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off[root@LVSDR1-211 ~]# vim /etc/sysconfig/selinuxSELINUX=disabled #永久关闭selinux[root@LVSDR1-211 ~]# /usr/sbin/ntpdate time.nist.gov #time.nist.gov为美国标准技术院，采用格灵威时间 [root@LVSDR1-211 ~]# vim /etc/security/limits.conf #打开linux文件描述符
39 #<domain> <type> <item> <value>
40 * soft nofile 65535
41 * halt nofile 65535
42 # * soft core 0
[root@LVSDR1-211 ~]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 3831
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65535 #这里是65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 3831
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
一、安装Nginx服务：
1、下载Nginx安装包：
[root@Tomcat1-219 tools]# wget http://nginx.org/download/nginx-1.8.1.tar.gz #Nginx官方下载nginx安装包，最新稳定版为1.8.1 --2016-09-08 18:25:05-- http://nginx.org/download/nginx-1.8.1.tar.gz Resolving nginx.org... 95.211.80.227, 206.251.255.63, 2001:1af8:4060:a004:21::e3, ... Connecting to nginx.org|95.211.80.227|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 833473 (814K) [application/octet-stream] Saving to: “nginx-1.8.1.tar.gz” 100%[===================================================================================================>] 833,473 11.3K/s in 63s [root@Tomcat1-219 tools]# tar -xf nginx-1.8.1.tar.gz #下载完成后，解压缩 [root@Tomcat1-219 tools]# cd nginx-1.8.1 [root@Tomcat1-219 nginx-1.8.1]# ll #安装包下的文件 total 660 drwxr-xr-x 6 1001 1001 4096 Sep 8 18:16 auto -rw-r--r-- 1 1001 1001 251319 Jan 26 2016 CHANGES -rw-r--r-- 1 1001 1001 383019 Jan 26 2016 CHANGES.ru drwxr-xr-x 2 1001 1001 4096 Sep 8 18:16 conf -rwxr-xr-x 1 1001 1001 2478 Jan 26 2016 configure drwxr-xr-x 4 1001 1001 4096 Sep 8 18:16 contrib drwxr-xr-x 2 1001 1001 4096 Sep 8 18:16 html -rw-r--r-- 1 1001 1001 1397 Jan 26 2016 LICENSE drwxr-xr-x 2 1001 1001 4096 Sep 8 18:16 man -rw-r--r-- 1 1001 1001 49 Jan 26 2016 README drwxr-xr-x 8 1001 1001 4096 Sep 8 18:16 src
2、我们开始安装nginx：
#编译nginx环境前需要先把gcc等开发库之类提前装好; [root@Tomcat1-219 nginx-1.8.1]# yum -y install gcc gcc-c++ automake autoconf libtool make #安装pcre是为了支持rewrite， #zlib是为了支持gzip压缩 #openssl是为了支持https; [root@Tomcat1-219 nginx-1.8.1]# yum install -y pcre-devel.x86_64 pcre-devel.x86_64 zlib.x86_64 zlib-devel.x86_64 openssl-devel.x86_64 openssl.x86_64 [root@Tomcat1-219 nginx-1.8.1]# groupadd www [root@Tomcat1-219 nginx-1.8.1]# useradd -r -g www www -s /sbin/nologin #运行nginx的用户和用户组 [root@realserver1 nginx-1.8.1]# mkdir -pv /opt/application/nginx mkdir: created directory `/opt/application' mkdir: created directory `/opt/application/nginx' [root@realserver1 nginx-1.8.1]# ./configure --prefix=/opt/application/nginx/ --user=www --group=www --with-http_stub_status_module --with-http_ssl_module --with-http_spdy_module --with-http_gzip_static_module --with-http_realip_module --with-ipv6 # --with-http_stub_status_module 启用nginx状态监控 # --with-http_ssl_module 启用HTTPS加密 # --with-http_spdy_module 启用spdy支持，缩短为网页加载时间 # --with-http_gzip_static_module 启用静态压缩 # --with-http_realip_module 做代理时获取客户端真实IP # --with-ipv6 支持ipv6 [root@realserver1 nginx-1.8.1]# make ;make install
3、创建nginx启动脚本
[root@Tomcat1-219 nginx-1.8.1]# vim /etc/rc.d/init.d/nginx #脚本内容见后面 "#nginx启动脚本" [root@Tomcat1-219 nginx-1.8.1]# chmod 755 /etc/rc.d/init.d/nginx #给脚本nginx+执行权限 [root@Tomcat1-219 nginx-1.8.1]# service nginx start Starting nginx: [ OK ] [root@Tomcat1-219 nginx-1.8.1]# chkconfig --add nginx #添加开机启动 [root@Tomcat1-219 nginx-1.8.1]# chkconfig nginx on [root@Tomcat1-219 nginx-1.8.1]# chkconfig --list nginx #查看nginx开机启动信息 nginx 0:off 1:off 2:on 3:on 4:on 5:on 6:off [root@Tomcat1-219 nginx-1.8.1]# cat /etc/rc.d/init.d/nginx #nginx启动脚本 #!/bin/sh # # nginx - this script starts and stops the nginx daemin # # chkconfig: - 85 15 # description: Nginx is an HTTP(S) server, HTTP(S) reverse \ # proxy and IMAP/POP3 proxy server # processname: nginx # config: /usr/local/nginx/conf/nginx.conf # pidfile: /usr/local/nginx/logs/nginx.pid # Source function library. . /etc/rc.d/init.d/functions # Source networking configuration. . /etc/sysconfig/network # Check that networking is up. [ "$NETWORKING" = "no" ] && exit 0 nginx="/opt/application/nginx/sbin/nginx" prog=$(basename $nginx) NGINX_CONF_FILE="/opt/application/nginx/conf/nginx.conf"lockfile=/var/lock/subsys/nginx start() { [ -x $nginx ] || exit 5 [ -f $NGINX_CONF_FILE ] || exit 6 echo -n $"Starting $prog: " daemon $nginx -c $NGINX_CONF_FILE retval=$? echo [ $retval -eq 0 ] && touch $lockfile return $retval } stop() { echo -n $"Stopping $prog: " killproc $prog -QUIT retval=$? echo [ $retval -eq 0 ] && rm -f $lockfile return $retval } restart() { configtest || return $? stop start } reload() { configtest || return $? echo -n $"Reloading $prog: " killproc $nginx -HUP RETVAL=$? echo } force_reload() { restart } configtest() { $nginx -t -c $NGINX_CONF_FILE } rh_status() { status $prog } rh_status_q() { rh_status >/dev/null 2>&1 } case "$1" in start) rh_status_q && exit 0 $1 ;; stop) rh_status_q || exit 0 $1 ;; restart|configtest) $1 ;; reload) rh_status_q || exit 7 $1 ;; force-reload) force_reload ;; status) rh_status ;; condrestart|try-restart) rh_status_q || exit 0 ;; *) echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}" exit 2 esac
4、测试Nginx服务
[root@Tomcat1-219 nginx-1.8.1]# service nginx status #查看当前nginx服务状态，为stop nginx is stopped [root@Tomcat1-219 nginx-1.8.1]# service nginx start #启动nginx服务 Starting nginx: [ OK ] [root@Tomcat1-219 nginx-1.8.1]# service nginx status #再次查看当前nginx服务状态，为running nginx (pid 6602 6600) is running... [root@Tomcat1-219 nginx-1.8.1]# netstat -tunlpa #查看端口，发现80端口已经起来了 Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 6600/nginx tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1348/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1427/master tcp 0 0 192.168.2.219:22 192.168.2.102:49608 ESTABLISHED 1447/sshd tcp 0 0 :::22 :::* LISTEN 1348/sshd tcp 0 0 ::1:25 :::* LISTEN 1427/master 浏览器打开http://192.168.2.219
