整合apache+tomcat+keepalived实现高可用tomcat集群
Apache是一个强大的Web服务器在处理静态页面、处理大量网络客户请求、支持服务的种类以及可配置方面都有优势，高速并且强壮。但是没有JSP/Servlet的解析能力。整合Apache和Tomcat可以看作是用Tomcat做Apache的jsp/servlet解析插件，将两者优势结合起来。不过Tomcat作为一个Web服务器，本身具备了基本的Web服务功能，在SUN的力推下，将来或许越来越强壮到不需要借助Apache优势的地步。
Tomcat是一个免费的开放源代码的Web 应用服务器，属于轻量级应用服务器，在中小型系统和并发访问用户不是很多的场合下被普遍使用，是开发和调试JSP 程序的首选。Tomcat 很受广大程序员的喜欢，因为它运行时占用的系统资源小，扩展性好，支持负载平衡与邮件服务等开发应用系统常用的功能；而且它还在不断的改进和完善中，任何一个感兴趣的程序员都可以更改它或在其中加入新的功能。
Apache与tomcat的异同：
apache支持静态页，tomcat支持动态的，比如servlet等。
一般使用apache+tomcat的话，apache只是作为一个转发，对jsp的处理是由tomcat来处理的。
apache可以支持php\cgi\perl,但是要使用java的话，你需要tomcat在apache后台支撑，将java请求由apache转发给tomcat处理。
apache是web服务器,Tomcat是应用（java）服务器，它只是一个servlet(jsp也翻译成servlet)容器，可以认为是apache的扩展，但是可以独立于apache运行。
Apache是专门用了提供HTTP服务的，以及相关配置的（例如虚拟主机、URL转发等等）。
Tomcat是Apache组织在符合J2EE的JSP、Servlet标准下开发的一个JSP服务器；
Apache与tomcat整合的好处：
如果客户端请求的是静态页面，则只需要Apache服务器响应请求。
如果客户端请求动态页面，则是Tomcat服务器响应请求。
因为jsp是服务器端解释代码的，这样整合就可以减少Tomcat的服务开销。
一.环境与功能说明
前端：apache配置负载均衡、反向代理（mod_jk模块）
后端：tomcat服务器
前端高可用：通过keepalived实现
系统版本：CentOS 6.4_x86_64（最小化）
apache版本：httpd-2.4.9
arp版本：apr-1.4.6
apr-util版本：apr-util-1.4.1
tomcat版本：apache-tomcat-7.0.53
Jdk版本：jdk-7u40-linux-x64
tomcat-commectors版本：tomcat-connectors-1.2.40
keepalived版本：keepalived-1.2.7-3.el6.x86_64(使用yum安装)
二.安装与配置apache服务器（172.16.10.72、172.16.10.75执行同样的安装与配置，这里只演示一次）
1.安装编译工具和依赖包
yum groupinstall "Development Tools" "Development Libraries" –y
2.下载软件包
yum -y install wget
wget http://mirror.esocc.com/apache//httpd/httpd-2.4.9.tar.gz
wget http://mirrors.axint.net/apache//apr/apr-1.4.6.tar.gz
wget http://mirrors.axint.net/apache//apr/apr-util-1.4.1.tar.gz
3.安装apache
3.1.安装apr和apr-tuil
httpd-2.4.9需要较新版本的apr和apr-util，因此需要事先对其进行升级。升级方式有两种，一种是通过源代码编译安装，一种是直接升级rpm包。以下是采用前者进行安装：
# tar xf apr-1.4.6.tar.gz
# tar xf apr-util-1.4.1.tar.gz
# cd apr-1.4.6
# ./configure --prefix=/usr/local/apr
# make
# make install
# cd ..
# cd apr-util-1.4.1
# ./configure --prefix=/usr/local/apr-util --with-apr=/usr/local/apr
# make
# make install
# cd ..
3.2.安装http-2.4.9
# ./configure --prefix=/usr/local/apache --sysconfdir=/etc/httpd24 --enable-so --enable-ssl --enable-cgi --enable-rewrite --with-zlib --with-pcre --with-apr=/usr/local/apr --with-apr-util=/usr/local/apr-util/ --enable-modules=most --enable-mpms-shared=all
# make && make install
4.配置SysV服务脚本/etc/init.d/httpd
# cp build/rpm/httpd.init /etc/init.d/httpd #修改如下内容：
# vi /etc/init.d/httpd httpd=${HTTPD-/usr/local/apache/bin/httpd} pidfile=${PIDFILE-/usr/local/apache/logs/${prog}.pid} lockfile=${LOCKFILE-/var/lock/subsys/${prog}} RETVAL=0 # check for 1.3 configuration check13 () { CONFFILE=/etc/httpd24/httpd.conf
#修改环境变量并把Httpd加入到系统服务
# echo "PATH=/usr/local/apache/bin:$PATH" >> /etc/profile.d/http.sh
# . /etc/profile.d/http.sh
# ln -s /usr/local/apache/include/ /usr/include/httpd
# chkconfig --add httpd
5.安装tomcat-connectors
# wget http://apache.fayea.com/apache-mirror/tomcat/tomcat-connectors/jk/tomcat-connectors-1.2.40-src.tar.gz
# tar xf tomcat-connectors-1.2.40-src.tar.gz
# cd tomcat-connectors-1.2.40-src/native/
# ./configure --with-apxs=/usr/local/apache/bin/apxs
# make
# make install
安装完成后会生成如下文件
# ls /usr/local/apache/modules/|grep 'mod_jk'
mod_jk.so
启动httpd服务
# service httpd start
执行./configure --with-apxs=/usr/local/apache/bin/apxs时可能会遇到以下报错：
need to check for Perl first, apxs depends on it...
checking for perl... /usr/bin/perl
could not find /usr/local/apache/bin/apxs
configure: error: You must specify a valid --with-apxs path
解法办法：
vi /usr/local/apache/bin/apxs
把第一行改成：#!/usr/bin/perl -w
另外检查系统中是否已经安装过perl，如果没有安装，用yum安装即可。
三.安装与配置tomcat服务器（172.16.10.61、172.16.10.62执行同样的安装，这里只演示一次）
1.安装JDK
下载地址：http://download.oracle.com/
# rpm -ivh jdk-7u40-linux-x64.rpm
# vim /etc/profile.d/java.sh
export JAVA_HOME=/usr/java/latest
export PATH=$JAVA_HOME/bin:$PATH
# . /etc/profile.d/java.sh
# java -version
java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
2.安装Tomcat
下载地址：http://apache.fayea.com/apache-mirror/tomcat/tomcat-7/v7.0.53/bin/apache-tomcat-7.0.53.tar.gz
# tar xf apache-tomcat-7.0.53.tar.gz -C /usr/local/
# cd /usr/local/
# ln -s apache-tomcat-7.0.53 tomcat
# vim /etc/profile.d/tomcat.sh
export CATALINA_HOME=/usr/local/tomcat
export PATH=$CATALINA_HOME/bin:$PATH
# . /etc/profile.d/tomcat.sh
# /usr/local/tomcat/bin/catalina.sh version
Using CATALINA_BASE: /usr/local/tomcat
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME: /usr/java/latest
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Server version: Apache Tomcat/7.0.53
Server built: Mar 25 2014 06:20:16
Server number: 7.0.53.0
OS Name: Linux
OS Version: 2.6.32-279.el6.x86_64
Architecture: amd64
JVM Version: 1.7.0_40-b43
JVM Vendor: Oracle Corporation
3.启动Tomcat服务并访问测试，默认访问端口为8080：
# /usr/local/tomcat/bin/catalina.sh start
Using CATALINA_BASE: /usr/local/tomcat
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME: /usr/java/latest
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Tomcat started.
# ss -antpl | grep 8080
0 100 :::8080 :::* users:(("java",1299,42))
4.开启Tomcat管理及状态页面
# vim /usr/local/tomcat/conf/tomcat-users.xml(在</tomcat-users>标签上一行添加如下两行)
<role rolename="manager-gui"/>
<user username="tomcat" password="tomcat" roles="manager-gui"/> #后台管理页面登录名及密码均为tomcat
# /usr/local/tomcat/bin/catalina.sh stop
# /usr/local/tomcat/bin/catalina.sh start
5.访问Tomcat后台管理
四.配置Apache使用mod_jk模块实现代理及负载均衡
1.编辑Apache主配置文件，实现代理功能（172.16.10.72）
echo "Include /etc/httpd24/extra/mod_jk.conf" >> /etc/httpd24/httpd.conf #将此指令添加到文件末尾即可
2.创建mod_jk.conf及workers.properties文件
# cat >>/etc/httpd24/extra/mod_jk.conf <<EOF
LoadModule jk_module modules/mod_jk.so
JkWorkersFile /etc/httpd24/extra/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel debug
JkMount /* TomcatA
JkMount /status/ stat1
EOF
# cat >>/etc/httpd24/extra/workers.properties <<EOF
worker.list=TomcatA,stat1
worker.TomcatA.port=8009
worker.TomcatA.host=172.16.10.61
worker.TomcatA.type=ajp13
worker.TomcatA.lbfactor=1
worker.stat1.type = status
EOF
3.编辑httpd.conf修改ServerName并重启httpd服务
#vi /etc/httpd24/httpd.conf
ServerName 172.16.10.72:80
# service httpd restart
4.修改tomcat服务器（172.16.10.61），添加测试页面
# cd /usr/local/tomcat/conf/
# cp server.xml server.xml.bak
# vi server.xml
<Engine name="Catalina" defaultHost="172.16.10.72">
<Host name="172.16.10.72" appBase="/web/eivll0m"
unpackWARs="true" autoDeploy="true">
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="eivll0m_access_log." suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
<Context path="" docBase="/web/eivll0m" />
</Host>
创建网站存放目录并创建测试页
# mkdir -p /web/eivll0m
# vi /web/eivll0m/index.jsp
<%@ page language="java" %>
<html>
<head><title>TomcatA</title></head>
<body>
<h1><font color="red">TomcatA </font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<% session.setAttribute("abc","abc"); %>
<td><%= session.getId() %></td>
</tr>
<tr>
<td>Created on</td>
<td><%= session.getCreationTime() %></td>
</tr>
</table>
</body>
</html>
重启tomcat服务并查看是否启动成功
# catalina.sh stop
# catalina.sh configtest
# catalina.sh start
# ss -antpl | grep java
0 1 ::ffff:127.0.0.1:8005 :::* users:(("java",1299,50))
0 100 :::8009 :::* users:(("java",1299,43))
0 100 :::8080 :::* users:(("java",1299,42))
5.访问Apache主机(172.16.10.72)，验证是否代理成功
6.修改Apache配置文件(mod_jk.conf、workers.properties)，实现负载均衡
# cat >>/etc/httpd24/extra/mod_jk.conf <<EOF
LoadModule jk_module modules/mod_jk.so
JkWorkersFile /etc/httpd24/extra/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel debug
JkMount /* lbcluster
JkMount /status/ stat1
EOF
# cat >>/etc/httpd/extra/workers.properties <<EOF
worker.list=lbcluster,stat1
worker.TomcatA.port=8009
worker.TomcatA.host=172.16.10.61
worker.TomcatA.type=ajp13
worker.TomcatA.lbfactor=1
worker.TomcatB.port = 8009
worker.TomcatB.host=172.16.10.62
worker.TomcatB.type = ajp13
worker.TomcatB.lbfactor = 1
worker.lbcluster.type = lb
worker.lbcluster.sticky_session = 0 #取值为{1|0}1将用户session与后端服务器绑定，0为不绑定，如果支持session复制或session共享可以设置为0
worker.lbcluster.balance_workers = TomcatA, TomcatB
worker.stat1.type = status
EOF
7.修改tomcat服务器（172.16.10.62），添加测试页面
# cd /usr/local/tomcat/conf/
# cp server.xml server.xml.bak
# vi server.xml
<Engine name="Catalina" defaultHost="172.16.10.72">
<Host name="172.16.10.72" appBase="/web/eivll0m"
unpackWARs="true" autoDeploy="true">
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="eivll0m_access_log." suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
<Context path="" docBase="/web/eivll0m" />
</Host>
创建网站存放目录并创建测试页
# mkdir -p /web/eivll0m
# vi /web/eivll0m/index.jsp
<%@ page language="java" %>
<html>
<head><title>TomcatB</title></head>
<body>
<h1><font color="blue">TomcatB </font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<% session.setAttribute("abc","abc"); %>
<td><%= session.getId() %></td>
</tr>
<tr>
<td>Created on</td>
<td><%= session.getCreationTime() %></td>
</tr>
</table>
</body>
</html>
重启tomcat服务并查看是否启动成功
# catalina.sh stop
# catalina.sh configtest
# catalina.sh start
# ss -antpl | grep java
0 1 ::ffff:127.0.0.1:8005 :::* users:(("java",1303,50))
0 100 :::8009 :::* users:(("java",1303,43))
0 100 :::8080 :::* users:(("java",1303,42))
8.访问Apache服务器（172.16.10.72），测试是否实现负载均衡（如下表示测试成功）
五.安装并配置Keepalived实现高可用
1.在apache服务器（172.16.10.72、172.16.10.75）安装keepalived
yum -y install keepalived
2.配置Keepalived(172.16.10.72)
# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from admin@eivll0m.com
smtp_server 172.16.10.0
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_httpd {
script "killall -0 httpd"
interval 1
weight -2
}
vrrp_instance httpd_1 {
state MASTER
interface eth0
virtual_router_id 58
priority 100 #定义优先级
advert_int 1
authentication {
auth_type PASS
auth_pass 1058
}
virtual_ipaddress {
172.16.10.77 #vip
}
track_script {
chk_httpd
}
}
# service keepalived start #启动keepalived服务
# chkconfig keepalived on
3.配置Keepalived(172.16.10.75)
# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from admin@eivll0m.com
smtp_server 172.17.10.0
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_httpd {
script "killall -0 httpd"
interval 1
weight -2
}
vrrp_instance httpd_1 {
state BACKUP
interface eth0
virtual_router_id 58
priority 99 #定义优先级
advert_int 1
authentication {
auth_type PASS
auth_pass 1058
}
virtual_ipaddress {
172.16.10.77 #vip
}
track_script {
chk_httpd
}
}
# service keepalived start
# chkconfig keepalived on
4.编辑httpd.conf(172.16.10.72)，并拷贝至172.16.10.75
# vi /etc/httpd24/httpd.conf
ServerName www.eivll0m.com:80
# service httpd restart
# cd /etc/httpd24/
# scp httpd.conf 172.16.10.75:/etc/httpd24/
# cd extra/
# scp mod_jk.conf workers.properties 172.16.10.75:/etc/httpd24/extra/
# service httpd restart #两台apache服务器分别执行
5.编辑tomcat服务器(172.17.10.61和172.16.10.62)的server.xml
# vi server.xml
<Engine name="Catalina" defaultHost="www.eivll0m.com">
<Host name="www.eivll0m.com" appBase="/web/eivll0m"
unpackWARs="true" autoDeploy="true">
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="eivll0m_access_log." suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
<Context path="" docBase="/web/eivll0m" />
</Host>
# catalina.sh stop
# catalina.sh start
5.通过www.eivll0m.com(172.16.10.77)进行访问测试
注意：测试时关闭所有服务器iptables，如果没有做DNS解析，可在测试机上修改hosts文件来实现。
6.模拟一台apache故障，验证keepalived是否可以成功转移服务
# ip addr show eth0 #172.16.10.72上执行
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:e9:b8:52 brd ff:ff:ff:ff:ff:ff
inet 172.16.10.72/24 brd 172.16.10.255 scope global eth0
inet 172.16.10.77/32 scope global eth0
inet6 fe80::20c:29ff:fee9:b852/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
# ip addr show eth0 #172.16.10.75上执行
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:c8:1a:08 brd ff:ff:ff:ff:ff:ff
inet 172.16.10.75/24 brd 172.16.10.255 scope global eth0
inet 192.168.200.16/32 scope global eth0
inet 192.168.200.17/32 scope global eth0
inet 192.168.200.18/32 scope global eth0
inet6 fe80::20c:29ff:fec8:1a08/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
# service httpd stop #停止172.16.10.72上的httpd服务
# ip addr show eth0 #172.16.10.72上执行
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:e9:b8:52 brd ff:ff:ff:ff:ff:ff
inet 172.16.10.72/24 brd 172.16.10.255 scope global eth0
inet6 fe80::20c:29ff:fee9:b852/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
# ip addr show eth0 #172.16.10.75上执行
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:c8:1a:08 brd ff:ff:ff:ff:ff:ff
inet 172.16.10.75/24 brd 172.16.10.255 scope global eth0
inet 192.168.200.16/32 scope global eth0
inet 192.168.200.17/32 scope global eth0
inet 192.168.200.18/32 scope global eth0
inet 172.16.10.77/32 scope global eth0 #vip已经成功转移过来
inet6 fe80::20c:29ff:fec8:1a08/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
通过上面的步骤我们可以看到，在172.16.10.72上停止apache服务，keepalived会检测到，另一台keepalived服务器会接管vip，web服务不会中断，从而实现了高可用性。
7.配置tomcat实现session共享
7.1.配置172.16.10.61这台tomcat服务器
# vi /usr/local/tomcat/conf/server.xml #在<Engine>标签内（对所有主机生效）添加或修改如下内容，另外也可以添加在<Host>标签内，对当前主机生效。并去掉此段前后注释 <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8"> <Manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" notifyListenersOnReplication="true"/> <Channel className="org.apache.catalina.tribes.group.GroupChannel"> <Membership className="org.apache.catalina.tribes.membership.McastService" address="224.0.0.1" #组播地址 port="45564" frequency="500" dropTime="3000"/> <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver" address="172.16.10.61" port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6"/> <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter"> <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/> </Sender> <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/> <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/> </Channel> <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/> <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/> <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer" tempDir="/tmp/war-temp/" deployDir="/tmp/war-deploy/" watchDir="/tmp/war-listen/" watchEnabled="false"/> <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/> <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/> </Cluster>
# mkdir /web/eivll0m/WEB-INF # cp /usr/local/tomcat/conf/web.xml /web/eivll0m/WEB-INF/ # vim /web/eivll0m/WEB-INF/web.xml <web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" version="3.0"> <distributable/> #添加此行内容
# catalina.sh stop # catalina.sh start # ss -tanlp | grep java 0 1 ::ffff:127.0.0.1:8005 :::* users:(("java",1935,55)) 0 100 :::8009 :::* users:(("java",1935,43)) 0 100 :::8080 :::* users:(("java",1935,42)) 0 50 ::ffff:172.16.10.61:4000 :::* users:(("java",1935,44))
7.2.配置172.16.10.62这台tomcat服务器
# scp /usr/local/tomcat/conf/server.xml 172.16.10.62:/usr/local/tomcat/conf/
# scp -r /web/eivll0m/WEB-INF 172.16.10.62:/web/eivll0m/
# vi /usr/local/tomcat/conf/server.xml address="172.16.10.62" #更改IP port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6"/> # catalina.sh stop # catalina.sh start # ss -antpl|grep java 0 1 ::ffff:127.0.0.1:8005 :::* users:(("java",2041,61)) 0 100 :::8009 :::* users:(("java",2041,43)) 0 100 :::8080 :::* users:(("java",2041,42)) 0 50 ::ffff:172.16.10.62:4000 :::* users:(("java",2041,44))
注：因为tomcat的session同步功能需要用到组播，windows默认情况下是开通组播服务的，但是linux默认情况下并没有开通，可以通过指令打开route add -net 224.0.0.0 netmask 240.0.0.0 dev eth0，如果需要服务器启动时即开通组播需在/etc/sysconfig/static-routes文件内加入eht0 net 224.0.0.0 netmask 240.0.0.0。
7.3.验证session共享是否实现
通过以上截图可以看出session共享已经实现。
