最近网站老出错,Google钩了好久才找到这篇文章,希望对大家有帮助。
我的环境是Apache 2.0.59+mod_jk 1.2+JBoss3.2.2,服务器最近访问量大的时候经常当掉,apache没有问题,重启一下JBoss就好了,但过了一阵又重来,郁闷,根据这篇文章的说法,改了“worker.properties”里配置"connection_pool_timeout"为5分钟,再继续观察一下,或许JBoss升级到4.0.5GA就不会有问题了。

Environment:  Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA 
JBoss Forum Reference: http://www.jboss.com/index.html?module=bb&op=viewtopic&t=110308 
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to connect to my application running within JBoss.

I have been running with this using JBoss 4.0.5 for many months, without issue. I just upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last night it took 6.5 hours) the mod_jk logs start showing these errors:

Code:

[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566): (node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928): (node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port

 

These errors do NOT happen with each request. They happen perhaps one out of five, although it seems to fluctuate. Most requests work, but many don't. There are NO errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP address and the same thing happens.

I can fix it by restarting JBoss, then everything works, for a few hours at least, before the problem starts up again.

 

-----------------

More information:

This appears to be related to open connections.

mod_jk was holding connections open to the JBoss web server on 8009, and the number of ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some of the connections would get reused, but not that many.

I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the number of established connections down around 50-60. However, the connections that have timed out of the pool, appear to persist forever.

The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:

10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1

The connection view from JBossWeb gets stuck in CLOSE_WAIT:

10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT


Once the number of connections hits 1024 (or thereabouts) everything falls apart, of course, and I have to kill -9 jboss and start it up again.

This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to run into this.....


did changing the connectionTimeout to 600000 for the ajp connector help as recommended in http://jira.jboss.com/jira/browse/JBWEB-82 ? I am experiencing the same issue, except I was NOT seeing connections stuck in a CLOSE_WAIT state until I made the recommended connectionTimeout change to 600000.

posted on 2007-06-27 10:18  nuelaleo  阅读(1033)  评论(0编辑  收藏  举报