Tomcat进程异常退出的一次记录

问题描述：

联调过程中，某Tomcat进程发现进程挂了，查看日志发现

16-Jul-2021 17:44:35.264 INFO [Thread-8] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["http-nio-80"]
16-Jul-2021 17:44:35.271 INFO [Thread-8] org.apache.catalina.core.StandardService.stopInternal Stopping service [Catalina]
[INFO ] 2021-07-16 17:44:35.290 [Curator-Framework-0]-[CuratorFrameworkImpl.java:821] - backgroundOperationsLoop exiting
[INFO ] 2021-07-16 17:44:35.294 [Thread-6]-[ZooKeeper.java:684] - Session: 0x10046c488240ac3 closed
[INFO ] 2021-07-16 17:44:35.294 [main-EventThread]-[ClientCnxn.java:512] - EventThread shut down
[INFO ] 2021-07-16 17:44:35.316 [Thread-6]-[ConnectionListener.java:22] - RabbitMQ连接关闭：org.springframework.amqp.rabbit.connection.SimpleConnection@7361caae
[INFO ] 2021-07-16 17:44:35.339 [Thread-6]-[ExecutorConfigurationSupport.java:208] - Shutting down ExecutorService
16-Jul-2021 17:44:35.352 WARNING [Thread-8] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [ROOT] registered the JDBC driver [com.alibaba.druid.proxy.DruidDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
16-Jul-2021 17:44:35.352 WARNING [Thread-8] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [ROOT] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
16-Jul-2021 17:44:35.353 WARNING [Thread-8] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Curator-ConnectionStateManager-0] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403)
org.apache.curator.framework.state.ConnectionStateManager.processEvents(ConnectionStateManager.java:245)
org.apache.curator.framework.state.ConnectionStateManager.access$000(ConnectionStateManager.java:43)
org.apache.curator.framework.state.ConnectionStateManager$1.call(ConnectionStateManager.java:111)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)

 

再往上翻，并没有异常的日志，服务在17:44:35.264这个时间点之前运行很正常，进程是"突然"关闭的，而且日志中还打印有关闭的信息。查看机器内存占用也非常乐观，不是因为内存占满而导致的关闭。

 

解决：在一番摸索及百度下发现了关闭的原因-脚本启动进程后shell关闭引起。

shell脚本如下：

 

 最后是通过tail命令监听了日志，tomcat启动为后，当前shell进程并没有退出，而是挂住在tail进程，往终端输出日志内容。这种情况下，如果用户直接关闭ssh终端的窗口(用鼠标或快捷键)，则java进程也会退出。而如果先ctrl-c终止脚本sh进程，然后再关闭ssh终端的话，则java进程不会退出。

问题可以复现，最后修改了脚本解决。