liveness] request_id [31] timed out after [5000ms]

ES连接超时，异常信息

2017-09-07 10:42:45.042 [elasticsearch[Bantam][transport_client_worker][T#17]{New I/O worker #1073}] WARN  o.e.n.c.s.nio.AbstractNioSelector  - Unexpected exception in the selector loop.
java.lang.OutOfMemoryError: Direct buffer memory
    at java.nio.Bits.reserveMemory(Bits.java:693)
    at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
    at org.jboss.netty.channel.socket.nio.SocketReceiveBufferAllocator.newBuffer(SocketReceiveBufferAllocator.java:64)
    at org.jboss.netty.channel.socket.nio.SocketReceiveBufferAllocator.get(SocketReceiveBufferAllocator.java:41)
    at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:62)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
    at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
    at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
2017-09-07 10:42:45.042 [elasticsearch[Battletide][generic][T#36]] INFO  org.elasticsearch.client.transport  - [Battletide] failed to get node info for {#transport#-1}{192.168.1.100}{192.168.1.100:9300}, disconnecting...
org.elasticsearch.transport.ReceiveTimeoutTransportException: [][192.168.1.100:9300][cluster:monitor/nodes/liveness] request_id [2379] timed out after [12187ms]
    at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:698)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
2017-09-07 10:42:45.042 [elasticsearch[Luna][generic][T#1]] INFO  org.elasticsearch.client.transport  - [Luna] failed to get node info for {#transport#-1}{192.168.1.100}{192.168.1.100:9300}, disconnecting...
org.elasticsearch.transport.ReceiveTimeoutTransportException: [][192.168.1.100:9300][cluster:monitor/nodes/liveness] request_id [31] timed out after [5000ms]
    at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:698)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
2017-09-07 10:42:46.105 [elasticsearch[Digitek][generic][T#1]] INFO  org.elasticsearch.client.transport  - [Digitek] failed to get node info for {#transport#-1}{192.168.1.100}{192.168.1.100:9300}, disconnecting...
org.elasticsearch.transport.ReceiveTimeoutTransportException: [][192.168.1.100:9300][cluster:monitor/nodes/liveness] request_id [13] timed out after [5000ms]
    at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:698)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

查看服务器的内存 free -h 发现内存不足

改了一台内存大的服务器，ES连接的地方改成了单利，全局就建立一个连接ES对象。

posted on 2017-09-07 11:17 流年染指了悲伤阅读(7492) 评论(0) 编辑收藏举报