JVM宕机分析

1、JVM崩溃常见原因

常见的能够引起JVM崩溃的原因主要有：

内存溢出
线程阻塞
CPU 使用率过高
JVM Crash
Java虚拟机自身的Bug
JDK与服务器（CPU、内存、操作系统）的兼容性

2、heapdump文件分析（内存溢出）

Java程序运行时，有时会产生JavaCore及HeapDump文件，它一般发生于Java程序遇到致命问题的情况下。

有时致命问题发生后，Java应用不会死掉，还能继续运行；
但有时致命问题发生，Java进程会死掉；

为了能够保留Java应用发生致命错误前的运行状态，JVM在死掉前产生两个文件，分别为JavaCore及HeapDump文件。javacore文件一般是文本文件，而heapdump则一般是二进制文件，需要使用专门的分析工具进行分析，一般常用的工具为IBM的heap anylizer。

2.1、heapdump 文件生成

手动调命令生成

可以在内存升高时手动通过 jmap 命令手动生成当前时刻的 dump 文件，参考：https://www.cnblogs.com/wenxuehai/p/16600216.html#_label2_0_0

配置自动生成

可以配置当程序发生内存溢出时自动生成 dump 文件，如下：

-XX:+HeapDumpOnOutOfMemoryError # 表示当JVM发生OOM时，自动生成DUMP文件。
-XX:HeapDumpPath=${目录} # 指定生成DUMP文件的路径，也可以指定文件名称，例如：-XX:HeapDumpPath=${目录}/java_heapdump.hprof。如果不指定文件名，默认会在项目根目录下生成一个文件，文件名格式为：java_<pid>_<date>_<time>_heapDump.hprof。

（注意，IBM JDK在内存溢出时会自动生成 heapdump 文件，无需在 jvm 配置参数，这点跟我们常用的 Oracle JDK 不一样）

示例如下，只指定了目录：

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=F:/myHeapdumpTest

参考：https://blog.csdn.net/AikesLs/article/details/124290879

（注意，IBM JDK 的 dump 文件以 .phd 为后缀，IBM 堆转储文件通常以以下方式命名：heapdump.<date>..<timestamp><pid>.phd ）

2.2、heapdump 文件分析

heapdump 文件可通过 MAT 分析工具分析，参考：https://www.cnblogs.com/wenxuehai/p/16600216.html#autoid-h2-5-0-2

3、Java Core文件

Javacore 又称 thread dump，是Java虚拟机（JVM）的一种转储文件格式，其中包含了当前系统中所有线程的状态和调用堆栈信息。这个文件通常用于诊断系统问题、分析性能瓶颈以及检查线程死锁等情况。JavaCore文件可以通过工具进行分析，以获取有关系统状态的详细信息。

Javacore 是线程堆栈快照文件，能够利用该文件对 JVM 当前时间节点正则运行的线程进行分析，一般间隔 10-20s 取一次，连续取三次。一般两个以上 Javacore 文件分析才比较有效果，可以通过两个时间节点的线程执行情况进行分析，如果两个 Javacore 文件都在执行某个方法的相关代码，说明该方法的执行耗时很长，需要进行优化。

常见需要利用 Javacore 分析的现象：

CPU 使用率高、load 高、响应很慢
需查找占用CPU最多的线程信息
CPU使用率不高但是响应时间很长或无法响应

3.1、Javacore 文件生成

可以使用 jdk 自带命令 jstack 生成 Javacore 文件（Linux 和 window 环境都可以），命令如下：

jstack pid > pid_stack.log
# 示例：
jstack 22656 > pid_stack.log

或使用 kill -3 pid 命令也可生成 Javacore 文件（但我在虚拟机上执行时只在控制台打印了堆栈信息没有生成文件）

3.2、Javacore 文件分析

可以通过 IBM Javacore 分析工具 jca 分析，如版本4618 下载路径：https://public.dhe.ibm.com/software/websphere/appserv/support/tools/jca/jca4618.jar，下载完后直接通过命令行 java -jar 4618.jar 即可启动。

通过工具打开生成的 Javacore 文件，可以看到下图：

线程状态解释：

Deadlock：死锁（重点关注）
Runnable：运行中或可运行（重点关注）
Waiting on condition：等待资源（重点关注）
Waiting on monitor：等待监控器检查资源
Suspended：暂停
Object.wait()：对象等待中
Blocked：阻塞（重点关注）
Parked：停止

Javacore 文件分析思路：

程序发生内存溢出时会自动生成Javacore 文件，一般有几份，首先应该关注最早生成的文件。

先在顶部概况查看Javacore生成原因。主要有以下四种：
1. 人工生成：Dump Event "user" (00004000) received
2. 内存溢出：Dump Event "systhrow" (00040000) Detail "java/lang/OutOfMemoryError" "Java heap space" received
3. 一般性保护错误：Dump Event "gpt" (00002000) received
4. 无法创建线程：Dump Event "systhrow" (00040000) Detail "java/lang/OutOfMemoryError" "Failed to create a thread: retVal -1073741830, errno 11"
查看当前线程数是否正常（一般几百到一千）
查看jdk版本，有可能是JDK版本低引起的故障
查看jvm参数，看下jvm配置有没配置不合理的
查看用户资源限制参数（User Process Resource Limit Analysis）。主要可以查看 RLIMIT_NPROC 配置，该配置的 soft limit定义允许创建线程数。
查看内存使用情况（Memory Segment Analysis）
当前线程堆栈信息（Current Thread）。可以从 java stack分析出当前线程正在进行的操作

然后分析线程堆栈

点击线程详情按钮（齿轮状），然后将线程按深度降序排序
分别查看几个线程的执行情况（可以查看按深度排序后第一个线程，第一个等待状态的线程，和占用锁的线程）
从右边堆栈信息（Java Stack）中查找相关业务代码，分析当前线程正在进行什么相关操作

3.3、JavaCore 和 HeapDump 的区别

HeapDump文件是关于内存的。

HeapDump文件是一个二进制文件，它保存了某一时刻JVM堆中对象使用情况，这种文件需要相应的工具进行分析，如IBM Heap Analyzer这类工具。这类文件最重要的作用就是分析系统中是否存在内存溢出的情况。

JavaCore是关于CPU的

JavaCore文件主要保存的是Java应用各线程在某一时刻的运行的位置，即JVM执行到哪一个类、哪一个方法、哪一个行上。它是一个文本文件，打开后可以看到每一个线程的执行栈，以stack trace的显示。通过对JavaCore文件的分析可以得到应用是否“卡”在某一点上，即在某一点运行的时间太长，例如数据库查询，长期得不到响应，最终导致系统崩溃等情况。

4、GC日志文件

4.1、生成 GC 日志

IBM JDK

-verbose:gc ，打开GC日志，如果应用运行在WAS下，可通过 WAS 下 Java 虚拟机页面勾选”详细垃圾回收“自动设置，无需手工设置。如果应用没有运行在 WAS 下但使用了 IBM JDK，则增加参数-verbose:gc 方式来打开 GC 日志。
-Xverbosegclog:${SERVER_LOG_ROOT}/verbosegc.%seq.log,10,21000 ，打开GC日志后，启用GC日志自动滚动，保留最近10个日志，每个日志文件约30M（注意，${SERVER_LOG_ROOT} 为WAS的内置变量，会自动替换为当前AppServer的日志目录，不需要修改）

IBM JDK的日志文件是 XML 格式，内容比较详细。

Oracle JDK

配置如下：

-Xloggc:/xxx/xxx/app_gc.log   #日志文件的输出路径，如window环境可配置为 -Xloggc:F:/MY_JVM_ANALYZE/GC_LOG/temp_gc.log
-XX:+PrintGCDetails     #输出GC的详细日志
-XX:+PrintGCDatestamps  #输出GC的时间戳（以日期的形式，如2020-05-04T21：53：59.234+0800）
-XX:+UseGCLogFileRotation  #启用GC日志滚动
-XX:NumberOfGCLogFiles=5    #GC日志文件保留数量，建议为5
-XX:GCLogFilesize=20M     #GC日志文件大小，建议为20M

配置以上参数后，项目启动时即会生成 gc 日志文件。发生垃圾回收时也会更新日志文件。Oracle JDK的日志文件是文本形式，内容比较简短。

4.2、GC日志分析

如何通过 GC日志判断应用的性能状况：

GC评分项	计算方式	评分占用分数	参考值	说明
吞吐率	1-(GC消耗总时间/程序运行时间）*100%	20	99.99%	吞吐率越高越好，即GC时间越短越好，表示GC对于程序运行的影响越低
FullGC后老年代内存或者堆内存使用率	(FuIlGC后老年代内存使用量或者整个堆内存使用量/老年代已分配内存或者整个队内存已分配内存）*100%	30	20%	FullGC后内存使用率越低越好，表示GC能够回收越多内存
FuIlGC发生次数	统计FulIlGC在工作日繁忙时段（9:00-18：00）内发生的次数	20	3	FullGC次数越少越好表示对象生命周期控制合理，老年代堆积程度较低
FullGC发生平均时间间隔	統计FullGC在工作日繁忙时段（9:00-18:00）发生的平均时间间隔（秒）	20	10800（即3小时）	老年代GC间隔越长越好，表示对象生命周期较短，老年代堆积程度较低
新生代GC发生平均时间间隔	统计新生代GC在工作日繁忙时段（9:00-18.00）发生的平均时间间隔（秒）	10	30	新生代GC间隔越长越好，表示堆内存配置较为合理

正常的 GC 日志曲线呈锯齿状，并且 FULL GCC后，内存使用空间会降低到一个比较平均的值。如果 FULL GC后，内存没有下降并且有上涨趋势，则可以认为不正常。

业界较为流行使用 GCViewer 工具分析GC日志，下载地址：https://github.com/chewiebug/GCViewer/tree/1.36

Full GC Lines：（full gc）
Inc GC Lines：（增量GC）
GC Times Line：（gc 时间）
GC Times Rectangles：（gc时间区域）
Total Heap：(总堆大小)
Red line that shows heap size
Tenured Generation：（老年代）
Young Generation：（年轻代）
Used Heap：（堆使用量）
Initial mark level：（cms或g1垃圾回收算法初始标记事件）
Concurrent collections：（cms或g1垃圾回收并发收集周期）

示例如下图，堆使用的越来越高，Full GC 频繁触发，且FULL GC后内存并没有降下来。

5、hs_err_pid 日志文件分析

hs_err_pid.log 文件是一个 Java 虚拟机（JVM）生成的错误报告日志文件，通常包含了 JVM 崩溃时的详细信息。当 JVM 因为各种原因发生致命错误时（包括内存溢出（OutOfMemoryError）、线程死锁、系统资源不足等问题），会生成一个hs_err_pid_xxx.log（后面的xxx为进程id）这样的文件，该文件包含了导致 JVM crash（JVM崩溃）的重要信息，我们可以通过分析该文件定位到导致 JVM Crash 的原因，从而修复保证系统稳定。

（注意，IBM JDK 崩溃时并不会生成 hs_err_pid.log 文件，只有 Oracle JDK和open JDK 才有这个机制）

5.1、生成hs_err_pid 日志文件

默认情况下，该文件是生成在当前程序工作目录下的（即启动该程序的目录，而不是该程序的存放目录，比如在 /usr/ 目录通过 java -jar 命令运行 /usr/mysoft/springbootTest01.jar 包，当 jvm 崩溃时，hs_err_pid 文件会生成在 /usr/ 目录下而不是 /usr/mysoft/ 目录）。默认该文件文件名为 hs_err_pid<pid>.log，后面的 pid 为进程号。如下：

当然也可以通过 JVM 参数指定生成路径：

-XX:ErrorFile=./hs_err_pid<pid>.log  # 注意 <pid>是进程号，需指定，直接写 <pid> 程序无法运行

示例：

5.1.1、查看生成路径

可以通过以下命令查看 hs_err_pid.log 文件的存放目录，如下：

ls -l /proc/<pid>/cwd

#示例：
ls -l /proc/2658/cwd

如下图：

5.1.2、模拟生成

模拟内存溢出代码：

@RequestMapping("/DumpTest")
@RestController
public class DumpTestController {

    @RequestMapping("test01")
    public String hello() throws Exception {
        String name = "Aikes";
        for (int i = 0; i < 10000000; i++) {
            name += name;
        }
        System.out.println(name);

        return "test";
    }
}

打包项目，然后通过 jar 包运行，如下：

java -XX:+CrashOnOutOfMemoryError -jar ./springbootTest01-1.0-SNAPSHOT.jar

-XX:+CrashOnOutOfMemoryError：使JVM在内存溢出时立即崩溃。该配置参数可指定当JVM检测到OutOfMemoryError（内存溢出错误）时立即崩溃，这个参数可以用来强制Java应用在发生内存溢出时立即终止，避免某些情况下由于自我修复机制（如JVM的垃圾收集器尝试释放更多内存）而使得问题被掩盖或延后发现。

然后访问服务器的接口，如：http://192.168.118.130:8080/DumpTest/test01，程序将会发生内存溢出，随之生成 hs_err_pid 日志文件和 core 文件（如果没生成core文件可能需要执行 ulimit -c unlimited 命令，core文件即 coredump 文件，该文件在 jvm 崩溃时也会自动生成）。如下：

以下为模拟 jvm 崩溃时生成的日志文件内容实例：

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (debug.cpp:312), pid=37121, tid=0x00007f1747dff700
#  OutOfMemory encountered: : Java heap space
#
# JRE version: Java(TM) SE Runtime Environment (8.0_341-b10) (build 1.8.0_341-b10)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.341-b10 mixed mode linux-amd64 compressed oops)
# Core dump written. Default location: /usr/core or core.37121
#

---------------  T H R E A D  ---------------

Current thread (0x00007f177845b800):  JavaThread "http-nio-8080-exec-1" daemon [_thread_in_vm, id=37153, stack(0x00007f1747cff000,0x00007f1747e00000)]

Stack: [0x00007f1747cff000,0x00007f1747e00000],  sp=0x00007f1747dfc3d0,  free space=1012k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xae0865]  VMError::report_and_die()+0x2e5
V  [libjvm.so+0x4e2ddd]  report_java_out_of_memory(char const*)+0x13d
V  [libjvm.so+0xaa0151]  TypeArrayKlass::allocate_common(int, bool, Thread*)+0x161
V  [libjvm.so+0x9c23b6]  OptoRuntime::new_array_nozero_C(Klass*, int, JavaThread*)+0x26
v  ~RuntimeStub::_new_array_nozero_Java
J 837 C2 java.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder; (8 bytes) @ 0x00007f1769324c08 [0x00007f1769324980+0x288]
j  start.controller.DumpTestController.hello()Ljava/lang/String;+23
v  ~StubRoutines::call_stub
V  [libjvm.so+0x68bc3a]  JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0xe1a
V  [libjvm.so+0x9a3e28]  Reflection::invoke(instanceKlassHandle, methodHandle, Handle, bool, objArrayHandle, BasicType, objArrayHandle, bool, Thread*)+0xae8
V  [libjvm.so+0x9a57f6]  Reflection::invoke_method(oopDesc*, Handle, objArrayHandle, Thread*)+0x126
V  [libjvm.so+0x703095]  JVM_InvokeMethod+0x115
J 2057  sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (0 bytes) @ 0x00007f176941d8ad [0x00007f176941d7c0+0xed]
J 2056 C1 sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (104 bytes) @ 0x00007f17693eac2b [0x00007f17693ea8a0+0x38b]
J 948 C1 sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (10 bytes) @ 0x00007f1769351c03 [0x00007f1769351b00+0x103]
J 947 C1 java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (62 bytes) @ 0x00007f1769351636 [0x00007f1769351280+0x3b6]
j  org.springframework.web.method.support.InvocableHandlerMethod.doInvoke([Ljava/lang/Object;)Ljava/lang/Object;+16
j  org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(Lorg/springframework/web/context/request/NativeWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)Ljava/lang/Object;+75
j  org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(Lorg/springframework/web/context/request/ServletWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)V+4
j  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView;+276
j  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView;+81
j  org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljava/lang/Object;)Lorg/springframework/web/servlet/ModelAndView;+7
j  org.springframework.web.servlet.DispatcherServlet.doDispatch(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+310
j  org.springframework.web.servlet.DispatcherServlet.doService(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+308
j  org.springframework.web.servlet.FrameworkServlet.processRequest(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+71
j  org.springframework.web.servlet.FrameworkServlet.doGet(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+3
j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+35
j  org.springframework.web.servlet.FrameworkServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+33
j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+30
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+304
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.apache.tomcat.websocket.server.WsFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+21
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.springframework.web.filter.RequestContextFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+21
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+95
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+87
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+53
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.apache.catalina.core.StandardWrapperValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+688
j  org.apache.catalina.core.StandardContextValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+166
j  org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+260
j  org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+138
j  org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6
j  org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+71
j  org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V+199
j  org.apache.coyote.http11.Http11Processor.service(Lorg/apache/tomcat/util/net/SocketWrapperBase;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+806
j  org.apache.coyote.AbstractProcessorLight.process(Lorg/apache/tomcat/util/net/SocketWrapperBase;Lorg/apache/tomcat/util/net/SocketEvent;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+113
j  org.apache.coyote.AbstractProtocol$ConnectionHandler.process(Lorg/apache/tomcat/util/net/SocketWrapperBase;Lorg/apache/tomcat/util/net/SocketEvent;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+414
j  org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun()V+191
j  org.apache.tomcat.util.net.SocketProcessorBase.run()V+21
j  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j  org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run()V+4
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub
V  [libjvm.so+0x68bc3a]  JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0xe1a
V  [libjvm.so+0x6892f3]  JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x263
V  [libjvm.so+0x6898c7]  JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*, Symbol*, Thread*)+0x57
V  [libjvm.so+0x6f72bc]  thread_entry(JavaThread*, Thread*)+0x6c
V  [libjvm.so+0xa88f8b]  JavaThread::thread_main_inner()+0xdb
V  [libjvm.so+0xa89291]  JavaThread::run()+0x2d1
V  [libjvm.so+0x9173e2]  java_start(Thread*)+0x102
C  [libpthread.so.0+0x7ea5]  start_thread+0xc5

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
v  ~RuntimeStub::_new_array_nozero_Java
J 837 C2 java.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder; (8 bytes) @ 0x00007f1769324c08 [0x00007f1769324980+0x288]
j  start.controller.DumpTestController.hello()Ljava/lang/String;+23
v  ~StubRoutines::call_stub
J 2057  sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (0 bytes) @ 0x00007f176941d837 [0x00007f176941d7c0+0x77]
J 2056 C1 sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (104 bytes) @ 0x00007f17693eac2b [0x00007f17693ea8a0+0x38b]
J 948 C1 sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (10 bytes) @ 0x00007f1769351c03 [0x00007f1769351b00+0x103]
J 947 C1 java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (62 bytes) @ 0x00007f1769351636 [0x00007f1769351280+0x3b6]
j  org.springframework.web.method.support.InvocableHandlerMethod.doInvoke([Ljava/lang/Object;)Ljava/lang/Object;+16
j  org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(Lorg/springframework/web/context/request/NativeWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)Ljava/lang/Object;+75
j  org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(Lorg/springframework/web/context/request/ServletWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)V+4
j  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView;+276
j  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView;+81
j  org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljava/lang/Object;)Lorg/springframework/web/servlet/ModelAndView;+7
j  org.springframework.web.servlet.DispatcherServlet.doDispatch(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+310
j  org.springframework.web.servlet.DispatcherServlet.doService(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+308
j  org.springframework.web.servlet.FrameworkServlet.processRequest(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+71
j  org.springframework.web.servlet.FrameworkServlet.doGet(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+3
j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+35
j  org.springframework.web.servlet.FrameworkServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+33
j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+30
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+304
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.apache.tomcat.websocket.server.WsFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+21
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.springframework.web.filter.RequestContextFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+21
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+95
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+87
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+53
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
j  org.apache.catalina.core.StandardWrapperValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+688
j  org.apache.catalina.core.StandardContextValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+166
j  org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+260
j  org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+138
j  org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6
j  org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+71
j  org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V+199
j  org.apache.coyote.http11.Http11Processor.service(Lorg/apache/tomcat/util/net/SocketWrapperBase;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+806
j  org.apache.coyote.AbstractProcessorLight.process(Lorg/apache/tomcat/util/net/SocketWrapperBase;Lorg/apache/tomcat/util/net/SocketEvent;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+113
j  org.apache.coyote.AbstractProtocol$ConnectionHandler.process(Lorg/apache/tomcat/util/net/SocketWrapperBase;Lorg/apache/tomcat/util/net/SocketEvent;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+414
j  org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun()V+191
j  org.apache.tomcat.util.net.SocketProcessorBase.run()V+21
j  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j  org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run()V+4
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x00007f1778009800 JavaThread "DestroyJavaVM" [_thread_blocked, id=37122, stack(0x00007f177f563000,0x00007f177f663000)]
  0x00007f1778878800 JavaThread "http-nio-8080-AsyncTimeout" daemon [_thread_blocked, id=37165, stack(0x00007f17470f3000,0x00007f17471f4000)]
  0x00007f1778810800 JavaThread "http-nio-8080-Acceptor-0" daemon [_thread_in_native, id=37164, stack(0x00007f17471f4000,0x00007f17472f5000)]
  0x00007f177880f000 JavaThread "http-nio-8080-ClientPoller-0" daemon [_thread_in_native, id=37163, stack(0x00007f17472f5000,0x00007f17473f6000)]
  0x00007f17785e8000 JavaThread "http-nio-8080-exec-10" daemon [_thread_blocked, id=37162, stack(0x00007f17473f6000,0x00007f17474f7000)]
  0x00007f17788d2800 JavaThread "http-nio-8080-exec-9" daemon [_thread_blocked, id=37161, stack(0x00007f17474f7000,0x00007f17475f8000)]
  0x00007f17781ee800 JavaThread "http-nio-8080-exec-8" daemon [_thread_blocked, id=37160, stack(0x00007f17475f8000,0x00007f17476f9000)]
  0x00007f17788c1000 JavaThread "http-nio-8080-exec-7" daemon [_thread_blocked, id=37159, stack(0x00007f17476f9000,0x00007f17477fa000)]
  0x00007f17788de000 JavaThread "http-nio-8080-exec-6" daemon [_thread_blocked, id=37158, stack(0x00007f17477fa000,0x00007f17478fb000)]
  0x00007f17786e3000 JavaThread "http-nio-8080-exec-5" daemon [_thread_blocked, id=37157, stack(0x00007f17478fb000,0x00007f17479fc000)]
  0x00007f1778377000 JavaThread "http-nio-8080-exec-4" daemon [_thread_blocked, id=37156, stack(0x00007f17479fc000,0x00007f1747afd000)]
  0x00007f1778316000 JavaThread "http-nio-8080-exec-3" daemon [_thread_blocked, id=37155, stack(0x00007f1747afd000,0x00007f1747bfe000)]
  0x00007f17784f5800 JavaThread "http-nio-8080-exec-2" daemon [_thread_blocked, id=37154, stack(0x00007f1747bfe000,0x00007f1747cff000)]
=>0x00007f177845b800 JavaThread "http-nio-8080-exec-1" daemon [_thread_in_vm, id=37153, stack(0x00007f1747cff000,0x00007f1747e00000)]
  0x00007f1778364000 JavaThread "NioBlockingSelector.BlockPoller-1" daemon [_thread_in_native, id=37152, stack(0x00007f1754035000,0x00007f1754136000)]
  0x00007f17788a4000 JavaThread "container-0" [_thread_blocked, id=37149, stack(0x00007f1754136000,0x00007f1754237000)]
  0x00007f1778899000 JavaThread "ContainerBackgroundProcessor[StandardEngine[Tomcat]]" daemon [_thread_blocked, id=37148, stack(0x00007f1754237000,0x00007f1754338000)]
  0x00007f17780df800 JavaThread "Service Thread" daemon [_thread_blocked, id=37129, stack(0x00007f1768718000,0x00007f1768819000)]
  0x00007f17780da800 JavaThread "C1 CompilerThread1" daemon [_thread_blocked, id=37128, stack(0x00007f176881a000,0x00007f176891a000)]
  0x00007f17780d8800 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=37127, stack(0x00007f176891b000,0x00007f1768a1b000)]
  0x00007f17780d7000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=37126, stack(0x00007f1768a1b000,0x00007f1768b1c000)]
  0x00007f17780a4000 JavaThread "Finalizer" daemon [_thread_blocked, id=37125, stack(0x00007f1768b1c000,0x00007f1768c1d000)]
  0x00007f177809f000 JavaThread "Reference Handler" daemon [_thread_blocked, id=37124, stack(0x00007f1768c1d000,0x00007f1768d1e000)]

Other Threads:
  0x00007f1778095800 VMThread [stack: 0x00007f1768d1f000,0x00007f1768e1f000] [id=37123]
  0x00007f17780e2800 WatcherThread [stack: 0x00007f1768618000,0x00007f1768718000] [id=37130]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

heap address: 0x00000000f0c00000, size: 244 MB, Compressed Oops mode: 32-bit
Narrow klass base: 0x0000000000000000, Narrow klass shift: 3
Compressed class space size: 1073741824 Address: 0x0000000100000000

Heap:
 def new generation   total 74944K, used 216K [0x00000000f0c00000, 0x00000000f5d50000, 0x00000000f5d50000)
  eden space 66624K,   0% used [0x00000000f0c00000, 0x00000000f0c36210, 0x00000000f4d10000)
  from space 8320K,   0% used [0x00000000f4d10000, 0x00000000f4d10000, 0x00000000f5530000)
  to   space 8320K,   0% used [0x00000000f5530000, 0x00000000f5530000, 0x00000000f5d50000)
 tenured generation   total 166592K, used 95847K [0x00000000f5d50000, 0x0000000100000000, 0x0000000100000000)
   the space 166592K,  57% used [0x00000000f5d50000, 0x00000000fbae9fb8, 0x00000000fbaea000, 0x0000000100000000)
 Metaspace       used 34695K, capacity 36218K, committed 36608K, reserved 1081344K
  class space    used 4483K, capacity 4785K, committed 4864K, reserved 1048576K

Card table byte_map: [0x00007f177f4e7000,0x00007f177f562000] byte_map_base: 0x00007f177ed61000

Polling page: 0x00007f177f67d000

CodeCache: size=245760Kb used=8230Kb max_used=8288Kb free=237529Kb
 bounds [0x00007f1769000000, 0x00007f1769840000, 0x00007f1778000000]
 total_blobs=3257 nmethods=2837 adapters=332
 compilation: enabled

Compilation events (10 events):
Event: 19.376 Thread 0x00007f17780d8800 3251       4       java.net.URLClassLoader$1::run (5 bytes)
Event: 19.376 Thread 0x00007f17780d8800 nmethod 3251 0x00007f1769608050 code [0x00007f17696081a0, 0x00007f1769608228]
Event: 19.376 Thread 0x00007f17780d8800 3254       4       java.util.AbstractMap::<init> (5 bytes)
Event: 19.376 Thread 0x00007f17780d8800 nmethod 3254 0x00007f17692b5fd0 code [0x00007f17692b6120, 0x00007f17692b6178]
Event: 19.376 Thread 0x00007f17780d8800 3269       4       java.lang.ref.Finalizer::access$100 (6 bytes)
Event: 19.383 Thread 0x00007f17780d8800 nmethod 3269 0x00007f1769178510 code [0x00007f17691786c0, 0x00007f1769178fa8]
Event: 19.383 Thread 0x00007f17780d8800 3270   !   4       java.lang.ref.Finalizer::runFinalizer (62 bytes)
Event: 19.414 Thread 0x00007f17780d8800 nmethod 3270 0x00007f176923e890 code [0x00007f176923ea40, 0x00007f176923f328]
Event: 19.414 Thread 0x00007f17780d8800 3271   !   4       java.lang.ref.Finalizer::remove (94 bytes)
Event: 19.416 Thread 0x00007f17780d8800 nmethod 3271 0x00007f1769533290 code [0x00007f17695333e0, 0x00007f17695336d8]

GC Heap History (10 events):
Event: 19.153 GC heap before
{Heap before GC invocations=123 (full 6):
 def new generation   total 35200K, used 20480K [0x00000000f0c00000, 0x00000000f3230000, 0x00000000f5d50000)
  eden space 31296K,  65% used [0x00000000f0c00000, 0x00000000f2000010, 0x00000000f2a90000)
  from space 3904K,   0% used [0x00000000f2e60000, 0x00000000f2e60048, 0x00000000f3230000)
  to   space 3904K,   0% used [0x00000000f2a90000, 0x00000000f2a90000, 0x00000000f2e60000)
 tenured generation   total 78124K, used 67352K [0x00000000f5d50000, 0x00000000fa99b000, 0x0000000100000000)
   the space 78124K,  86% used [0x00000000f5d50000, 0x00000000f9f161a0, 0x00000000f9f16200, 0x00000000fa99b000)
 Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
Event: 19.258 GC heap after
Heap after GC invocations=124 (full 7):
 def new generation   total 42880K, used 0K [0x00000000f0c00000, 0x00000000f3a80000, 0x00000000f5d50000)
  eden space 38144K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f3140000)
  from space 4736K,   0% used [0x00000000f3140000, 0x00000000f3140000, 0x00000000f35e0000)
  to   space 4736K,   0% used [0x00000000f35e0000, 0x00000000f35e0000, 0x00000000f3a80000)
 tenured generation   total 95188K, used 57112K [0x00000000f5d50000, 0x00000000fba45000, 0x0000000100000000)
   the space 95188K,  59% used [0x00000000f5d50000, 0x00000000f9516188, 0x00000000f9516200, 0x00000000fba45000)
 Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
}
Event: 19.302 GC heap before
{Heap before GC invocations=124 (full 7):
 def new generation   total 42880K, used 736K [0x00000000f0c00000, 0x00000000f3a80000, 0x00000000f5d50000)
  eden space 38144K,   1% used [0x00000000f0c00000, 0x00000000f0cb8028, 0x00000000f3140000)
  from space 4736K,   0% used [0x00000000f3140000, 0x00000000f3140000, 0x00000000f35e0000)
  to   space 4736K,   0% used [0x00000000f35e0000, 0x00000000f35e0000, 0x00000000f3a80000)
 tenured generation   total 136152K, used 98072K [0x00000000f5d50000, 0x00000000fe246000, 0x0000000100000000)
   the space 136152K,  72% used [0x00000000f5d50000, 0x00000000fbd161a0, 0x00000000fbd16200, 0x00000000fe246000)
 Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
Event: 19.351 GC heap after
Heap after GC invocations=125 (full 8):
 def new generation   total 61312K, used 0K [0x00000000f0c00000, 0x00000000f4e80000, 0x00000000f5d50000)
  eden space 54528K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f4140000)
  from space 6784K,   0% used [0x00000000f4140000, 0x00000000f4140000, 0x00000000f47e0000)
  to   space 6784K,   0% used [0x00000000f47e0000, 0x00000000f47e0000, 0x00000000f4e80000)
 tenured generation   total 136152K, used 77573K [0x00000000f5d50000, 0x00000000fe246000, 0x0000000100000000)
   the space 136152K,  56% used [0x00000000f5d50000, 0x00000000fa911500, 0x00000000fa911600, 0x00000000fe246000)
 Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
}
Event: 19.387 GC heap before
{Heap before GC invocations=125 (full 8):
 def new generation   total 61312K, used 42012K [0x00000000f0c00000, 0x00000000f4e80000, 0x00000000f5d50000)
  eden space 54528K,  77% used [0x00000000f0c00000, 0x00000000f35070f8, 0x00000000f4140000)
  from space 6784K,   0% used [0x00000000f4140000, 0x00000000f4140000, 0x00000000f47e0000)
  to   space 6784K,   0% used [0x00000000f47e0000, 0x00000000f47e0000, 0x00000000f4e80000)
 tenured generation   total 136152K, used 77573K [0x00000000f5d50000, 0x00000000fe246000, 0x0000000100000000)
   the space 136152K,  56% used [0x00000000f5d50000, 0x00000000fa911500, 0x00000000fa911600, 0x00000000fe246000)
 Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
Event: 19.411 GC heap after
Heap after GC invocations=126 (full 8):
 def new generation   total 61312K, used 0K [0x00000000f0c00000, 0x00000000f4e80000, 0x00000000f5d50000)
  eden space 54528K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f4140000)
  from space 6784K,   0% used [0x00000000f47e0000, 0x00000000f47e0048, 0x00000000f4e80000)
  to   space 6784K,   0% used [0x00000000f4140000, 0x00000000f4140000, 0x00000000f47e0000)
 tenured generation   total 136152K, used 118533K [0x00000000f5d50000, 0x00000000fe246000, 0x0000000100000000)
   the space 136152K,  87% used [0x00000000f5d50000, 0x00000000fd111510, 0x00000000fd111600, 0x00000000fe246000)
 Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
}
Event: 19.426 GC heap before
{Heap before GC invocations=126 (full 8):
 def new generation   total 61312K, used 40960K [0x00000000f0c00000, 0x00000000f4e80000, 0x00000000f5d50000)
  eden space 54528K,  75% used [0x00000000f0c00000, 0x00000000f3400010, 0x00000000f4140000)
  from space 6784K,   0% used [0x00000000f47e0000, 0x00000000f47e0048, 0x00000000f4e80000)
  to   space 6784K,   0% used [0x00000000f4140000, 0x00000000f4140000, 0x00000000f47e0000)
 tenured generation   total 136152K, used 118533K [0x00000000f5d50000, 0x00000000fe246000, 0x0000000100000000)
   the space 136152K,  87% used [0x00000000f5d50000, 0x00000000fd111510, 0x00000000fd111600, 0x00000000fe246000)
 Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
Event: 19.513 GC heap after
Heap after GC invocations=127 (full 9):
 def new generation   total 73600K, used 0K [0x00000000f0c00000, 0x00000000f5bd0000, 0x00000000f5d50000)
  eden space 65472K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f4bf0000)
  from space 8128K,   0% used [0x00000000f4bf0000, 0x00000000f4bf0000, 0x00000000f53e0000)
  to   space 8128K,   0% used [0x00000000f53e0000, 0x00000000f53e0000, 0x00000000f5bd0000)
 tenured generation   total 163424K, used 98053K [0x00000000f5d50000, 0x00000000ffce8000, 0x0000000100000000)
   the space 163424K,  59% used [0x00000000f5d50000, 0x00000000fbd114f8, 0x00000000fbd11600, 0x00000000ffce8000)
 Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
}
Event: 19.513 GC heap before
{Heap before GC invocations=127 (full 9):
 def new generation   total 73600K, used 0K [0x00000000f0c00000, 0x00000000f5bd0000, 0x00000000f5d50000)
  eden space 65472K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f4bf0000)
  from space 8128K,   0% used [0x00000000f4bf0000, 0x00000000f4bf0000, 0x00000000f53e0000)
  to   space 8128K,   0% used [0x00000000f53e0000, 0x00000000f53e0000, 0x00000000f5bd0000)
 tenured generation   total 166592K, used 98053K [0x00000000f5d50000, 0x0000000100000000, 0x0000000100000000)
   the space 166592K,  58% used [0x00000000f5d50000, 0x00000000fbd114f8, 0x00000000fbd11600, 0x0000000100000000)
 Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
Event: 19.577 GC heap after
Heap after GC invocations=128 (full 10):
 def new generation   total 74944K, used 0K [0x00000000f0c00000, 0x00000000f5d50000, 0x00000000f5d50000)
  eden space 66624K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f4d10000)
  from space 8320K,   0% used [0x00000000f4d10000, 0x00000000f4d10000, 0x00000000f5530000)
  to   space 8320K,   0% used [0x00000000f5530000, 0x00000000f5530000, 0x00000000f5d50000)
 tenured generation   total 166592K, used 95847K [0x00000000f5d50000, 0x0000000100000000, 0x0000000100000000)
   the space 166592K,  57% used [0x00000000f5d50000, 0x00000000fbae9fb8, 0x00000000fbaea000, 0x0000000100000000)
 Metaspace       used 34695K, capacity 36218K, committed 36608K, reserved 1081344K
  class space    used 4483K, capacity 4785K, committed 4864K, reserved 1048576K
}

Deoptimization events (10 events):
Event: 2.693 Thread 0x00007f1778009800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f176942be4c method=sun.util.calendar.CalendarDate.setMillis(I)Lsun/util/calendar/CalendarDate; @ 5
Event: 4.904 Thread 0x00007f1778009800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f176963f620 method=java.lang.reflect.Method.equals(Ljava/lang/Object;)Z @ 1
Event: 5.143 Thread 0x00007f1778009800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f176960483c method=java.util.Properties$LineReader.readLine()I @ 323
Event: 5.352 Thread 0x00007f1778009800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f17695503b4 method=java.util.Properties.loadConvert([CII[C)Ljava/lang/String; @ 57
Event: 5.491 Thread 0x00007f1778009800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f1769497d60 method=sun.misc.ASCIICaseInsensitiveComparator.compare(Ljava/lang/String;Ljava/lang/String;)I @ 83
Event: 5.966 Thread 0x00007f17483e6000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00007f17696f9ec4 method=java.net.URL.getURLStreamHandler(Ljava/lang/String;)Ljava/net/URLStreamHandler; @ 214
Event: 6.363 Thread 0x00007f1778009800 Uncommon trap: reason=bimorphic action=maybe_recompile pc=0x00007f1769552060 method=java.util.Hashtable.get(Ljava/lang/Object;)Ljava/lang/Object; @ 6
Event: 6.363 Thread 0x00007f1778009800 Uncommon trap: reason=bimorphic action=maybe_recompile pc=0x00007f1769552060 method=java.util.Hashtable.get(Ljava/lang/Object;)Ljava/lang/Object; @ 6
Event: 7.889 Thread 0x00007f1778009800 Uncommon trap: reason=bimorphic action=maybe_recompile pc=0x00007f1769552060 method=java.util.Hashtable.get(Ljava/lang/Object;)Ljava/lang/Object; @ 6
Event: 7.906 Thread 0x00007f1778009800 Uncommon trap: reason=bimorphic action=maybe_recompile pc=0x00007f1769552060 method=java.util.Hashtable.get(Ljava/lang/Object;)Ljava/lang/Object; @ 6

Classes redefined (0 events):
No events

Internal exceptions (10 events):
Event: 18.546 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/theme/AbstractThemeResolverCustomizer> (0x00000000f0c52d40) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDicti
Event: 18.546 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/theme/FixedThemeResolverCustomizer> (0x00000000f0c8a9e8) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDictiona
Event: 18.546 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/ThemeResolverCustomizer> (0x00000000f0cbf458) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDictionary.cpp, lin
Event: 18.548 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': java/lang/ObjectCustomizer> (0x00000000f0cfc658) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 212]
Event: 18.549 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/view/DefaultRequestToViewNameTranslatorCustomizer> (0x00000000f0d39118) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile
Event: 18.549 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/RequestToViewNameTranslatorCustomizer> (0x00000000f0d800a8) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDicti
Event: 18.570 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': java/lang/ObjectCustomizer> (0x00000000f0df5870) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 212]
Event: 18.571 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/support/AbstractFlashMapManagerCustomizer> (0x00000000f0e2fa60) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemD
Event: 18.571 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/support/SessionFlashMapManagerCustomizer> (0x00000000f0e6d8e8) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDi
Event: 18.571 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/FlashMapManagerCustomizer> (0x00000000f0ea21f8) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDictionary.cpp, l

Events (10 events):
Event: 19.153 Executing VM operation: GenCollectForAllocation
Event: 19.258 Executing VM operation: GenCollectForAllocation done
Event: 19.266 Thread 0x00007f17780d8800 flushing nmethod 0x00007f1769708f50
Event: 19.286 Thread 0x00007f17780d8800 flushing nmethod 0x00007f1769742650
Event: 19.302 Executing VM operation: GenCollectForAllocation
Event: 19.351 Executing VM operation: GenCollectForAllocation done
Event: 19.386 Executing VM operation: GenCollectForAllocation
Event: 19.411 Executing VM operation: GenCollectForAllocation done
Event: 19.426 Executing VM operation: GenCollectForAllocation
Event: 19.577 Executing VM operation: GenCollectForAllocation done


Dynamic libraries:
f0c00000-f5d50000 rw-p 00000000 00:00 0 
f5d50000-100000000 rw-p 00000000 00:00 0 
100000000-1004c0000 rw-p 00000000 00:00 0 
1004c0000-140000000 ---p 00000000 00:00 0 
55711a3d5000-55711a3d6000 r-xp 00000000 fd:00 18321644                   /usr/java/jdk1.8.0_341-amd64/bin/java
55711a5d5000-55711a5d6000 r--p 00000000 fd:00 18321644                   /usr/java/jdk1.8.0_341-amd64/bin/java
55711a5d6000-55711a5d7000 rw-p 00001000 fd:00 18321644                   /usr/java/jdk1.8.0_341-amd64/bin/java
55711b577000-55711b598000 rw-p 00000000 00:00 0                          [heap]
7f1740000000-7f174040f000 rw-p 00000000 00:00 0 
7f174040f000-7f1744000000 ---p 00000000 00:00 0 
7f1746ef3000-7f1746ff3000 rw-p 00000000 00:00 0 
7f1746ff3000-7f17470f3000 ---p 00000000 00:00 0 
7f17470f3000-7f17470f6000 ---p 00000000 00:00 0 
7f17470f6000-7f17471f4000 rw-p 00000000 00:00 0 
7f17471f4000-7f17471f7000 ---p 00000000 00:00 0 
7f17471f7000-7f17472f5000 rw-p 00000000 00:00 0 
7f17472f5000-7f17472f8000 ---p 00000000 00:00 0 
7f17472f8000-7f17473f6000 rw-p 00000000 00:00 0 
7f17473f6000-7f17473f9000 ---p 00000000 00:00 0 
7f17473f9000-7f17474f7000 rw-p 00000000 00:00 0 
7f17474f7000-7f17474fa000 ---p 00000000 00:00 0 
7f17474fa000-7f17475f8000 rw-p 00000000 00:00 0 
7f17475f8000-7f17475fb000 ---p 00000000 00:00 0 
7f17475fb000-7f17476f9000 rw-p 00000000 00:00 0 
7f17476f9000-7f17476fc000 ---p 00000000 00:00 0 
7f17476fc000-7f17477fa000 rw-p 00000000 00:00 0 
7f17477fa000-7f17477fd000 ---p 00000000 00:00 0 
7f17477fd000-7f17478fb000 rw-p 00000000 00:00 0 
7f17478fb000-7f17478fe000 ---p 00000000 00:00 0 
7f17478fe000-7f17479fc000 rw-p 00000000 00:00 0 
7f17479fc000-7f17479ff000 ---p 00000000 00:00 0 
7f17479ff000-7f1747afd000 rw-p 00000000 00:00 0 
7f1747afd000-7f1747b00000 ---p 00000000 00:00 0 
7f1747b00000-7f1747bfe000 rw-p 00000000 00:00 0 
7f1747bfe000-7f1747c01000 ---p 00000000 00:00 0 
7f1747c01000-7f1747cff000 rw-p 00000000 00:00 0 
7f1747cff000-7f1747d02000 ---p 00000000 00:00 0 
7f1747d02000-7f1748000000 rw-p 00000000 00:00 0 
7f1748000000-7f1749c27000 rw-p 00000000 00:00 0 
7f1749c27000-7f174c000000 ---p 00000000 00:00 0 
7f174c000000-7f174cd44000 rw-p 00000000 00:00 0 
7f174cd44000-7f1750000000 ---p 00000000 00:00 0 
7f1750000000-7f1750170000 rw-p 00000000 00:00 0 
7f1750170000-7f1754000000 ---p 00000000 00:00 0 
7f1754035000-7f1754038000 ---p 00000000 00:00 0 
7f1754038000-7f1754136000 rw-p 00000000 00:00 0 
7f1754136000-7f1754139000 ---p 00000000 00:00 0 
7f1754139000-7f1754237000 rw-p 00000000 00:00 0 
7f1754237000-7f175423a000 ---p 00000000 00:00 0 
7f175423a000-7f1754538000 rw-p 00000000 00:00 0 
7f1754538000-7f1754738000 rw-p 00000000 00:00 0 
7f175478a000-7f175498a000 rw-p 00000000 00:00 0 
7f175498a000-7f175499b000 r-xp 00000000 fd:00 34650825                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnio.so
7f175499b000-7f1754b9a000 ---p 00011000 fd:00 34650825                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnio.so
7f1754b9a000-7f1754b9b000 r--p 00010000 fd:00 34650825                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnio.so
7f1754b9b000-7f1754b9c000 rw-p 00011000 fd:00 34650825                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnio.so
7f1754b9c000-7f1754d9c000 rw-p 00000000 00:00 0 
7f1754d9c000-7f1754d9f000 ---p 00000000 00:00 0 
7f1754d9f000-7f175509d000 rw-p 00000000 00:00 0 
7f175509d000-7f175529d000 rw-p 00000000 00:00 0 
7f175529d000-7f175549d000 rw-p 00000000 00:00 0 
7f175549d000-7f175569d000 rw-p 00000000 00:00 0 
7f175569d000-7f17556b3000 r-xp 00000000 fd:00 34650824                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnet.so
7f17556b3000-7f17558b2000 ---p 00016000 fd:00 34650824                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnet.so
7f17558b2000-7f17558b3000 r--p 00015000 fd:00 34650824                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnet.so
7f17558b3000-7f17558b4000 rw-p 00016000 fd:00 34650824                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnet.so
7f17558b4000-7f17558bd000 r-xp 00000000 fd:00 34650822                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libmanagement.so
7f17558bd000-7f1755abc000 ---p 00009000 fd:00 34650822                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libmanagement.so
7f1755abc000-7f1755abd000 r--p 00008000 fd:00 34650822                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libmanagement.so
7f1755abd000-7f1755abe000 rw-p 00009000 fd:00 34650822                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libmanagement.so
7f1755abe000-7f175c000000 r--p 00000000 fd:00 34119214                   /usr/lib/locale/locale-archive
7f175c000000-7f175c021000 rw-p 00000000 00:00 0 
7f175c021000-7f1760000000 ---p 00000000 00:00 0 
7f1760000000-7f1760021000 rw-p 00000000 00:00 0 
7f1760021000-7f1764000000 ---p 00000000 00:00 0 
7f1764000000-7f1764294000 rw-p 00000000 00:00 0 
7f1764294000-7f1768000000 ---p 00000000 00:00 0 
7f1768042000-7f1768053000 r--s 001c7000 fd:00 17815594                   /usr/java/jdk1.8.0_341-amd64/jre/lib/jsse.jar
7f1768053000-7f1768253000 rw-p 00000000 00:00 0 
7f1768253000-7f17682e9000 r--s 007bf000 fd:00 17505309                   /usr/java/jdk1.8.0_341-amd64/jre/lib/ext/jfxrt.jar
7f17682e9000-7f17684e9000 rw-p 00000000 00:00 0 
7f17684e9000-7f17684ec000 ---p 00000000 00:00 0 
7f17684ec000-7f17685ea000 rw-p 00000000 00:00 0 
7f17685ea000-7f17685f4000 r--s 0021b000 fd:00 16925688                   /usr/java/jdk1.8.0_341-amd64/jre/lib/ext/localedata.jar
7f17685f4000-7f1768617000 r--s 0038e000 fd:00 17505306                   /usr/java/jdk1.8.0_341-amd64/jre/lib/ext/cldrdata.jar
7f1768617000-7f1768618000 ---p 00000000 00:00 0 
7f1768618000-7f1768718000 rw-p 00000000 00:00 0 
7f1768718000-7f176871b000 ---p 00000000 00:00 0 
7f176871b000-7f1768819000 rw-p 00000000 00:00 0 
7f1768819000-7f176881d000 ---p 00000000 00:00 0 
7f176881d000-7f176891a000 rw-p 00000000 00:00 0 
7f176891a000-7f176891e000 ---p 00000000 00:00 0 
7f176891e000-7f1768a1b000 rw-p 00000000 00:00 0 
7f1768a1b000-7f1768a1e000 ---p 00000000 00:00 0 
7f1768a1e000-7f1768b1c000 rw-p 00000000 00:00 0 
7f1768b1c000-7f1768b1f000 ---p 00000000 00:00 0 
7f1768b1f000-7f1768c1d000 rw-p 00000000 00:00 0 
7f1768c1d000-7f1768c20000 ---p 00000000 00:00 0 
7f1768c20000-7f1768d1e000 rw-p 00000000 00:00 0 
7f1768d1e000-7f1768d1f000 ---p 00000000 00:00 0 
7f1768d1f000-7f1768e1f000 rw-p 00000000 00:00 0 
7f1768e1f000-7f1769000000 r--s 03e5d000 fd:00 17505278                   /usr/java/jdk1.8.0_341-amd64/jre/lib/rt.jar
7f1769000000-7f1769840000 rwxp 00000000 00:00 0 
7f1769840000-7f1778000000 ---p 00000000 00:00 0 
7f1778000000-7f1778bc5000 rw-p 00000000 00:00 0 
7f1778bc5000-7f177c000000 ---p 00000000 00:00 0 
7f177c001000-7f177c007000 r--s 002f8000 fd:00 17579673                   /usr/java/jdk1.8.0_341-amd64/jre/lib/charsets.jar
7f177c007000-7f177c928000 rw-p 00000000 00:00 0 
7f177c928000-7f177c949000 rw-p 00000000 00:00 0 
7f177c949000-7f177cce8000 ---p 00000000 00:00 0 
7f177cce8000-7f177cd03000 r-xp 00000000 fd:00 33837055                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libzip.so
7f177cd03000-7f177cf02000 ---p 0001b000 fd:00 33837055                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libzip.so
7f177cf02000-7f177cf03000 r--p 0001a000 fd:00 33837055                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libzip.so
7f177cf03000-7f177cf04000 rw-p 0001b000 fd:00 33837055                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libzip.so
7f177cf04000-7f177cf10000 r-xp 00000000 fd:00 219669                     /usr/lib64/libnss_files-2.17.so
7f177cf10000-7f177d10f000 ---p 0000c000 fd:00 219669                     /usr/lib64/libnss_files-2.17.so
7f177d10f000-7f177d110000 r--p 0000b000 fd:00 219669                     /usr/lib64/libnss_files-2.17.so
7f177d110000-7f177d111000 rw-p 0000c000 fd:00 219669                     /usr/lib64/libnss_files-2.17.so
7f177d111000-7f177d117000 rw-p 00000000 00:00 0 
7f177d117000-7f177d144000 r-xp 00000000 fd:00 34650805                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libjava.so
7f177d144000-7f177d343000 ---p 0002d000 fd:00 34650805                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libjava.so
7f177d343000-7f177d344000 r--p 0002c000 fd:00 34650805                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libjava.so
7f177d344000-7f177d346000 rw-p 0002d000 fd:00 34650805                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libjava.so
7f177d346000-7f177d353000 r-xp 00000000 fd:00 33836982                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libverify.so
7f177d353000-7f177d552000 ---p 0000d000 fd:00 33836982                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libverify.so
7f177d552000-7f177d554000 r--p 0000c000 fd:00 33836982                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libverify.so
7f177d554000-7f177d555000 rw-p 0000e000 fd:00 33836982                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libverify.so
7f177d555000-7f177d55c000 r-xp 00000000 fd:00 222081                     /usr/lib64/librt-2.17.so
7f177d55c000-7f177d75b000 ---p 00007000 fd:00 222081                     /usr/lib64/librt-2.17.so
7f177d75b000-7f177d75c000 r--p 00006000 fd:00 222081                     /usr/lib64/librt-2.17.so
7f177d75c000-7f177d75d000 rw-p 00007000 fd:00 222081                     /usr/lib64/librt-2.17.so
7f177d75d000-7f177d85e000 r-xp 00000000 fd:00 219659                     /usr/lib64/libm-2.17.so
7f177d85e000-7f177da5d000 ---p 00101000 fd:00 219659                     /usr/lib64/libm-2.17.so
7f177da5d000-7f177da5e000 r--p 00100000 fd:00 219659                     /usr/lib64/libm-2.17.so
7f177da5e000-7f177da5f000 rw-p 00101000 fd:00 219659                     /usr/lib64/libm-2.17.so
7f177da5f000-7f177e757000 r-xp 00000000 fd:00 1597372                    /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/server/libjvm.so
7f177e757000-7f177e956000 ---p 00cf8000 fd:00 1597372                    /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/server/libjvm.so
7f177e956000-7f177e9ec000 r--p 00cf7000 fd:00 1597372                    /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/server/libjvm.so
7f177e9ec000-7f177ea1d000 rw-p 00d8d000 fd:00 1597372                    /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/server/libjvm.so
7f177ea1d000-7f177ea58000 rw-p 00000000 00:00 0 
7f177ea58000-7f177ec1c000 r-xp 00000000 fd:00 219651                     /usr/lib64/libc-2.17.so
7f177ec1c000-7f177ee1b000 ---p 001c4000 fd:00 219651                     /usr/lib64/libc-2.17.so
7f177ee1b000-7f177ee1f000 r--p 001c3000 fd:00 219651                     /usr/lib64/libc-2.17.so
7f177ee1f000-7f177ee21000 rw-p 001c7000 fd:00 219651                     /usr/lib64/libc-2.17.so
7f177ee21000-7f177ee26000 rw-p 00000000 00:00 0 
7f177ee26000-7f177ee28000 r-xp 00000000 fd:00 219657                     /usr/lib64/libdl-2.17.so
7f177ee28000-7f177f028000 ---p 00002000 fd:00 219657                     /usr/lib64/libdl-2.17.so
7f177f028000-7f177f029000 r--p 00002000 fd:00 219657                     /usr/lib64/libdl-2.17.so
7f177f029000-7f177f02a000 rw-p 00003000 fd:00 219657                     /usr/lib64/libdl-2.17.so
7f177f02a000-7f177f041000 r-xp 00000000 fd:00 17815613                   /usr/java/jdk1.8.0_341-amd64/lib/amd64/jli/libjli.so
7f177f041000-7f177f240000 ---p 00017000 fd:00 17815613                   /usr/java/jdk1.8.0_341-amd64/lib/amd64/jli/libjli.so
7f177f240000-7f177f241000 r--p 00016000 fd:00 17815613                   /usr/java/jdk1.8.0_341-amd64/lib/amd64/jli/libjli.so
7f177f241000-7f177f242000 rw-p 00017000 fd:00 17815613                   /usr/java/jdk1.8.0_341-amd64/lib/amd64/jli/libjli.so
7f177f242000-7f177f259000 r-xp 00000000 fd:00 219677                     /usr/lib64/libpthread-2.17.so
7f177f259000-7f177f458000 ---p 00017000 fd:00 219677                     /usr/lib64/libpthread-2.17.so
7f177f458000-7f177f459000 r--p 00016000 fd:00 219677                     /usr/lib64/libpthread-2.17.so
7f177f459000-7f177f45a000 rw-p 00017000 fd:00 219677                     /usr/lib64/libpthread-2.17.so
7f177f45a000-7f177f45e000 rw-p 00000000 00:00 0 
7f177f45e000-7f177f480000 r-xp 00000000 fd:00 219644                     /usr/lib64/ld-2.17.so
7f177f481000-7f177f495000 r--p 00000000 fd:00 219642                     /usr/share/locale/zh_CN/LC_MESSAGES/libc.mo
7f177f495000-7f177f4e7000 rw-p 00000000 00:00 0 
7f177f4e7000-7f177f50f000 rw-p 00000000 00:00 0 
7f177f50f000-7f177f561000 rw-p 00000000 00:00 0 
7f177f561000-7f177f562000 rw-p 00000000 00:00 0 
7f177f562000-7f177f566000 ---p 00000000 00:00 0 
7f177f566000-7f177f668000 rw-p 00000000 00:00 0 
7f177f668000-7f177f669000 rw-p 00000000 00:00 0 
7f177f669000-7f177f670000 r--s 00000000 fd:00 50443394                   /usr/lib64/gconv/gconv-modules.cache
7f177f670000-7f177f674000 r--s 00f82000 fd:00 54846606                   /usr/mysoft/springbootTest01-1.0-SNAPSHOT.jar
7f177f674000-7f177f67c000 rw-s 00000000 fd:00 1863095                    /tmp/hsperfdata_root/37121
7f177f67c000-7f177f67d000 rw-p 00000000 00:00 0 
7f177f67d000-7f177f67e000 r--p 00000000 00:00 0 
7f177f67e000-7f177f67f000 rw-p 00000000 00:00 0 
7f177f67f000-7f177f680000 r--p 00021000 fd:00 219644                     /usr/lib64/ld-2.17.so
7f177f680000-7f177f681000 rw-p 00022000 fd:00 219644                     /usr/lib64/ld-2.17.so
7f177f681000-7f177f682000 rw-p 00000000 00:00 0 
7ffc98d91000-7ffc98db2000 rw-p 00000000 00:00 0                          [stack]
7ffc98dbb000-7ffc98dbd000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

VM Arguments:
jvm_args: -XX:+CrashOnOutOfMemoryError -XX:+CreateMinidumpOnCrash -XX:ErrorFile=/usr/myfolder/MY_JVM_ANALYZE/HS_ERR_PID/hs_err_pid123.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/myfolder/MY_JVM_ANALYZE/HEAP_DUMP/heapdump0626.hprof 
java_command: ./mysoft/springbootTest01-1.0-SNAPSHOT.jar
java_class_path (initial): ./mysoft/springbootTest01-1.0-SNAPSHOT.jar
Launcher Type: SUN_STANDARD

Environment Variables:
PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/root/bin
USERNAME=root
SHELL=/bin/bash
DISPLAY=:0

Signal Handlers:
SIGSEGV: [libjvm.so+0xae1180], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGBUS: [libjvm.so+0xae1180], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGFPE: [libjvm.so+0xae1180], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGPIPE: [libjvm.so+0x915340], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGXFSZ: [libjvm.so+0x915340], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGILL: [libjvm.so+0xae1180], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGUSR1: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGUSR2: [libjvm.so+0x915210], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
SIGHUP: [libjvm.so+0x915830], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGINT: [libjvm.so+0x915830], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGTERM: [libjvm.so+0x915830], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGQUIT: [libjvm.so+0x915830], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO


---------------  S Y S T E M  ---------------

OS:CentOS Linux release 7.9.2009 (Core)

uname:Linux 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64
libc:glibc 2.17 NPTL 2.17 
rlimit: STACK 8192k, CORE infinity, NPROC 4096, NOFILE 4096, AS infinity
load average:0.74 0.41 0.36

/proc/meminfo:
MemTotal:         995676 kB
MemFree:           61276 kB
MemAvailable:      37036 kB
Buffers:              36 kB
Cached:            95872 kB
SwapCached:        34524 kB
Active:           337668 kB
Inactive:         337100 kB
Active(anon):     315152 kB
Inactive(anon):   300456 kB
Active(file):      22516 kB
Inactive(file):    36644 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       2097148 kB
SwapFree:        1678856 kB
Dirty:               100 kB
Writeback:             0 kB
AnonPages:        576440 kB
Mapped:            69308 kB
Shmem:             36648 kB
Slab:              99000 kB
SReclaimable:      35136 kB
SUnreclaim:        63864 kB
KernelStack:       10256 kB
PageTables:        43052 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     2594984 kB
Committed_AS:    4605680 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      214628 kB
VmallocChunk:   34359277564 kB
Percpu:            55808 kB
HardwareCorrupted:     0 kB
AnonHugePages:     26624 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      112512 kB
DirectMap2M:      935936 kB
DirectMap1G:           0 kB

container (cgroup) information:
container_type: cgroupv1
cpu_cpuset_cpus: 0
cpu_memory_nodes: 0
active_processor_count: 1
cpu_quota: -1
cpu_period: 100000
cpu_shares: -1
memory_limit_in_bytes: -1
memory_and_swap_limit_in_bytes: -1
memory_soft_limit_in_bytes: -1
memory_usage_in_bytes: 689020928
memory_max_usage_in_bytes: 0


CPU:total 1 (initial active 1) (1 cores per cpu, 1 threads per core) family 6 model 158 stepping 9, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, 3dnowpref, lzcnt, tsc, tscinvbit, bmi1, bmi2, adx

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 158
model name	: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
stepping	: 9
microcode	: 0xb4
cpu MHz		: 2807.998
cache size	: 6144 KB
physical id	: 0
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 22
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 arat md_clear spec_ctrl intel_stibp flush_l1d arch_capabilities
bogomips	: 5615.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 45 bits physical, 48 bits virtual
power management:



Memory: 4k page, physical 995676k(61152k free), swap 2097148k(1678856k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (25.341-b10) for linux-amd64 JRE (1.8.0_341-b10), built on Jun 16 2022 15:46:36 by "java_re" with gcc 7.3.0

time: Tue Jul  2 19:42:13 2024
timezone: CST
elapsed time: 19.620335 seconds (0d 0h 0m 19s)

View Code

通过第23、24行可以发现是代码中的 start.controller.DumpTestController.hello() 方法的问题。

5.2、hs_err_pid 日志文件分析

5.2.1、报告头

报告头即前面一长串注释的内容，该内容一般会说明 jvm 崩溃的原因。同时也会标明 JDK 版本、JVM 类型等信息。

产生 core dump 文件，会说明 core dump 文件默认存放位置和文件名。如果提示 “Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again” 则是因为文件被限制了大小，可直接在 Linux 环境执行 ulimit -c unlimited 命令解除限制，下一次就能生成 core dump 文件了。

5.2.2、THREAD

这部分主要是线程相关的信息。

Current thread (0x00007f177845b800):  JavaThread "http-nio-8080-exec-1" daemon [_thread_in_vm, id=37153, stack(0x00007f1747cff000,0x00007f1747e00000)]

Current thread：发生致命错误的线程
JavaThread：线程类型为java线程，后面的是线程名
id=xxx：线程 id
stack(xxxx)：栈区间

后面的 Stack:xxxx 等一系列东西主要是线程的执行过程

5.2.3、PROCESS

这部分主要是进程相关信息

6、Nmon 文件

nmon是一个非常轻量级的用于监控系统性能的工具，因为几乎没有性能开销，所以它可以长期运行而不会对系统造成负担。它主要用于实时监测 Linux/UNIX 系统的各种资源使用情况，并且提供了一种利用命令行界面来可视化和分析系统性能的方式。

nmon 工具可以为AIX和Linux 性能专家提供监视和分析性能数据，其中包括：

CPU 使用率
内存使用情况
内核统计信息和运行队列信息
磁盘V/0速度、传输和读/写比率
文件系统中的可用空间
磁盘适配器
页面空间和页面速度
CPU和AIX规范
消耗资源最多的进程
IBM HTTP Web 缓存
用户自定义的磁盘组
计算机详细信息和资源
异步I/O，仅适用于AIX
工作负载管理器（WLM)，仅适用于AIX
IBM TotalStorage@ Enterprise Storage Server®(ESS)磁盘，仅适用于AIX
网络文件系统(NFS)
动态 LPAR(DLPAR)重改，仅适用于面向 AIX 成 Lipux 的 pSeries p5和 OpenPower

posted @ 2024-06-10 23:58 wenxuehai 阅读(281) 评论(0) 编辑收藏举报

刷新页面返回顶部

提升自己，才有选择的能力和勇气