JVM宕机分析

 

1、JVM崩溃常见原因

常见的能够引起JVM崩溃的原因主要有：

  • 内存溢出
  • 线程阻塞
  • CPU 使用率过高
  • JVM Crash
  • Java虚拟机自身的Bug
  • JDK与服务器（CPU、内存、操作系统）的兼容性

 

2、heapdump文件分析（内存溢出）

Java程序运行时，有时会产生JavaCore及HeapDump文件，它一般发生于Java程序遇到致命问题的情况下。

  • 有时致命问题发生后，Java应用不会死掉，还能继续运行；
  • 但有时致命问题发生，Java进程会死掉；

为了能够保留Java应用发生致命错误前的运行状态，JVM在死掉前产生两个文件，分别为JavaCore及HeapDump文件。javacore文件一般是文本文件，而heapdump则一般是二进制文件，需要使用专门的分析工具进行分析，一般常用的工具为IBM的heap anylizer。

 

2.1、heapdump 文件生成

  • 手动调命令生成

可以在内存升高时手动通过 jmap 命令手动生成当前时刻的 dump 文件，参考：https://www.cnblogs.com/wenxuehai/p/16600216.html#_label2_0_0

  • 配置自动生成

可以配置当程序发生内存溢出时自动生成 dump 文件，如下：

  1. -XX:+HeapDumpOnOutOfMemoryError # 表示当JVM发生OOM时，自动生成DUMP文件。
  2. -XX:HeapDumpPath=${目录} # 指定生成DUMP文件的路径，也可以指定文件名称，例如：-XX:HeapDumpPath=${目录}/java_heapdump.hprof。如果不指定文件名，默认会在项目根目录下生成一个文件，文件名格式为：java_<pid>_<date>_<time>_heapDump.hprof。

（注意，IBM JDK在内存溢出时会自动生成 heapdump 文件，无需在 jvm 配置参数，这点跟我们常用的 Oracle JDK 不一样）

示例如下，只指定了目录：

  1. -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=F:/myHeapdumpTest

参考：https://blog.csdn.net/AikesLs/article/details/124290879

（注意，IBM JDK 的 dump 文件以 .phd 为后缀，IBM 堆转储文件通常以以下方式命名：heapdump.<date>..<timestamp><pid>.phd ）

 

2.2、heapdump 文件分析

heapdump 文件可通过 MAT 分析工具分析，参考：https://www.cnblogs.com/wenxuehai/p/16600216.html#autoid-h2-5-0-2

 

3、Java Core文件

Javacore 又称 thread dump，是Java虚拟机（JVM）的一种转储文件格式，其中包含了当前系统中所有线程的状态和调用堆栈信息。这个文件通常用于诊断系统问题、分析性能瓶颈以及检查线程死锁等情况。JavaCore文件可以通过工具进行分析，以获取有关系统状态的详细信息。

Javacore 是线程堆栈快照文件，能够利用该文件对 JVM 当前时间节点正则运行的线程进行分析，一般间隔 10-20s 取一次，连续取三次。一般两个以上 Javacore 文件分析才比较有效果，可以通过两个时间节点的线程执行情况进行分析，如果两个 Javacore 文件都在执行某个方法的相关代码，说明该方法的执行耗时很长，需要进行优化。

常见需要利用 Javacore 分析的现象：

  1. CPU 使用率高、load 高、响应很慢
  2. 需查找占用CPU最多的线程信息
  3. CPU使用率不高但是响应时间很长或无法响应

 

3.1、Javacore 文件生成

可以使用 jdk 自带命令 jstack 生成 Javacore 文件（Linux 和 window 环境都可以），命令如下：

  1. jstack pid > pid_stack.log
  2. # 示例：
  3. jstack 22656 > pid_stack.log

或使用 kill -3 pid 命令也可生成 Javacore 文件（但我在虚拟机上执行时只在控制台打印了堆栈信息没有生成文件）

 

3.2、Javacore 文件分析

可以通过 IBM Javacore 分析工具 jca 分析，如 版本4618 下载路径：https://public.dhe.ibm.com/software/websphere/appserv/support/tools/jca/jca4618.jar，下载完后直接通过命令行 java -jar 4618.jar 即可启动。

通过工具打开生成的 Javacore 文件，可以看到下图：

线程状态解释：

  • Deadlock：死锁（重点关注）
  • Runnable：运行中或可运行（重点关注）
  • Waiting on condition：等待资源（重点关注）
  • Waiting on monitor：等待监控器检查资源
  • Suspended：暂停
  • Object.wait()：对象等待中
  • Blocked：阻塞（重点关注）
  • Parked：停止 

 

Javacore 文件分析思路：

程序发生内存溢出时会自动生成Javacore 文件，一般有几份，首先应该关注最早生成的文件。

  1. 先在顶部概况查看Javacore生成原因。主要有以下四种：
    1. 人工生成：Dump Event "user" (00004000) received
    2. 内存溢出：Dump Event "systhrow" (00040000) Detail "java/lang/OutOfMemoryError" "Java heap space" received
    3. 一般性保护错误：Dump Event "gpt" (00002000) received
    4. 无法创建线程：Dump Event "systhrow" (00040000) Detail "java/lang/OutOfMemoryError" "Failed to create a thread: retVal -1073741830, errno 11"
  2. 查看当前线程数是否正常（一般几百到一千）
  3. 查看jdk版本，有可能是JDK版本低引起的故障
  4. 查看jvm参数，看下jvm配置有没配置不合理的
  5. 查看用户资源限制参数（User Process Resource Limit Analysis）。主要可以查看 RLIMIT_NPROC 配置，该配置的 soft limit定义允许创建线程数。
  6. 查看内存使用情况（Memory Segment Analysis）
  7. 当前线程堆栈信息（Current Thread）。可以从 java stack分析出当前线程正在进行的操作

然后分析线程堆栈

  1. 点击线程详情按钮（齿轮状），然后将线程按深度降序排序
  2. 分别查看几个线程的执行情况（可以查看按深度排序后第一个线程，第一个等待状态的线程，和占用锁的线程）
  3. 从右边堆栈信息（Java Stack）中查找相关业务代码，分析当前线程正在进行什么相关操作

 

3.3、JavaCore 和 HeapDump 的区别

  • HeapDump文件是关于内存的。

HeapDump文件是一个二进制文件，它保存了某一时刻JVM堆中对象使用情况，这种文件需要相应的工具进行分析，如IBM   Heap Analyzer这类工具。这类文件最重要的作用就是分析系统中是否存在内存溢出的情况。

  • JavaCore是关于CPU的

JavaCore文件主要保存的是Java应用各线程在某一时刻的运行的位置，即JVM执行到哪一个类、哪一个方法、哪一个行上。它是一个文本文件，打开后可以看到每一个线程的执行栈，以stack trace的显示。通过对JavaCore文件的分析可以得到应用是否“卡”在某一点上，即在某一点运行的时间太长，例如数据库查询，长期得不到响应，最终导致系统崩溃等情况。 

 

 4、GC日志文件

 

4.1、生成 GC 日志

  • IBM JDK

-verbose:gc ，打开GC日志，如果应用运行在WAS下，可通过 WAS 下 Java 虚拟机页面勾选”详细垃圾回收“自动设置，无需手工设置。如果应用没有运行在 WAS 下但使用了 IBM JDK，则增加参数-verbose:gc 方式来打开 GC 日志。
-Xverbosegclog:${SERVER_LOG_ROOT}/verbosegc.%seq.log,10,21000  ，打开GC日志后，启用GC日志自动滚动，保留最近10个日志，每个日志文件约30M（注意，${SERVER_LOG_ROOT} 为WAS的内置变量，会自动替换为当前AppServer的日志目录，不需要修改）

IBM JDK的日志文件是 XML 格式，内容比较详细。

 

  • Oracle JDK

配置如下：

复制代码
  1. -Xloggc:/xxx/xxx/app_gc.log   #日志文件的输出路径，如window环境可配置为 -Xloggc:F:/MY_JVM_ANALYZE/GC_LOG/temp_gc.log
  2. -XX:+PrintGCDetails     #输出GC的详细日志
  3. -XX:+PrintGCDatestamps  #输出GC的时间戳（以日期的形式，如2020-05-04T21：53：59.234+0800）
  4. -XX:+UseGCLogFileRotation  #启用GC日志滚动
  5. -XX:NumberOfGCLogFiles=5    #GC日志文件保留数量，建议为5
  6. -XX:GCLogFilesize=20M     #GC日志文件大小，建议为20M
复制代码

配置以上参数后，项目启动时即会生成 gc 日志文件。发生垃圾回收时也会更新日志文件。Oracle JDK的日志文件是文本形式，内容比较简短。

 

 4.2、GC日志分析

如何通过 GC日志判断应用的性能状况：

GC评分项 计算方式 评分占用分数 参考值 说明
吞吐率

1-(GC消耗总时间/程序运行时间）*100%

 20 99.99% 吞吐率越高越好，即GC时间越短越好，表示GC对于程序运行的影响越低
FullGC后老年代内存或者堆内存使用率 (FuIlGC后老年代内存使用量或者整个堆内存使用量/老年代已分配内存或者整个队内存已分配内存）*100% 30 20% FullGC后内存使用率越低越好，表示GC能够回收越多内存

FuIlGC发生次数

统计FulIlGC在工作日繁忙时段（9:00-18：00）内发生的次数

 20 3 FullGC次数越少越好表示对象生命周期控制合理，老年代堆积程度较低

FullGC发生平均时间间隔

 統计FullGC在工作日繁忙时段（9:00-18:00）发生的平均时间间隔（秒） 20 10800（即3小时） 老年代GC间隔越长越好，表示对象生命周期较短，老年代堆积程度较低

新生代GC发生平均时间间隔

 统计新生代GC在工作日繁忙时段（9:00-18.00）发生的平均时间间隔（秒） 10 30 新生代GC间隔越长越好，表示堆内存配置较为合理

正常的 GC 日志曲线呈锯齿状，并且 FULL GCC后，内存使用空间会降低到一个比较平均的值。如果 FULL GC后，内存没有下降并且有上涨趋势，则可以认为不正常。

 

业界较为流行使用 GCViewer 工具分析GC日志，下载地址：https://github.com/chewiebug/GCViewer/tree/1.36

  • Full GC Lines： （full gc）
  • Inc GC Lines：（增量GC）
  • GC Times Line： （gc 时间）
  • GC Times Rectangles： （gc时间区域）
  • Total Heap：(总堆大小)
  • Red line that shows heap size
  • Tenured Generation：（老年代）
  • Young Generation：（年轻代）
  • Used Heap：（堆使用量）
  • Initial mark level：（cms或g1垃圾回收算法初始标记事件）
  • Concurrent collections：（cms或g1垃圾回收并发收集周期）

 

 示例如下图，堆使用的越来越高，Full GC 频繁触发，且FULL GC后内存并没有降下来。

 

5、hs_err_pid 日志文件分析

hs_err_pid.log 文件是一个 Java 虚拟机（JVM）生成的错误报告日志文件，通常包含了 JVM 崩溃时的详细信息。当 JVM 因为各种原因发生致命错误时（包括内存溢出（OutOfMemoryError）、线程死锁、系统资源不足等问题），会生成一个hs_err_pid_xxx.log（后面的xxx为进程id）这样的文件，该文件包含了导致 JVM crash（JVM崩溃）的重要信息，我们可以通过分析该文件定位到导致 JVM Crash 的原因，从而修复保证系统稳定。

（注意，IBM JDK 崩溃时并不会生成 hs_err_pid.log 文件，只有 Oracle JDK和open JDK 才有这个机制）

 

5.1、生成hs_err_pid 日志文件

默认情况下，该文件是生成在当前程序工作目录下的（即启动该程序的目录，而不是该程序的存放目录，比如在 /usr/ 目录通过 java -jar 命令运行 /usr/mysoft/springbootTest01.jar 包，当 jvm 崩溃时，hs_err_pid 文件会生成在 /usr/ 目录下而不是 /usr/mysoft/ 目录）。默认该文件文件名为 hs_err_pid<pid>.log，后面的 pid 为进程号。如下：

当然也可以通过 JVM 参数指定生成路径：

  1. -XX:ErrorFile=./hs_err_pid<pid>.log  # 注意 <pid>是进程号，需指定，直接写 <pid> 程序无法运行

示例：

 

5.1.1、查看生成路径

可以通过以下命令查看  hs_err_pid.log 文件的存放目录，如下：

  1. ls -l /proc/<pid>/cwd
  3. #示例：
  4. ls -l /proc/2658/cwd

如下图：

 

5.1.2、模拟生成

模拟内存溢出代码：

复制代码
  1. @RequestMapping("/DumpTest")
  2. @RestController
  3. public class DumpTestController {
  5.     @RequestMapping("test01")
  6.     public String hello() throws Exception {
  7.         String name = "Aikes";
  8.         for (int i = 0; i < 10000000; i++) {
  9.             name += name;
  10.         }
  11.         System.out.println(name);
  13.         return "test";
  14.     }
  15. }
复制代码

打包项目，然后通过 jar 包运行，如下：

  1. java -XX:+CrashOnOutOfMemoryError -jar ./springbootTest01-1.0-SNAPSHOT.jar
  • -XX:+CrashOnOutOfMemoryError：使JVM在内存溢出时立即崩溃。该配置参数可指定当JVM检测到OutOfMemoryError（内存溢出错误）时立即崩溃，这个参数可以用来强制Java应用在发生内存溢出时立即终止，避免某些情况下由于自我修复机制（如JVM的垃圾收集器尝试释放更多内存）而使得问题被掩盖或延后发现。

然后访问服务器的接口，如：http://192.168.118.130:8080/DumpTest/test01，程序将会发生内存溢出，随之生成 hs_err_pid 日志文件和 core 文件（如果没生成core文件可能需要执行 ulimit -c unlimited 命令，core文件即 coredump 文件，该文件在 jvm 崩溃时也会自动生成）。如下：

 

以下为模拟 jvm 崩溃时生成的日志文件内容实例：

复制代码
  1. #
  2. # A fatal error has been detected by the Java Runtime Environment:
  3. #
  4. #  Internal Error (debug.cpp:312), pid=37121, tid=0x00007f1747dff700
  5. #  OutOfMemory encountered: : Java heap space
  6. #
  7. # JRE version: Java(TM) SE Runtime Environment (8.0_341-b10) (build 1.8.0_341-b10)
  8. # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.341-b10 mixed mode linux-amd64 compressed oops)
  9. # Core dump written. Default location: /usr/core or core.37121
  10. #
  11.  
  12. ---------------  T H R E A D  ---------------
  13.  
  14. Current thread (0x00007f177845b800):  JavaThread "http-nio-8080-exec-1" daemon [_thread_in_vm, id=37153, stack(0x00007f1747cff000,0x00007f1747e00000)]
  15.  
  16. Stack: [0x00007f1747cff000,0x00007f1747e00000],  sp=0x00007f1747dfc3d0,  free space=1012k
  17. Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
  18. V  [libjvm.so+0xae0865]  VMError::report_and_die()+0x2e5
  19. V  [libjvm.so+0x4e2ddd]  report_java_out_of_memory(char const*)+0x13d
  20. V  [libjvm.so+0xaa0151]  TypeArrayKlass::allocate_common(int, bool, Thread*)+0x161
  21. V  [libjvm.so+0x9c23b6]  OptoRuntime::new_array_nozero_C(Klass*, int, JavaThread*)+0x26
  22. v  ~RuntimeStub::_new_array_nozero_Java
  23. J 837 C2 java.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder; (8 bytes) @ 0x00007f1769324c08 [0x00007f1769324980+0x288]
  24. j  start.controller.DumpTestController.hello()Ljava/lang/String;+23
  25. v  ~StubRoutines::call_stub
  26. V  [libjvm.so+0x68bc3a]  JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0xe1a
  27. V  [libjvm.so+0x9a3e28]  Reflection::invoke(instanceKlassHandle, methodHandle, Handle, bool, objArrayHandle, BasicType, objArrayHandle, bool, Thread*)+0xae8
  28. V  [libjvm.so+0x9a57f6]  Reflection::invoke_method(oopDesc*, Handle, objArrayHandle, Thread*)+0x126
  29. V  [libjvm.so+0x703095]  JVM_InvokeMethod+0x115
  30. J 2057  sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (0 bytes) @ 0x00007f176941d8ad [0x00007f176941d7c0+0xed]
  31. J 2056 C1 sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (104 bytes) @ 0x00007f17693eac2b [0x00007f17693ea8a0+0x38b]
  32. J 948 C1 sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (10 bytes) @ 0x00007f1769351c03 [0x00007f1769351b00+0x103]
  33. J 947 C1 java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (62 bytes) @ 0x00007f1769351636 [0x00007f1769351280+0x3b6]
  34. j  org.springframework.web.method.support.InvocableHandlerMethod.doInvoke([Ljava/lang/Object;)Ljava/lang/Object;+16
  35. j  org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(Lorg/springframework/web/context/request/NativeWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)Ljava/lang/Object;+75
  36. j  org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(Lorg/springframework/web/context/request/ServletWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)V+4
  37. j  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView;+276
  38. j  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView;+81
  39. j  org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljava/lang/Object;)Lorg/springframework/web/servlet/ModelAndView;+7
  40. j  org.springframework.web.servlet.DispatcherServlet.doDispatch(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+310
  41. j  org.springframework.web.servlet.DispatcherServlet.doService(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+308
  42. j  org.springframework.web.servlet.FrameworkServlet.processRequest(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+71
  43. j  org.springframework.web.servlet.FrameworkServlet.doGet(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+3
  44. j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+35
  45. j  org.springframework.web.servlet.FrameworkServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+33
  46. j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+30
  47. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+304
  48. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  49. j  org.apache.tomcat.websocket.server.WsFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+21
  50. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
  51. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  52. j  org.springframework.web.filter.RequestContextFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+21
  53. j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
  54. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
  55. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  56. j  org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+95
  57. j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
  58. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
  59. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  60. j  org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+87
  61. j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
  62. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
  63. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  64. j  org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+53
  65. j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
  66. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
  67. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  68. j  org.apache.catalina.core.StandardWrapperValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+688
  69. j  org.apache.catalina.core.StandardContextValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+166
  70. j  org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+260
  71. j  org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+138
  72. j  org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6
  73. j  org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+71
  74. j  org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V+199
  75. j  org.apache.coyote.http11.Http11Processor.service(Lorg/apache/tomcat/util/net/SocketWrapperBase;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+806
  76. j  org.apache.coyote.AbstractProcessorLight.process(Lorg/apache/tomcat/util/net/SocketWrapperBase;Lorg/apache/tomcat/util/net/SocketEvent;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+113
  77. j  org.apache.coyote.AbstractProtocol$ConnectionHandler.process(Lorg/apache/tomcat/util/net/SocketWrapperBase;Lorg/apache/tomcat/util/net/SocketEvent;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+414
  78. j  org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun()V+191
  79. j  org.apache.tomcat.util.net.SocketProcessorBase.run()V+21
  80. j  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
  81. j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
  82. j  org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run()V+4
  83. j  java.lang.Thread.run()V+11
  84. v  ~StubRoutines::call_stub
  85. V  [libjvm.so+0x68bc3a]  JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0xe1a
  86. V  [libjvm.so+0x6892f3]  JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x263
  87. V  [libjvm.so+0x6898c7]  JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*, Symbol*, Thread*)+0x57
  88. V  [libjvm.so+0x6f72bc]  thread_entry(JavaThread*, Thread*)+0x6c
  89. V  [libjvm.so+0xa88f8b]  JavaThread::thread_main_inner()+0xdb
  90. V  [libjvm.so+0xa89291]  JavaThread::run()+0x2d1
  91. V  [libjvm.so+0x9173e2]  java_start(Thread*)+0x102
  92. C  [libpthread.so.0+0x7ea5]  start_thread+0xc5
  93.  
  94. Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
  95. v  ~RuntimeStub::_new_array_nozero_Java
  96. J 837 C2 java.lang.StringBuilder.append(Ljava/lang/String;)Ljava/lang/StringBuilder; (8 bytes) @ 0x00007f1769324c08 [0x00007f1769324980+0x288]
  97. j  start.controller.DumpTestController.hello()Ljava/lang/String;+23
  98. v  ~StubRoutines::call_stub
  99. J 2057  sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (0 bytes) @ 0x00007f176941d837 [0x00007f176941d7c0+0x77]
  100. J 2056 C1 sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (104 bytes) @ 0x00007f17693eac2b [0x00007f17693ea8a0+0x38b]
  101. J 948 C1 sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (10 bytes) @ 0x00007f1769351c03 [0x00007f1769351b00+0x103]
  102. J 947 C1 java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (62 bytes) @ 0x00007f1769351636 [0x00007f1769351280+0x3b6]
  103. j  org.springframework.web.method.support.InvocableHandlerMethod.doInvoke([Ljava/lang/Object;)Ljava/lang/Object;+16
  104. j  org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(Lorg/springframework/web/context/request/NativeWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)Ljava/lang/Object;+75
  105. j  org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(Lorg/springframework/web/context/request/ServletWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)V+4
  106. j  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView;+276
  107. j  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView;+81
  108. j  org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljava/lang/Object;)Lorg/springframework/web/servlet/ModelAndView;+7
  109. j  org.springframework.web.servlet.DispatcherServlet.doDispatch(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+310
  110. j  org.springframework.web.servlet.DispatcherServlet.doService(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+308
  111. j  org.springframework.web.servlet.FrameworkServlet.processRequest(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+71
  112. j  org.springframework.web.servlet.FrameworkServlet.doGet(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+3
  113. j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+35
  114. j  org.springframework.web.servlet.FrameworkServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+33
  115. j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+30
  116. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+304
  117. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  118. j  org.apache.tomcat.websocket.server.WsFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+21
  119. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
  120. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  121. j  org.springframework.web.filter.RequestContextFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+21
  122. j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
  123. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
  124. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  125. j  org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+95
  126. j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
  127. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
  128. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  129. j  org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+87
  130. j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
  131. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
  132. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  133. j  org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+53
  134. j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
  135. j  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+135
  136. j  org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+101
  137. j  org.apache.catalina.core.StandardWrapperValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+688
  138. j  org.apache.catalina.core.StandardContextValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+166
  139. j  org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+260
  140. j  org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+138
  141. j  org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6
  142. j  org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+71
  143. j  org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V+199
  144. j  org.apache.coyote.http11.Http11Processor.service(Lorg/apache/tomcat/util/net/SocketWrapperBase;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+806
  145. j  org.apache.coyote.AbstractProcessorLight.process(Lorg/apache/tomcat/util/net/SocketWrapperBase;Lorg/apache/tomcat/util/net/SocketEvent;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+113
  146. j  org.apache.coyote.AbstractProtocol$ConnectionHandler.process(Lorg/apache/tomcat/util/net/SocketWrapperBase;Lorg/apache/tomcat/util/net/SocketEvent;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+414
  147. j  org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun()V+191
  148. j  org.apache.tomcat.util.net.SocketProcessorBase.run()V+21
  149. j  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
  150. j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
  151. j  org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run()V+4
  152. j  java.lang.Thread.run()V+11
  153. v  ~StubRoutines::call_stub
  154.  
  155. ---------------  P R O C E S S  ---------------
  156.  
  157. Java Threads: ( => current thread )
  158.   0x00007f1778009800 JavaThread "DestroyJavaVM" [_thread_blocked, id=37122, stack(0x00007f177f563000,0x00007f177f663000)]
  159.   0x00007f1778878800 JavaThread "http-nio-8080-AsyncTimeout" daemon [_thread_blocked, id=37165, stack(0x00007f17470f3000,0x00007f17471f4000)]
  160.   0x00007f1778810800 JavaThread "http-nio-8080-Acceptor-0" daemon [_thread_in_native, id=37164, stack(0x00007f17471f4000,0x00007f17472f5000)]
  161.   0x00007f177880f000 JavaThread "http-nio-8080-ClientPoller-0" daemon [_thread_in_native, id=37163, stack(0x00007f17472f5000,0x00007f17473f6000)]
  162.   0x00007f17785e8000 JavaThread "http-nio-8080-exec-10" daemon [_thread_blocked, id=37162, stack(0x00007f17473f6000,0x00007f17474f7000)]
  163.   0x00007f17788d2800 JavaThread "http-nio-8080-exec-9" daemon [_thread_blocked, id=37161, stack(0x00007f17474f7000,0x00007f17475f8000)]
  164.   0x00007f17781ee800 JavaThread "http-nio-8080-exec-8" daemon [_thread_blocked, id=37160, stack(0x00007f17475f8000,0x00007f17476f9000)]
  165.   0x00007f17788c1000 JavaThread "http-nio-8080-exec-7" daemon [_thread_blocked, id=37159, stack(0x00007f17476f9000,0x00007f17477fa000)]
  166.   0x00007f17788de000 JavaThread "http-nio-8080-exec-6" daemon [_thread_blocked, id=37158, stack(0x00007f17477fa000,0x00007f17478fb000)]
  167.   0x00007f17786e3000 JavaThread "http-nio-8080-exec-5" daemon [_thread_blocked, id=37157, stack(0x00007f17478fb000,0x00007f17479fc000)]
  168.   0x00007f1778377000 JavaThread "http-nio-8080-exec-4" daemon [_thread_blocked, id=37156, stack(0x00007f17479fc000,0x00007f1747afd000)]
  169.   0x00007f1778316000 JavaThread "http-nio-8080-exec-3" daemon [_thread_blocked, id=37155, stack(0x00007f1747afd000,0x00007f1747bfe000)]
  170.   0x00007f17784f5800 JavaThread "http-nio-8080-exec-2" daemon [_thread_blocked, id=37154, stack(0x00007f1747bfe000,0x00007f1747cff000)]
  171. =>0x00007f177845b800 JavaThread "http-nio-8080-exec-1" daemon [_thread_in_vm, id=37153, stack(0x00007f1747cff000,0x00007f1747e00000)]
  172.   0x00007f1778364000 JavaThread "NioBlockingSelector.BlockPoller-1" daemon [_thread_in_native, id=37152, stack(0x00007f1754035000,0x00007f1754136000)]
  173.   0x00007f17788a4000 JavaThread "container-0" [_thread_blocked, id=37149, stack(0x00007f1754136000,0x00007f1754237000)]
  174.   0x00007f1778899000 JavaThread "ContainerBackgroundProcessor[StandardEngine[Tomcat]]" daemon [_thread_blocked, id=37148, stack(0x00007f1754237000,0x00007f1754338000)]
  175.   0x00007f17780df800 JavaThread "Service Thread" daemon [_thread_blocked, id=37129, stack(0x00007f1768718000,0x00007f1768819000)]
  176.   0x00007f17780da800 JavaThread "C1 CompilerThread1" daemon [_thread_blocked, id=37128, stack(0x00007f176881a000,0x00007f176891a000)]
  177.   0x00007f17780d8800 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=37127, stack(0x00007f176891b000,0x00007f1768a1b000)]
  178.   0x00007f17780d7000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=37126, stack(0x00007f1768a1b000,0x00007f1768b1c000)]
  179.   0x00007f17780a4000 JavaThread "Finalizer" daemon [_thread_blocked, id=37125, stack(0x00007f1768b1c000,0x00007f1768c1d000)]
  180.   0x00007f177809f000 JavaThread "Reference Handler" daemon [_thread_blocked, id=37124, stack(0x00007f1768c1d000,0x00007f1768d1e000)]
  181.  
  182. Other Threads:
  183.   0x00007f1778095800 VMThread [stack: 0x00007f1768d1f000,0x00007f1768e1f000] [id=37123]
  184.   0x00007f17780e2800 WatcherThread [stack: 0x00007f1768618000,0x00007f1768718000] [id=37130]
  185.  
  186. VM state:not at safepoint (normal execution)
  187.  
  188. VM Mutex/Monitor currently owned by a thread: None
  189.  
  190. heap address: 0x00000000f0c00000, size: 244 MB, Compressed Oops mode: 32-bit
  191. Narrow klass base: 0x0000000000000000, Narrow klass shift: 3
  192. Compressed class space size: 1073741824 Address: 0x0000000100000000
  193.  
  194. Heap:
  195.  def new generation   total 74944K, used 216K [0x00000000f0c00000, 0x00000000f5d50000, 0x00000000f5d50000)
  196.   eden space 66624K,   0% used [0x00000000f0c00000, 0x00000000f0c36210, 0x00000000f4d10000)
  197.   from space 8320K,   0% used [0x00000000f4d10000, 0x00000000f4d10000, 0x00000000f5530000)
  198.   to   space 8320K,   0% used [0x00000000f5530000, 0x00000000f5530000, 0x00000000f5d50000)
  199.  tenured generation   total 166592K, used 95847K [0x00000000f5d50000, 0x0000000100000000, 0x0000000100000000)
  200.    the space 166592K,  57% used [0x00000000f5d50000, 0x00000000fbae9fb8, 0x00000000fbaea000, 0x0000000100000000)
  201.  Metaspace       used 34695K, capacity 36218K, committed 36608K, reserved 1081344K
  202.   class space    used 4483K, capacity 4785K, committed 4864K, reserved 1048576K
  203.  
  204. Card table byte_map: [0x00007f177f4e7000,0x00007f177f562000] byte_map_base: 0x00007f177ed61000
  205.  
  206. Polling page: 0x00007f177f67d000
  207.  
  208. CodeCache: size=245760Kb used=8230Kb max_used=8288Kb free=237529Kb
  209.  bounds [0x00007f1769000000, 0x00007f1769840000, 0x00007f1778000000]
  210.  total_blobs=3257 nmethods=2837 adapters=332
  211.  compilation: enabled
  212.  
  213. Compilation events (10 events):
  214. Event: 19.376 Thread 0x00007f17780d8800 3251       4       java.net.URLClassLoader$1::run (5 bytes)
  215. Event: 19.376 Thread 0x00007f17780d8800 nmethod 3251 0x00007f1769608050 code [0x00007f17696081a0, 0x00007f1769608228]
  216. Event: 19.376 Thread 0x00007f17780d8800 3254       4       java.util.AbstractMap::<init> (5 bytes)
  217. Event: 19.376 Thread 0x00007f17780d8800 nmethod 3254 0x00007f17692b5fd0 code [0x00007f17692b6120, 0x00007f17692b6178]
  218. Event: 19.376 Thread 0x00007f17780d8800 3269       4       java.lang.ref.Finalizer::access$100 (6 bytes)
  219. Event: 19.383 Thread 0x00007f17780d8800 nmethod 3269 0x00007f1769178510 code [0x00007f17691786c0, 0x00007f1769178fa8]
  220. Event: 19.383 Thread 0x00007f17780d8800 3270   !   4       java.lang.ref.Finalizer::runFinalizer (62 bytes)
  221. Event: 19.414 Thread 0x00007f17780d8800 nmethod 3270 0x00007f176923e890 code [0x00007f176923ea40, 0x00007f176923f328]
  222. Event: 19.414 Thread 0x00007f17780d8800 3271   !   4       java.lang.ref.Finalizer::remove (94 bytes)
  223. Event: 19.416 Thread 0x00007f17780d8800 nmethod 3271 0x00007f1769533290 code [0x00007f17695333e0, 0x00007f17695336d8]
  224.  
  225. GC Heap History (10 events):
  226. Event: 19.153 GC heap before
  227. {Heap before GC invocations=123 (full 6):
  228.  def new generation   total 35200K, used 20480K [0x00000000f0c00000, 0x00000000f3230000, 0x00000000f5d50000)
  229.   eden space 31296K,  65% used [0x00000000f0c00000, 0x00000000f2000010, 0x00000000f2a90000)
  230.   from space 3904K,   0% used [0x00000000f2e60000, 0x00000000f2e60048, 0x00000000f3230000)
  231.   to   space 3904K,   0% used [0x00000000f2a90000, 0x00000000f2a90000, 0x00000000f2e60000)
  232.  tenured generation   total 78124K, used 67352K [0x00000000f5d50000, 0x00000000fa99b000, 0x0000000100000000)
  233.    the space 78124K,  86% used [0x00000000f5d50000, 0x00000000f9f161a0, 0x00000000f9f16200, 0x00000000fa99b000)
  234.  Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  235.   class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
  236. Event: 19.258 GC heap after
  237. Heap after GC invocations=124 (full 7):
  238.  def new generation   total 42880K, used 0K [0x00000000f0c00000, 0x00000000f3a80000, 0x00000000f5d50000)
  239.   eden space 38144K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f3140000)
  240.   from space 4736K,   0% used [0x00000000f3140000, 0x00000000f3140000, 0x00000000f35e0000)
  241.   to   space 4736K,   0% used [0x00000000f35e0000, 0x00000000f35e0000, 0x00000000f3a80000)
  242.  tenured generation   total 95188K, used 57112K [0x00000000f5d50000, 0x00000000fba45000, 0x0000000100000000)
  243.    the space 95188K,  59% used [0x00000000f5d50000, 0x00000000f9516188, 0x00000000f9516200, 0x00000000fba45000)
  244.  Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  245.   class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
  246. }
  247. Event: 19.302 GC heap before
  248. {Heap before GC invocations=124 (full 7):
  249.  def new generation   total 42880K, used 736K [0x00000000f0c00000, 0x00000000f3a80000, 0x00000000f5d50000)
  250.   eden space 38144K,   1% used [0x00000000f0c00000, 0x00000000f0cb8028, 0x00000000f3140000)
  251.   from space 4736K,   0% used [0x00000000f3140000, 0x00000000f3140000, 0x00000000f35e0000)
  252.   to   space 4736K,   0% used [0x00000000f35e0000, 0x00000000f35e0000, 0x00000000f3a80000)
  253.  tenured generation   total 136152K, used 98072K [0x00000000f5d50000, 0x00000000fe246000, 0x0000000100000000)
  254.    the space 136152K,  72% used [0x00000000f5d50000, 0x00000000fbd161a0, 0x00000000fbd16200, 0x00000000fe246000)
  255.  Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  256.   class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
  257. Event: 19.351 GC heap after
  258. Heap after GC invocations=125 (full 8):
  259.  def new generation   total 61312K, used 0K [0x00000000f0c00000, 0x00000000f4e80000, 0x00000000f5d50000)
  260.   eden space 54528K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f4140000)
  261.   from space 6784K,   0% used [0x00000000f4140000, 0x00000000f4140000, 0x00000000f47e0000)
  262.   to   space 6784K,   0% used [0x00000000f47e0000, 0x00000000f47e0000, 0x00000000f4e80000)
  263.  tenured generation   total 136152K, used 77573K [0x00000000f5d50000, 0x00000000fe246000, 0x0000000100000000)
  264.    the space 136152K,  56% used [0x00000000f5d50000, 0x00000000fa911500, 0x00000000fa911600, 0x00000000fe246000)
  265.  Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  266.   class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
  267. }
  268. Event: 19.387 GC heap before
  269. {Heap before GC invocations=125 (full 8):
  270.  def new generation   total 61312K, used 42012K [0x00000000f0c00000, 0x00000000f4e80000, 0x00000000f5d50000)
  271.   eden space 54528K,  77% used [0x00000000f0c00000, 0x00000000f35070f8, 0x00000000f4140000)
  272.   from space 6784K,   0% used [0x00000000f4140000, 0x00000000f4140000, 0x00000000f47e0000)
  273.   to   space 6784K,   0% used [0x00000000f47e0000, 0x00000000f47e0000, 0x00000000f4e80000)
  274.  tenured generation   total 136152K, used 77573K [0x00000000f5d50000, 0x00000000fe246000, 0x0000000100000000)
  275.    the space 136152K,  56% used [0x00000000f5d50000, 0x00000000fa911500, 0x00000000fa911600, 0x00000000fe246000)
  276.  Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  277.   class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
  278. Event: 19.411 GC heap after
  279. Heap after GC invocations=126 (full 8):
  280.  def new generation   total 61312K, used 0K [0x00000000f0c00000, 0x00000000f4e80000, 0x00000000f5d50000)
  281.   eden space 54528K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f4140000)
  282.   from space 6784K,   0% used [0x00000000f47e0000, 0x00000000f47e0048, 0x00000000f4e80000)
  283.   to   space 6784K,   0% used [0x00000000f4140000, 0x00000000f4140000, 0x00000000f47e0000)
  284.  tenured generation   total 136152K, used 118533K [0x00000000f5d50000, 0x00000000fe246000, 0x0000000100000000)
  285.    the space 136152K,  87% used [0x00000000f5d50000, 0x00000000fd111510, 0x00000000fd111600, 0x00000000fe246000)
  286.  Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  287.   class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
  288. }
  289. Event: 19.426 GC heap before
  290. {Heap before GC invocations=126 (full 8):
  291.  def new generation   total 61312K, used 40960K [0x00000000f0c00000, 0x00000000f4e80000, 0x00000000f5d50000)
  292.   eden space 54528K,  75% used [0x00000000f0c00000, 0x00000000f3400010, 0x00000000f4140000)
  293.   from space 6784K,   0% used [0x00000000f47e0000, 0x00000000f47e0048, 0x00000000f4e80000)
  294.   to   space 6784K,   0% used [0x00000000f4140000, 0x00000000f4140000, 0x00000000f47e0000)
  295.  tenured generation   total 136152K, used 118533K [0x00000000f5d50000, 0x00000000fe246000, 0x0000000100000000)
  296.    the space 136152K,  87% used [0x00000000f5d50000, 0x00000000fd111510, 0x00000000fd111600, 0x00000000fe246000)
  297.  Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  298.   class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
  299. Event: 19.513 GC heap after
  300. Heap after GC invocations=127 (full 9):
  301.  def new generation   total 73600K, used 0K [0x00000000f0c00000, 0x00000000f5bd0000, 0x00000000f5d50000)
  302.   eden space 65472K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f4bf0000)
  303.   from space 8128K,   0% used [0x00000000f4bf0000, 0x00000000f4bf0000, 0x00000000f53e0000)
  304.   to   space 8128K,   0% used [0x00000000f53e0000, 0x00000000f53e0000, 0x00000000f5bd0000)
  305.  tenured generation   total 163424K, used 98053K [0x00000000f5d50000, 0x00000000ffce8000, 0x0000000100000000)
  306.    the space 163424K,  59% used [0x00000000f5d50000, 0x00000000fbd114f8, 0x00000000fbd11600, 0x00000000ffce8000)
  307.  Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  308.   class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
  309. }
  310. Event: 19.513 GC heap before
  311. {Heap before GC invocations=127 (full 9):
  312.  def new generation   total 73600K, used 0K [0x00000000f0c00000, 0x00000000f5bd0000, 0x00000000f5d50000)
  313.   eden space 65472K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f4bf0000)
  314.   from space 8128K,   0% used [0x00000000f4bf0000, 0x00000000f4bf0000, 0x00000000f53e0000)
  315.   to   space 8128K,   0% used [0x00000000f53e0000, 0x00000000f53e0000, 0x00000000f5bd0000)
  316.  tenured generation   total 166592K, used 98053K [0x00000000f5d50000, 0x0000000100000000, 0x0000000100000000)
  317.    the space 166592K,  58% used [0x00000000f5d50000, 0x00000000fbd114f8, 0x00000000fbd11600, 0x0000000100000000)
  318.  Metaspace       used 34794K, capacity 36376K, committed 36608K, reserved 1081344K
  319.   class space    used 4500K, capacity 4816K, committed 4864K, reserved 1048576K
  320. Event: 19.577 GC heap after
  321. Heap after GC invocations=128 (full 10):
  322.  def new generation   total 74944K, used 0K [0x00000000f0c00000, 0x00000000f5d50000, 0x00000000f5d50000)
  323.   eden space 66624K,   0% used [0x00000000f0c00000, 0x00000000f0c00000, 0x00000000f4d10000)
  324.   from space 8320K,   0% used [0x00000000f4d10000, 0x00000000f4d10000, 0x00000000f5530000)
  325.   to   space 8320K,   0% used [0x00000000f5530000, 0x00000000f5530000, 0x00000000f5d50000)
  326.  tenured generation   total 166592K, used 95847K [0x00000000f5d50000, 0x0000000100000000, 0x0000000100000000)
  327.    the space 166592K,  57% used [0x00000000f5d50000, 0x00000000fbae9fb8, 0x00000000fbaea000, 0x0000000100000000)
  328.  Metaspace       used 34695K, capacity 36218K, committed 36608K, reserved 1081344K
  329.   class space    used 4483K, capacity 4785K, committed 4864K, reserved 1048576K
  330. }
  331.  
  332. Deoptimization events (10 events):
  333. Event: 2.693 Thread 0x00007f1778009800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f176942be4c method=sun.util.calendar.CalendarDate.setMillis(I)Lsun/util/calendar/CalendarDate; @ 5
  334. Event: 4.904 Thread 0x00007f1778009800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f176963f620 method=java.lang.reflect.Method.equals(Ljava/lang/Object;)Z @ 1
  335. Event: 5.143 Thread 0x00007f1778009800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f176960483c method=java.util.Properties$LineReader.readLine()I @ 323
  336. Event: 5.352 Thread 0x00007f1778009800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f17695503b4 method=java.util.Properties.loadConvert([CII[C)Ljava/lang/String; @ 57
  337. Event: 5.491 Thread 0x00007f1778009800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f1769497d60 method=sun.misc.ASCIICaseInsensitiveComparator.compare(Ljava/lang/String;Ljava/lang/String;)I @ 83
  338. Event: 5.966 Thread 0x00007f17483e6000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00007f17696f9ec4 method=java.net.URL.getURLStreamHandler(Ljava/lang/String;)Ljava/net/URLStreamHandler; @ 214
  339. Event: 6.363 Thread 0x00007f1778009800 Uncommon trap: reason=bimorphic action=maybe_recompile pc=0x00007f1769552060 method=java.util.Hashtable.get(Ljava/lang/Object;)Ljava/lang/Object; @ 6
  340. Event: 6.363 Thread 0x00007f1778009800 Uncommon trap: reason=bimorphic action=maybe_recompile pc=0x00007f1769552060 method=java.util.Hashtable.get(Ljava/lang/Object;)Ljava/lang/Object; @ 6
  341. Event: 7.889 Thread 0x00007f1778009800 Uncommon trap: reason=bimorphic action=maybe_recompile pc=0x00007f1769552060 method=java.util.Hashtable.get(Ljava/lang/Object;)Ljava/lang/Object; @ 6
  342. Event: 7.906 Thread 0x00007f1778009800 Uncommon trap: reason=bimorphic action=maybe_recompile pc=0x00007f1769552060 method=java.util.Hashtable.get(Ljava/lang/Object;)Ljava/lang/Object; @ 6
  343.  
  344. Classes redefined (0 events):
  345. No events
  346.  
  347. Internal exceptions (10 events):
  348. Event: 18.546 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/theme/AbstractThemeResolverCustomizer> (0x00000000f0c52d40) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDicti
  349. Event: 18.546 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/theme/FixedThemeResolverCustomizer> (0x00000000f0c8a9e8) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDictiona
  350. Event: 18.546 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/ThemeResolverCustomizer> (0x00000000f0cbf458) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDictionary.cpp, lin
  351. Event: 18.548 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': java/lang/ObjectCustomizer> (0x00000000f0cfc658) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 212]
  352. Event: 18.549 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/view/DefaultRequestToViewNameTranslatorCustomizer> (0x00000000f0d39118) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile
  353. Event: 18.549 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/RequestToViewNameTranslatorCustomizer> (0x00000000f0d800a8) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDicti
  354. Event: 18.570 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': java/lang/ObjectCustomizer> (0x00000000f0df5870) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 212]
  355. Event: 18.571 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/support/AbstractFlashMapManagerCustomizer> (0x00000000f0e2fa60) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemD
  356. Event: 18.571 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/support/SessionFlashMapManagerCustomizer> (0x00000000f0e6d8e8) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDi
  357. Event: 18.571 Thread 0x00007f177845b800 Exception <a 'java/lang/ClassNotFoundException': org/springframework/web/servlet/FlashMapManagerCustomizer> (0x00000000f0ea21f8) thrown at [/scratch/jenkins/workspace/8-2-build-linux-x64/jdk8u341/2692/hotspot/src/share/vm/classfile/systemDictionary.cpp, l
  358.  
  359. Events (10 events):
  360. Event: 19.153 Executing VM operation: GenCollectForAllocation
  361. Event: 19.258 Executing VM operation: GenCollectForAllocation done
  362. Event: 19.266 Thread 0x00007f17780d8800 flushing nmethod 0x00007f1769708f50
  363. Event: 19.286 Thread 0x00007f17780d8800 flushing nmethod 0x00007f1769742650
  364. Event: 19.302 Executing VM operation: GenCollectForAllocation
  365. Event: 19.351 Executing VM operation: GenCollectForAllocation done
  366. Event: 19.386 Executing VM operation: GenCollectForAllocation
  367. Event: 19.411 Executing VM operation: GenCollectForAllocation done
  368. Event: 19.426 Executing VM operation: GenCollectForAllocation
  369. Event: 19.577 Executing VM operation: GenCollectForAllocation done
  370.  
  371.  
  372. Dynamic libraries:
  373. f0c00000-f5d50000 rw-p 00000000 00:00 0 
  374. f5d50000-100000000 rw-p 00000000 00:00 0 
  375. 100000000-1004c0000 rw-p 00000000 00:00 0 
  376. 1004c0000-140000000 ---p 00000000 00:00 0 
  377. 55711a3d5000-55711a3d6000 r-xp 00000000 fd:00 18321644                   /usr/java/jdk1.8.0_341-amd64/bin/java
  378. 55711a5d5000-55711a5d6000 r--p 00000000 fd:00 18321644                   /usr/java/jdk1.8.0_341-amd64/bin/java
  379. 55711a5d6000-55711a5d7000 rw-p 00001000 fd:00 18321644                   /usr/java/jdk1.8.0_341-amd64/bin/java
  380. 55711b577000-55711b598000 rw-p 00000000 00:00 0                          [heap]
  381. 7f1740000000-7f174040f000 rw-p 00000000 00:00 0 
  382. 7f174040f000-7f1744000000 ---p 00000000 00:00 0 
  383. 7f1746ef3000-7f1746ff3000 rw-p 00000000 00:00 0 
  384. 7f1746ff3000-7f17470f3000 ---p 00000000 00:00 0 
  385. 7f17470f3000-7f17470f6000 ---p 00000000 00:00 0 
  386. 7f17470f6000-7f17471f4000 rw-p 00000000 00:00 0 
  387. 7f17471f4000-7f17471f7000 ---p 00000000 00:00 0 
  388. 7f17471f7000-7f17472f5000 rw-p 00000000 00:00 0 
  389. 7f17472f5000-7f17472f8000 ---p 00000000 00:00 0 
  390. 7f17472f8000-7f17473f6000 rw-p 00000000 00:00 0 
  391. 7f17473f6000-7f17473f9000 ---p 00000000 00:00 0 
  392. 7f17473f9000-7f17474f7000 rw-p 00000000 00:00 0 
  393. 7f17474f7000-7f17474fa000 ---p 00000000 00:00 0 
  394. 7f17474fa000-7f17475f8000 rw-p 00000000 00:00 0 
  395. 7f17475f8000-7f17475fb000 ---p 00000000 00:00 0 
  396. 7f17475fb000-7f17476f9000 rw-p 00000000 00:00 0 
  397. 7f17476f9000-7f17476fc000 ---p 00000000 00:00 0 
  398. 7f17476fc000-7f17477fa000 rw-p 00000000 00:00 0 
  399. 7f17477fa000-7f17477fd000 ---p 00000000 00:00 0 
  400. 7f17477fd000-7f17478fb000 rw-p 00000000 00:00 0 
  401. 7f17478fb000-7f17478fe000 ---p 00000000 00:00 0 
  402. 7f17478fe000-7f17479fc000 rw-p 00000000 00:00 0 
  403. 7f17479fc000-7f17479ff000 ---p 00000000 00:00 0 
  404. 7f17479ff000-7f1747afd000 rw-p 00000000 00:00 0 
  405. 7f1747afd000-7f1747b00000 ---p 00000000 00:00 0 
  406. 7f1747b00000-7f1747bfe000 rw-p 00000000 00:00 0 
  407. 7f1747bfe000-7f1747c01000 ---p 00000000 00:00 0 
  408. 7f1747c01000-7f1747cff000 rw-p 00000000 00:00 0 
  409. 7f1747cff000-7f1747d02000 ---p 00000000 00:00 0 
  410. 7f1747d02000-7f1748000000 rw-p 00000000 00:00 0 
  411. 7f1748000000-7f1749c27000 rw-p 00000000 00:00 0 
  412. 7f1749c27000-7f174c000000 ---p 00000000 00:00 0 
  413. 7f174c000000-7f174cd44000 rw-p 00000000 00:00 0 
  414. 7f174cd44000-7f1750000000 ---p 00000000 00:00 0 
  415. 7f1750000000-7f1750170000 rw-p 00000000 00:00 0 
  416. 7f1750170000-7f1754000000 ---p 00000000 00:00 0 
  417. 7f1754035000-7f1754038000 ---p 00000000 00:00 0 
  418. 7f1754038000-7f1754136000 rw-p 00000000 00:00 0 
  419. 7f1754136000-7f1754139000 ---p 00000000 00:00 0 
  420. 7f1754139000-7f1754237000 rw-p 00000000 00:00 0 
  421. 7f1754237000-7f175423a000 ---p 00000000 00:00 0 
  422. 7f175423a000-7f1754538000 rw-p 00000000 00:00 0 
  423. 7f1754538000-7f1754738000 rw-p 00000000 00:00 0 
  424. 7f175478a000-7f175498a000 rw-p 00000000 00:00 0 
  425. 7f175498a000-7f175499b000 r-xp 00000000 fd:00 34650825                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnio.so
  426. 7f175499b000-7f1754b9a000 ---p 00011000 fd:00 34650825                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnio.so
  427. 7f1754b9a000-7f1754b9b000 r--p 00010000 fd:00 34650825                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnio.so
  428. 7f1754b9b000-7f1754b9c000 rw-p 00011000 fd:00 34650825                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnio.so
  429. 7f1754b9c000-7f1754d9c000 rw-p 00000000 00:00 0 
  430. 7f1754d9c000-7f1754d9f000 ---p 00000000 00:00 0 
  431. 7f1754d9f000-7f175509d000 rw-p 00000000 00:00 0 
  432. 7f175509d000-7f175529d000 rw-p 00000000 00:00 0 
  433. 7f175529d000-7f175549d000 rw-p 00000000 00:00 0 
  434. 7f175549d000-7f175569d000 rw-p 00000000 00:00 0 
  435. 7f175569d000-7f17556b3000 r-xp 00000000 fd:00 34650824                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnet.so
  436. 7f17556b3000-7f17558b2000 ---p 00016000 fd:00 34650824                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnet.so
  437. 7f17558b2000-7f17558b3000 r--p 00015000 fd:00 34650824                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnet.so
  438. 7f17558b3000-7f17558b4000 rw-p 00016000 fd:00 34650824                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libnet.so
  439. 7f17558b4000-7f17558bd000 r-xp 00000000 fd:00 34650822                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libmanagement.so
  440. 7f17558bd000-7f1755abc000 ---p 00009000 fd:00 34650822                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libmanagement.so
  441. 7f1755abc000-7f1755abd000 r--p 00008000 fd:00 34650822                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libmanagement.so
  442. 7f1755abd000-7f1755abe000 rw-p 00009000 fd:00 34650822                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libmanagement.so
  443. 7f1755abe000-7f175c000000 r--p 00000000 fd:00 34119214                   /usr/lib/locale/locale-archive
  444. 7f175c000000-7f175c021000 rw-p 00000000 00:00 0 
  445. 7f175c021000-7f1760000000 ---p 00000000 00:00 0 
  446. 7f1760000000-7f1760021000 rw-p 00000000 00:00 0 
  447. 7f1760021000-7f1764000000 ---p 00000000 00:00 0 
  448. 7f1764000000-7f1764294000 rw-p 00000000 00:00 0 
  449. 7f1764294000-7f1768000000 ---p 00000000 00:00 0 
  450. 7f1768042000-7f1768053000 r--s 001c7000 fd:00 17815594                   /usr/java/jdk1.8.0_341-amd64/jre/lib/jsse.jar
  451. 7f1768053000-7f1768253000 rw-p 00000000 00:00 0 
  452. 7f1768253000-7f17682e9000 r--s 007bf000 fd:00 17505309                   /usr/java/jdk1.8.0_341-amd64/jre/lib/ext/jfxrt.jar
  453. 7f17682e9000-7f17684e9000 rw-p 00000000 00:00 0 
  454. 7f17684e9000-7f17684ec000 ---p 00000000 00:00 0 
  455. 7f17684ec000-7f17685ea000 rw-p 00000000 00:00 0 
  456. 7f17685ea000-7f17685f4000 r--s 0021b000 fd:00 16925688                   /usr/java/jdk1.8.0_341-amd64/jre/lib/ext/localedata.jar
  457. 7f17685f4000-7f1768617000 r--s 0038e000 fd:00 17505306                   /usr/java/jdk1.8.0_341-amd64/jre/lib/ext/cldrdata.jar
  458. 7f1768617000-7f1768618000 ---p 00000000 00:00 0 
  459. 7f1768618000-7f1768718000 rw-p 00000000 00:00 0 
  460. 7f1768718000-7f176871b000 ---p 00000000 00:00 0 
  461. 7f176871b000-7f1768819000 rw-p 00000000 00:00 0 
  462. 7f1768819000-7f176881d000 ---p 00000000 00:00 0 
  463. 7f176881d000-7f176891a000 rw-p 00000000 00:00 0 
  464. 7f176891a000-7f176891e000 ---p 00000000 00:00 0 
  465. 7f176891e000-7f1768a1b000 rw-p 00000000 00:00 0 
  466. 7f1768a1b000-7f1768a1e000 ---p 00000000 00:00 0 
  467. 7f1768a1e000-7f1768b1c000 rw-p 00000000 00:00 0 
  468. 7f1768b1c000-7f1768b1f000 ---p 00000000 00:00 0 
  469. 7f1768b1f000-7f1768c1d000 rw-p 00000000 00:00 0 
  470. 7f1768c1d000-7f1768c20000 ---p 00000000 00:00 0 
  471. 7f1768c20000-7f1768d1e000 rw-p 00000000 00:00 0 
  472. 7f1768d1e000-7f1768d1f000 ---p 00000000 00:00 0 
  473. 7f1768d1f000-7f1768e1f000 rw-p 00000000 00:00 0 
  474. 7f1768e1f000-7f1769000000 r--s 03e5d000 fd:00 17505278                   /usr/java/jdk1.8.0_341-amd64/jre/lib/rt.jar
  475. 7f1769000000-7f1769840000 rwxp 00000000 00:00 0 
  476. 7f1769840000-7f1778000000 ---p 00000000 00:00 0 
  477. 7f1778000000-7f1778bc5000 rw-p 00000000 00:00 0 
  478. 7f1778bc5000-7f177c000000 ---p 00000000 00:00 0 
  479. 7f177c001000-7f177c007000 r--s 002f8000 fd:00 17579673                   /usr/java/jdk1.8.0_341-amd64/jre/lib/charsets.jar
  480. 7f177c007000-7f177c928000 rw-p 00000000 00:00 0 
  481. 7f177c928000-7f177c949000 rw-p 00000000 00:00 0 
  482. 7f177c949000-7f177cce8000 ---p 00000000 00:00 0 
  483. 7f177cce8000-7f177cd03000 r-xp 00000000 fd:00 33837055                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libzip.so
  484. 7f177cd03000-7f177cf02000 ---p 0001b000 fd:00 33837055                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libzip.so
  485. 7f177cf02000-7f177cf03000 r--p 0001a000 fd:00 33837055                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libzip.so
  486. 7f177cf03000-7f177cf04000 rw-p 0001b000 fd:00 33837055                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libzip.so
  487. 7f177cf04000-7f177cf10000 r-xp 00000000 fd:00 219669                     /usr/lib64/libnss_files-2.17.so
  488. 7f177cf10000-7f177d10f000 ---p 0000c000 fd:00 219669                     /usr/lib64/libnss_files-2.17.so
  489. 7f177d10f000-7f177d110000 r--p 0000b000 fd:00 219669                     /usr/lib64/libnss_files-2.17.so
  490. 7f177d110000-7f177d111000 rw-p 0000c000 fd:00 219669                     /usr/lib64/libnss_files-2.17.so
  491. 7f177d111000-7f177d117000 rw-p 00000000 00:00 0 
  492. 7f177d117000-7f177d144000 r-xp 00000000 fd:00 34650805                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libjava.so
  493. 7f177d144000-7f177d343000 ---p 0002d000 fd:00 34650805                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libjava.so
  494. 7f177d343000-7f177d344000 r--p 0002c000 fd:00 34650805                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libjava.so
  495. 7f177d344000-7f177d346000 rw-p 0002d000 fd:00 34650805                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libjava.so
  496. 7f177d346000-7f177d353000 r-xp 00000000 fd:00 33836982                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libverify.so
  497. 7f177d353000-7f177d552000 ---p 0000d000 fd:00 33836982                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libverify.so
  498. 7f177d552000-7f177d554000 r--p 0000c000 fd:00 33836982                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libverify.so
  499. 7f177d554000-7f177d555000 rw-p 0000e000 fd:00 33836982                   /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/libverify.so
  500. 7f177d555000-7f177d55c000 r-xp 00000000 fd:00 222081                     /usr/lib64/librt-2.17.so
  501. 7f177d55c000-7f177d75b000 ---p 00007000 fd:00 222081                     /usr/lib64/librt-2.17.so
  502. 7f177d75b000-7f177d75c000 r--p 00006000 fd:00 222081                     /usr/lib64/librt-2.17.so
  503. 7f177d75c000-7f177d75d000 rw-p 00007000 fd:00 222081                     /usr/lib64/librt-2.17.so
  504. 7f177d75d000-7f177d85e000 r-xp 00000000 fd:00 219659                     /usr/lib64/libm-2.17.so
  505. 7f177d85e000-7f177da5d000 ---p 00101000 fd:00 219659                     /usr/lib64/libm-2.17.so
  506. 7f177da5d000-7f177da5e000 r--p 00100000 fd:00 219659                     /usr/lib64/libm-2.17.so
  507. 7f177da5e000-7f177da5f000 rw-p 00101000 fd:00 219659                     /usr/lib64/libm-2.17.so
  508. 7f177da5f000-7f177e757000 r-xp 00000000 fd:00 1597372                    /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/server/libjvm.so
  509. 7f177e757000-7f177e956000 ---p 00cf8000 fd:00 1597372                    /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/server/libjvm.so
  510. 7f177e956000-7f177e9ec000 r--p 00cf7000 fd:00 1597372                    /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/server/libjvm.so
  511. 7f177e9ec000-7f177ea1d000 rw-p 00d8d000 fd:00 1597372                    /usr/java/jdk1.8.0_341-amd64/jre/lib/amd64/server/libjvm.so
  512. 7f177ea1d000-7f177ea58000 rw-p 00000000 00:00 0 
  513. 7f177ea58000-7f177ec1c000 r-xp 00000000 fd:00 219651                     /usr/lib64/libc-2.17.so
  514. 7f177ec1c000-7f177ee1b000 ---p 001c4000 fd:00 219651                     /usr/lib64/libc-2.17.so
  515. 7f177ee1b000-7f177ee1f000 r--p 001c3000 fd:00 219651                     /usr/lib64/libc-2.17.so
  516. 7f177ee1f000-7f177ee21000 rw-p 001c7000 fd:00 219651                     /usr/lib64/libc-2.17.so
  517. 7f177ee21000-7f177ee26000 rw-p 00000000 00:00 0 
  518. 7f177ee26000-7f177ee28000 r-xp 00000000 fd:00 219657                     /usr/lib64/libdl-2.17.so
  519. 7f177ee28000-7f177f028000 ---p 00002000 fd:00 219657                     /usr/lib64/libdl-2.17.so
  520. 7f177f028000-7f177f029000 r--p 00002000 fd:00 219657                     /usr/lib64/libdl-2.17.so
  521. 7f177f029000-7f177f02a000 rw-p 00003000 fd:00 219657                     /usr/lib64/libdl-2.17.so
  522. 7f177f02a000-7f177f041000 r-xp 00000000 fd:00 17815613                   /usr/java/jdk1.8.0_341-amd64/lib/amd64/jli/libjli.so
  523. 7f177f041000-7f177f240000 ---p 00017000 fd:00 17815613                   /usr/java/jdk1.8.0_341-amd64/lib/amd64/jli/libjli.so
  524. 7f177f240000-7f177f241000 r--p 00016000 fd:00 17815613                   /usr/java/jdk1.8.0_341-amd64/lib/amd64/jli/libjli.so
  525. 7f177f241000-7f177f242000 rw-p 00017000 fd:00 17815613                   /usr/java/jdk1.8.0_341-amd64/lib/amd64/jli/libjli.so
  526. 7f177f242000-7f177f259000 r-xp 00000000 fd:00 219677                     /usr/lib64/libpthread-2.17.so
  527. 7f177f259000-7f177f458000 ---p 00017000 fd:00 219677                     /usr/lib64/libpthread-2.17.so
  528. 7f177f458000-7f177f459000 r--p 00016000 fd:00 219677                     /usr/lib64/libpthread-2.17.so
  529. 7f177f459000-7f177f45a000 rw-p 00017000 fd:00 219677                     /usr/lib64/libpthread-2.17.so
  530. 7f177f45a000-7f177f45e000 rw-p 00000000 00:00 0 
  531. 7f177f45e000-7f177f480000 r-xp 00000000 fd:00 219644                     /usr/lib64/ld-2.17.so
  532. 7f177f481000-7f177f495000 r--p 00000000 fd:00 219642                     /usr/share/locale/zh_CN/LC_MESSAGES/libc.mo
  533. 7f177f495000-7f177f4e7000 rw-p 00000000 00:00 0 
  534. 7f177f4e7000-7f177f50f000 rw-p 00000000 00:00 0 
  535. 7f177f50f000-7f177f561000 rw-p 00000000 00:00 0 
  536. 7f177f561000-7f177f562000 rw-p 00000000 00:00 0 
  537. 7f177f562000-7f177f566000 ---p 00000000 00:00 0 
  538. 7f177f566000-7f177f668000 rw-p 00000000 00:00 0 
  539. 7f177f668000-7f177f669000 rw-p 00000000 00:00 0 
  540. 7f177f669000-7f177f670000 r--s 00000000 fd:00 50443394                   /usr/lib64/gconv/gconv-modules.cache
  541. 7f177f670000-7f177f674000 r--s 00f82000 fd:00 54846606                   /usr/mysoft/springbootTest01-1.0-SNAPSHOT.jar
  542. 7f177f674000-7f177f67c000 rw-s 00000000 fd:00 1863095                    /tmp/hsperfdata_root/37121
  543. 7f177f67c000-7f177f67d000 rw-p 00000000 00:00 0 
  544. 7f177f67d000-7f177f67e000 r--p 00000000 00:00 0 
  545. 7f177f67e000-7f177f67f000 rw-p 00000000 00:00 0 
  546. 7f177f67f000-7f177f680000 r--p 00021000 fd:00 219644                     /usr/lib64/ld-2.17.so
  547. 7f177f680000-7f177f681000 rw-p 00022000 fd:00 219644                     /usr/lib64/ld-2.17.so
  548. 7f177f681000-7f177f682000 rw-p 00000000 00:00 0 
  549. 7ffc98d91000-7ffc98db2000 rw-p 00000000 00:00 0                          [stack]
  550. 7ffc98dbb000-7ffc98dbd000 r-xp 00000000 00:00 0                          [vdso]
  551. ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
  552.  
  553. VM Arguments:
  554. jvm_args: -XX:+CrashOnOutOfMemoryError -XX:+CreateMinidumpOnCrash -XX:ErrorFile=/usr/myfolder/MY_JVM_ANALYZE/HS_ERR_PID/hs_err_pid123.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/myfolder/MY_JVM_ANALYZE/HEAP_DUMP/heapdump0626.hprof 
  555. java_command: ./mysoft/springbootTest01-1.0-SNAPSHOT.jar
  556. java_class_path (initial): ./mysoft/springbootTest01-1.0-SNAPSHOT.jar
  557. Launcher Type: SUN_STANDARD
  558.  
  559. Environment Variables:
  560. PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/root/bin
  561. USERNAME=root
  562. SHELL=/bin/bash
  563. DISPLAY=:0
  564.  
  565. Signal Handlers:
  566. SIGSEGV: [libjvm.so+0xae1180], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
  567. SIGBUS: [libjvm.so+0xae1180], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
  568. SIGFPE: [libjvm.so+0xae1180], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
  569. SIGPIPE: [libjvm.so+0x915340], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
  570. SIGXFSZ: [libjvm.so+0x915340], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
  571. SIGILL: [libjvm.so+0xae1180], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
  572. SIGUSR1: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
  573. SIGUSR2: [libjvm.so+0x915210], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
  574. SIGHUP: [libjvm.so+0x915830], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
  575. SIGINT: [libjvm.so+0x915830], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
  576. SIGTERM: [libjvm.so+0x915830], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
  577. SIGQUIT: [libjvm.so+0x915830], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
  578.  
  579.  
  580. ---------------  S Y S T E M  ---------------
  581.  
  582. OS:CentOS Linux release 7.9.2009 (Core)
  583.  
  584. uname:Linux 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64
  585. libc:glibc 2.17 NPTL 2.17 
  586. rlimit: STACK 8192k, CORE infinity, NPROC 4096, NOFILE 4096, AS infinity
  587. load average:0.74 0.41 0.36
  588.  
  589. /proc/meminfo:
  590. MemTotal:         995676 kB
  591. MemFree:           61276 kB
  592. MemAvailable:      37036 kB
  593. Buffers:              36 kB
  594. Cached:            95872 kB
  595. SwapCached:        34524 kB
  596. Active:           337668 kB
  597. Inactive:         337100 kB
  598. Active(anon):     315152 kB
  599. Inactive(anon):   300456 kB
  600. Active(file):      22516 kB
  601. Inactive(file):    36644 kB
  602. Unevictable:           0 kB
  603. Mlocked:               0 kB
  604. SwapTotal:       2097148 kB
  605. SwapFree:        1678856 kB
  606. Dirty:               100 kB
  607. Writeback:             0 kB
  608. AnonPages:        576440 kB
  609. Mapped:            69308 kB
  610. Shmem:             36648 kB
  611. Slab:              99000 kB
  612. SReclaimable:      35136 kB
  613. SUnreclaim:        63864 kB
  614. KernelStack:       10256 kB
  615. PageTables:        43052 kB
  616. NFS_Unstable:          0 kB
  617. Bounce:                0 kB
  618. WritebackTmp:          0 kB
  619. CommitLimit:     2594984 kB
  620. Committed_AS:    4605680 kB
  621. VmallocTotal:   34359738367 kB
  622. VmallocUsed:      214628 kB
  623. VmallocChunk:   34359277564 kB
  624. Percpu:            55808 kB
  625. HardwareCorrupted:     0 kB
  626. AnonHugePages:     26624 kB
  627. CmaTotal:              0 kB
  628. CmaFree:               0 kB
  629. HugePages_Total:       0
  630. HugePages_Free:        0
  631. HugePages_Rsvd:        0
  632. HugePages_Surp:        0
  633. Hugepagesize:       2048 kB
  634. DirectMap4k:      112512 kB
  635. DirectMap2M:      935936 kB
  636. DirectMap1G:           0 kB
  637.  
  638. container (cgroup) information:
  639. container_type: cgroupv1
  640. cpu_cpuset_cpus: 0
  641. cpu_memory_nodes: 0
  642. active_processor_count: 1
  643. cpu_quota: -1
  644. cpu_period: 100000
  645. cpu_shares: -1
  646. memory_limit_in_bytes: -1
  647. memory_and_swap_limit_in_bytes: -1
  648. memory_soft_limit_in_bytes: -1
  649. memory_usage_in_bytes: 689020928
  650. memory_max_usage_in_bytes: 0
  651.  
  652.  
  653. CPU:total 1 (initial active 1) (1 cores per cpu, 1 threads per core) family 6 model 158 stepping 9, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, 3dnowpref, lzcnt, tsc, tscinvbit, bmi1, bmi2, adx
  654.  
  655. /proc/cpuinfo:
  656. processor	: 0
  657. vendor_id	: GenuineIntel
  658. cpu family	: 6
  659. model		: 158
  660. model name	: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  661. stepping	: 9
  662. microcode	: 0xb4
  663. cpu MHz		: 2807.998
  664. cache size	: 6144 KB
  665. physical id	: 0
  666. siblings	: 1
  667. core id		: 0
  668. cpu cores	: 1
  669. apicid		: 0
  670. initial apicid	: 0
  671. fpu		: yes
  672. fpu_exception	: yes
  673. cpuid level	: 22
  674. wp		: yes
  675. flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 arat md_clear spec_ctrl intel_stibp flush_l1d arch_capabilities
  676. bogomips	: 5615.99
  677. clflush size	: 64
  678. cache_alignment	: 64
  679. address sizes	: 45 bits physical, 48 bits virtual
  680. power management:
  681.  
  682.  
  683.  
  684. Memory: 4k page, physical 995676k(61152k free), swap 2097148k(1678856k free)
  685.  
  686. vm_info: Java HotSpot(TM) 64-Bit Server VM (25.341-b10) for linux-amd64 JRE (1.8.0_341-b10), built on Jun 16 2022 15:46:36 by "java_re" with gcc 7.3.0
  687.  
  688. time: Tue Jul  2 19:42:13 2024
  689. timezone: CST
  690. elapsed time: 19.620335 seconds (0d 0h 0m 19s)
View Code
复制代码

通过第23、24行可以发现是代码中的 start.controller.DumpTestController.hello() 方法的问题。

 

5.2、hs_err_pid 日志文件分析

5.2.1、报告头

报告头即前面一长串注释的内容，该内容一般会说明 jvm 崩溃的原因。同时也会标明 JDK 版本、JVM 类型等信息。

产生 core dump 文件，会说明 core dump 文件默认存放位置和文件名。如果提示 “Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again” 则是因为文件被限制了大小，可直接在 Linux 环境执行 ulimit -c unlimited 命令解除限制，下一次就能生成 core dump 文件了。

 

5.2.2、THREAD

这部分主要是线程相关的信息。

  1. Current thread (0x00007f177845b800):  JavaThread "http-nio-8080-exec-1" daemon [_thread_in_vm, id=37153, stack(0x00007f1747cff000,0x00007f1747e00000)]
  • Current thread：发生致命错误的线程
  • JavaThread：线程类型为java线程，后面的是线程名
  • id=xxx：线程 id
  • stack(xxxx)：栈区间

后面的 Stack:xxxx 等一系列东西主要是线程的执行过程

 

5.2.3、PROCESS

这部分主要是进程相关信息

 

6、Nmon 文件

 nmon是一个非常轻量级的用于监控系统性能的工具，因为几乎没有性能开销，所以它可以长期运行而不会对系统造成负担。它主要用于实时监测 Linux/UNIX 系统的各种资源使用情况，并且提供了一种利用命令行界面来可视化和分析系统性能的方式。

nmon 工具可以为AIX和Linux 性能专家提供监视和分析性能数据，其中包括：

  • CPU 使用率
  • 内存使用情况
  • 内核统计信息和运行队列信息
  • 磁盘V/0速度、传输和读/写比率
  • 文件系统中的可用空间
  • 磁盘适配器
  • 页面空间和页面速度
  • CPU和AIX规范
  • 消耗资源最多的进程
  • IBM HTTP Web 缓存
  • 用户自定义的磁盘组
  • 计算机详细信息和资源
  • 异步I/O，仅适用于AIX
  • 工作负载管理器（WLM)，仅适用于AIX
  • IBM TotalStorage@ Enterprise Storage Server®(ESS)磁盘，仅适用于AIX
  • 网络文件系统(NFS)
  • 动态 LPAR(DLPAR)重改，仅适用于面向 AIX 成 Lipux 的 pSeries p5和 OpenPower

 

posted @   wenxuehai  阅读(450)  评论(0)  编辑  收藏  举报
相关博文：
· JVM理解
· Java内存分析相关工具
· java进程内存占用分析
· 线上JAVA应用平稳运行一段时间后出现JVM崩溃问题
· 怎么分析一个java的core dump文件？
阅读排行：
· DeepSeek 开源周回顾「GitHub 热点速览」
· 物流快递公司核心技术能力-地址解析分单基础技术分享
· .NET 10首个预览版发布：重大改进与新特性概览！
· AI与.NET技术实操系列（二）：开始使用ML.NET
· 单线程的Redis速度为什么快？
//右下角添加目录
点击右上角即可分享
微信分享提示
AI FOR CODE 大赛