Arthas(阿尔萨斯)使用
Arthas 是一款线上监控诊断产品，通过全局视角实时查看应用 load、内存、gc、线程的状态信息，并能在不修改应用代码的情况下，对业务问题进行诊断，包括查看方法调用的出入参、异常，监测方法执行耗时，类加载信息等，大大提升线上问题排查效率
一、启动安装
##1.下载
wget https://arthas.aliyun.com/arthas-boot.jar;
##2.运行
java -jar arthas-boot.jar;
例如： Windows上使用参考
1.执行java -jar arthas-boot.jar
D:\Soft\arthas-bin>java -jar arthas-boot.jar
[INFO] arthas-boot version: 3.5.4
[INFO] Process 10668 already using port 3658
[INFO] Process 10668 already using port 8563
[INFO] Found existing java process, please choose one and input the serial number of the process, eg : 1. Then hit ENTER.
* [1]: 10668 org.jboss.Main
[2]: 5952
[3]: 2240
2.10668为需要监控的进程，是一个启动的Jboss服务，输入10668 前方的序号1回车，开始诊断监控，如下
[INFO] Process 10668 already using port 8563
[INFO] Found existing java process, please choose one and input the serial number of the process, eg : 1. Then hit ENTER.
* [1]: 10668 org.jboss.Main
[2]: 5952
[3]: 2240
1
[INFO] arthas home: D:\Soft\arthas-bin
[INFO] The target process already listen port 3658, skip attach.
[INFO] arthas-client connect 127.0.0.1 3658
,---. ,------. ,--------.,--. ,--. ,---. ,---.
/ O \ | .--. ''--. .--'| '--' | / O \ ' .-'
| .-. || '--'.' | | | .--. || .-. |`. `-.
| | | || |\ \ | | | | | || | | |.-' |
`--' `--'`--' '--' `--' `--' `--'`--' `--'`-----'
wiki https://arthas.aliyun.com/doc
tutorials https://arthas.aliyun.com/doc/arthas-tutorials.html
version 3.5.4
main_class
pid 10668
time 2022-12-23 16:33:09
二、能使用Arthas(阿尔萨斯）做什么
1. 全局查看系统的运行状况
dashboard 命令，查看当前系统实时数据面板
参数说明
-i : 指刷新的时间间隔
-n: 指刷新次数
例如：
[arthas@10668]$ dashboard
ID NAME GROUP PRIORITY STATE %CPU DELTA_TIME TIME INTERRUPTED DAE
129 Thread-24 jboss 5 TIMED_WAITING 0.0 0.000 4:23.345 false fal
14 ScannerThread jboss 5 TIMED_WAITING 0.0 0.000 4:22.596 false tru
157 http-0.0.0.0-8091-1 jboss 5 WAITING 0.0 0.000 0:34.304 false tru
-1 C2 CompilerThread0 - -1 - 0.0 0.000 0:27.019 false tru
-1 C2 CompilerThread1 - -1 - 0.0 0.000 0:26.894 false tru
-1 VM Thread - -1 - 0.0 0.000 0:19.734 false tru
-1 GC task thread#2 (ParallelGC) - -1 - 0.0 0.000 0:10.420 false tru
-1 GC task thread#4 (ParallelGC) - -1 - 0.0 0.000 0:10.405 false tru
-1 GC task thread#7 (ParallelGC) - -1 - 0.0 0.000 0:10.233 false tru
-1 GC task thread#3 (ParallelGC) - -1 - 0.0 0.000 0:10.186 false tru
-1 GC task thread#6 (ParallelGC) - -1 - 0.0 0.000 0:9.968 false tru
-1 GC task thread#0 (ParallelGC) - -1 - 0.0 0.000 0:9.906 false tru
-1 GC task thread#5 (ParallelGC) - -1 - 0.0 0.000 0:9.765 false tru
-1 GC task thread#1 (ParallelGC) - -1 - 0.0 0.000 0:9.672 false tru
52 JCA PoolFiller jboss 5 WAITING 0.0 0.000 0:7.113 false fal
154 arthas-command-execute system 5 RUNNABLE 0.0 0.000 0:5.210 false tru
25 Thread-5 jboss 5 TIMED_WAITING 0.0 0.000 0:4.882 false fal
153 arthas-NettyHttpTelnetBootstrap-3-2 system 5 RUNNABLE 0.0 0.000 0:1.762 false tru
88 IdleRemover jboss 5 TIMED_WAITING 0.0 0.000 0:0.374 false tru
-1 VM Periodic Task Thread - -1 - 0.0 0.000 0:0.296 false tru
49 DefaultQuartzScheduler_QuartzSchedulerThread QuartzScheduler:DefaultQuart 5 TIMED_WAITING 0.0 0.000 0:0.280 false fal
24 Thread-4 jboss 5 TIMED_WAITING 0.0 0.000 0:0.234 false tru
133 AWT-Windows system 6 RUNNABLE 0.0 0.000 0:0.093 false tru
3 Finalizer system 8 WAITING 0.0 0.000 0:0.062 false tru
10 DestroyJavaVM main 5 RUNNABLE 0.0 0.000 0:0.062 false fal
2 Reference Handler system 10 WAITING 0.0 0.000 0:0.046 false tru
145 logback-1 system 5 WAITING 0.0 0.000 0:0.046 false tru
35 ContainerBackgroundProcessor[StandardEngine[jboss.web]] jboss 5 TIMED_WAITING 0.0 0.000 0:0.046 false tru
39 DefaultQuartzScheduler_Worker-0 jboss 5 TIMED_WAITING 0.0 0.000 0:0.046 false fal
40 DefaultQuartzScheduler_Worker-1 jboss 5 TIMED_WAITING 0.0 0.000 0:0.046 false fal
70 Framework Active Thread jboss 5 TIMED_WAITING 0.0 0.000 0:0.031 false fal
138 http-0.0.0.0-8091-Acceptor-0 jboss 5 RUNNABLE 0.0 0.000 0:0.031 false tru
160 stariboss-payment.data jboss 5 TIMED_WAITING 0.0 0.000 0:0.031 false fal
5 Attach Listener system 5 RUNNABLE 0.0 0.000 0:0.015 false tru
19 GC Daemon system 2 TIMED_WAITING 0.0 0.000 0:0.015 false tru
146 arthas-NettyHttpTelnetBootstrap-3-1 system 5 RUNNABLE 0.0 0.000 0:0.015 false tru
Memory used total max usage GC
heap 201M 346M 455M 44.31% gc.ps_scavenge.count 660
ps_eden_space 31M 83M 169M 18.46% gc.ps_scavenge.time(ms) 10991
ps_survivor_space 448K 1024K 1024K 43.75% gc.ps_marksweep.count 69
ps_old_gen 169M 262M 341M 49.85% gc.ps_marksweep.time(ms) 18641
nonheap 175M 178M 304M 57.80%
code_cache 7M 10M 48M 16.48%
ps_perm_gen 167M 168M 256M 65.55%
direct 0K 0K - 0.00%
mapped 1M 1M - 100.00%
Runtime
os.name Windows 7
os.version 6.1
java.version 1.7.0_80
java.home C:\Java\jdk1.7.0_80\jre
systemload.average -1.00
processors 8
timestamp/uptime Mon Dec 26 10:56:25 CST 2022/239168s
[arthas@10668]$
说明
ID：Java级别的线程ID，注意这个ID不能跟jstack中的nativeID一一对应
NAME：线程名
GROUP：线程组名
PRIORITY：线程优先级, 1~10之间的数字，越大表示优先级越高
STATE：线程的状态
CPU%：线程消耗的cpu占比，采样100ms，将所有线程在这100ms内的cpu使用量求和，再算出每个线程的cpu使用占比。
TIME：线程运行总时间，数据格式为
分：秒
INTERRUPTED：线程当前的中断位状态
DAEMON：是否是daemon线程
2. 查看指定线程信息
Thread命令
参数说明
n：指定最忙的钱N个线程
b：找出当前是阻塞其他线程的线程
i ：指定CPU占比统计的采样时间，单位毫秒
例如 ：
thread 显示所有线程信息
thread 1 显示1号线程的运行堆栈
thread -b 查看阻塞的线程信息
thread -n 3 查看CPU使用率最高的3个线程
thread -i 1000 -n 3 ,每过1000秒采用，显示CPU使用最高的线程
#查看最耗CPU的两个线程
[arthas@10668]$ thread -n
"Reference Handler" Id=2 cpuUsage=0.0% deltaTime=0ms time=46ms WAITING on java.lang.ref.Reference$Lock@2ff15375
at java.lang.Object.wait(Native Method)
- waiting on java.lang.ref.Reference$Lock@2ff15375
at java.lang.Object.wait(Object.java:503)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
"http-0.0.0.0-8091-1" Id=157 cpuUsage=0.0% deltaTime=0ms time=9812ms RUNNABLE (in native)
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at oracle.net.ns.Packet.receive(Unknown Source)
at oracle.net.ns.DataPacket.receive(Unknown Source)
at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
at oracle.net.ns.NetInputStream.read(Unknown Source)
at oracle.net.ns.NetInputStream.read(Unknown Source)
at oracle.net.ns.NetInputStream.read(Unknown Source)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1104)
at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1075)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:480)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:219)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:970)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1190)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3370)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3454)
at org.jboss.resource.adapter.jdbc.WrappedPreparedStatement.executeUpdate(WrappedPreparedStatement.java:365)
at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:824)
at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:818)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:589)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:818)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:874)
at com.star.sms.dao.utils.SmsJdbcTemplate.update(SmsJdbcTemplate.java:119)
at com.star.sms.dao.utils.SmsJdbcTemplate.modify(SmsJdbcTemplate.java:227)
at com.star.sms.dao.customer.jdbc.CustomerInfoDaoJdbcImpl.modifyCustomer(CustomerInfoDaoJdbcImpl.java:216)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at com.star.common.interceptor.ThreadCacheInterceptor.invoke(ThreadCacheInterceptor.java:66)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
#查看157号线程信息
[arthas@10668]$ thread 157
"http-0.0.0.0-8091-1" Id=157 RUNNABLE (in native)
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at oracle.net.ns.Packet.receive(Unknown Source)
at oracle.net.ns.DataPacket.receive(Unknown Source)
at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
at oracle.net.ns.NetInputStream.read(Unknown Source)
at oracle.net.ns.NetInputStream.read(Unknown Source)
at oracle.net.ns.NetInputStream.read(Unknown Source)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1104)
at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1075)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:480)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:219)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:970)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1190)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3370)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3454)
at org.jboss.resource.adapter.jdbc.WrappedPreparedStatement.executeUpdate(WrappedPreparedStatement.java:365)
at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:824)
at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:818)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:589)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:818)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:874)
at com.star.sms.dao.utils.SmsJdbcTemplate.update(SmsJdbcTemplate.java:119)
at com.star.sms.dao.utils.SmsJdbcTemplate.modify(SmsJdbcTemplate.java:227)
at com.star.sms.dao.customer.jdbc.CustomerInfoDaoJdbcImpl.modifyCustomer(CustomerInfoDaoJdbcImpl.java:216)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at com.star.common.interceptor.ThreadCacheInterceptor.invoke(ThreadCacheInterceptor.java:66)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy1908.modifyCustomer(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
如果堆栈信息过大， 可以输出日志到文件
默认情况下arthas执行结果是不保存日志的，当然我们通过arthas提供的命令行options进行开启:
关闭日志打印
[arthas@10668]$ options save-result false NAME BEFORE-VALUE AFTER-VALUE
save-result true false
开启日志打印 [arthas@10668]$ options save-result true NAME BEFORE-VALUE AFTER-VALUE
save-result false true
[arthas@10668]$
日志会异步保存在 {user.home}/logs/arthas-cache/result.log;
3.排查为什么没有执行到修改的代码
jad 命令
参数说明
jad 包名.类名 反编译类
jad包名.类名 方法名 只反编译某个方法
例如：
反编译com.star.sms.service.accept2.dto.NewInstallAcceptDTO如下。jad命令不仅可以查看到ClassLoader还能看到运行的Location信息。 这样可以很清楚了解到运行的是那个包下的类。类覆盖问题就变的很容易发现。
[arthas@10668]$
[arthas@10668]$ jad com.star.sms.service.accept2.dto.NewInstallAcceptDTO
ClassLoader:
+-com.star.osgi.patch.classloader.BundlePatchURLClassLoader[stariboss-accept_interface]
+-org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader@4911704b[stariboss-accept_interface:1.0.0(id=99)]
+-org.eclipse.osgi.baseadaptor.BaseAdaptor$1@3983efa2
Location:
/E:/Jboss/jboss-4.2.3-8.2.1R1-8091-liaoning_821-5g/server/default/./deploy/stariboss.war/tmp/.patchtmp/1671784240888/a_780103_server_22112711_DB-9452.stariboss-accept_interface.jar
/*
* Decompiled with CFR.
*
* Could not load the following classes:
* com.star.common.lang.Money
* com.star.sms.model.customer.Customer
* com.star.sms.service.accept2.dto.AcceptSheetDTO
* com.star.sms.service.accept2.dto.BatchNoteInfoDTO
*/
package com.star.sms.service.accept2.dto;
import com.star.common.lang.Money;
import com.star.sms.model.customer.Customer;
import com.star.sms.service.accept2.dto.AcceptSheetDTO;
import com.star.sms.service.accept2.dto.BatchNoteInfoDTO;
import java.io.Serializable;
import java.util.Map;
/*
* This class specifies class file version 49.0 but uses Java 6 signatures. Assumed Java 6.
*/
public class NewInstallAcceptDTO
implements Serializable {
private static final long serialVersionUID = 5229054864090922612L;
private int rowNum;
private AcceptSheetDTO acceptSheetDTO;
private Customer customer;
private Money businessFee;
private Money preSavingFee;
private Money transferFee;
private Map<String, BatchNoteInfoDTO> notedtos;
private boolean contactTelRepeatAble = false;
public boolean getContactTelRepeatAble() {
/*86*/ return this.contactTelRepeatAble;
}
public void setContactTelRepeatAble(boolean contactTelRepeatAble) {
/*90*/ this.contactTelRepeatAble = contactTelRepeatAble;
}
public Customer getCustomer() {
/*38*/ return this.customer;
}
public void setCustomer(Customer customer) {
/*42*/ this.customer = customer;
}
public AcceptSheetDTO getAcceptSheetDTO() {
/*30*/ return this.acceptSheetDTO;
}
public void setAcceptSheetDTO(AcceptSheetDTO acceptSheetDTO) {
/*34*/ this.acceptSheetDTO = acceptSheetDTO;
}
public Money getBusinessFee() {
/*46*/ return this.businessFee;
}
public void setBusinessFee(Money businessFee) {
/*50*/ this.businessFee = businessFee;
}
public Money getPreSavingFee() {
/*54*/ return this.preSavingFee;
}
public void setPreSavingFee(Money preSavingFee) {
/*58*/ this.preSavingFee = preSavingFee;
}
public Money getTransferFee() {
/*62*/ return this.transferFee;
}
public void setTransferFee(Money transferFee) {
/*66*/ this.transferFee = transferFee;
}
public Map<String, BatchNoteInfoDTO> getNotedtos() {
/*70*/ return this.notedtos;
}
public void setNotedtos(Map<String, BatchNoteInfoDTO> notedtos) {
/*74*/ this.notedtos = notedtos;
}
public int getRowNum() {
/*78*/ return this.rowNum;
}
public void setRowNum(int rowNum) {
/*82*/ this.rowNum = rowNum;
}
}
如果类过大，反编译内容会很多，建议只反编译某个方法，例如
[arthas@10668]$
[arthas@10668]$ jad com.star.sms.service.support.GroupParamContainer getCustomer
Affect(row-cnt:0) cost in 37 ms.
No class found for: com.star.sms.service.support.GroupParamContainer
[arthas@10668]$ jad com.star.sms.business.customer.CustomerInfoService getCustomer
ClassLoader:
+-com.star.osgi.patch.classloader.BundlePatchURLClassLoader[stariboss-customer]
+-org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader@4089eba3[stariboss-customer:1.0.0(id=124)]
+-org.eclipse.osgi.baseadaptor.BaseAdaptor$1@3983efa2
Location:
/E:/Jboss/jboss-4.2.3-8.2.1R1-8091-liaoning_821-5g/server/default/./deploy/stariboss.war/tmp/.patchtmp/1671784240888/a_780108_server_22112201_DB-9505.stariboss-customer.jar
private Customer getCustomer(CustomerInfoDTO result, Long certificateTypeId, String certificateNum, SubscriberResultDTO subscriberResultDTO) {
Customer customer = new Customer();
/*730*/ if (this.getCustomerCodeGenerator() == null) {
throw new MsrRuntimeException("customer.code.is.required", false, false, null);
}
/*731*/ this.getCustomerCodeGenerator().generateAndSetCustomerCode(customer);
/*736*/ customer.setName(result.getCUST_NAME());
/*748*/ customer.setCertificate(this.getCertificate(result, certificateTypeId, certificateNum));
/*752*/ customer.setContactAddress("");
/*753*/ customer.setContactTel(result.getCONT_NUMBER());
/*755*/ customer.setEnrolDate(new Date());
/*758*/ customer.setMobileTel(result.getCONT_NUMBER());
/*760*/ customer.setCustomerAddress("");
/*762*/ customer.setSaleChannel((SaleChannel)ApplicationSessionHolder.getApplicationSession().getValue("currentSaleChannel"));
/*765*/ Sex sex = Sex.getByCode((String)result.getGENDER());
/*766*/ customer.setSex(sex == null ? Sex.MAN : sex);
/*769*/ customer.setStatus(CustomerStatus.EXISTING);
/*771*/ customer.setType(CustomerType.INDIVIDUAL);
/*773*/ SaleArea saleArea = this.getSaleAreaByGw(subscriberResultDTO.getCITY_CODE());
/*774*/ customer.setSaleArea(saleArea);
/*777*/ Address address = this.getAddress(saleArea.getId());
/*778*/ customer.setAddress(address);
/*779*/ customer.setContactMan(result.getCUST_NAME());
/*785*/ customer.setDetailAddressCode("");
/*796*/ customer.setRemark("createByGwInfo,");
/*797*/ return customer;
}
4.方法调用路径，以及性能分析
trace 命令： 方法内部调用路径，并输出方法路径上的每个节点上耗时
例如： 查看GwPaymentService#callFeePay 调用路径和耗时
[arthas@10668]$ trace com.star.sms.business.payment.gw.GwPaymentService callFeePay
Press Q or Ctrl+C to abort.
Affect(class count: 1 , method count: 1) cost in 184 ms, listenerId: 3
`---ts=2022-12-26 13:37:22;thread_name=http-0.0.0.0-8091-1;id=9d;is_daemon=true;priority=5;TCCL=org.jboss.web.tomc
`---[41.232898ms] com.star.sms.business.payment.gw.GwPaymentService:callFeePay()
+---[0.020813ms] com.star.sms.business.payment.gw.helper.GwPaymentConverter:getInstance() #65
+---[0.018817ms] com.star.sms.business.payment.gw.helper.GwPaymentConverter:callFeePayConvert() #65
+---[0.019102ms] com.star.sms.gw.payment.GwPayDTO:getPayRecordId() #66
`---[41.047583ms] com.star.sms.business.payment.gw.GwPaymentService:doPayFee() #66
trace 命令只会 trace 匹配到的函数里的子调用，并不会向下 trace 多层。因为 trace 是代价比较贵的
可以用正则表匹配路径上的多个类和函数，一定程度上达到多层 trace 的效果
例如： trace -E com.test.ClassA|org.test.ClassB method1|method2|method3
[arthas@13688]$ trace -E com.star.sms.business.provision.send.InstructionSender sendInstruction|sendGroupByGroup
Press Q or Ctrl+C to abort.
Affect(class count: 1 , method count: 2) cost in 218 ms, listenerId: 7
`---ts=2023-01-06 13:52:59;thread_name=http-0.0.0.0-8089-2;id=a2;is_daemon=true;priority=5;TCCL=org.jboss.web.tomcat.service.WebAppClassLoader@2c08c092
`---[3.387799ms] com.star.sms.business.provision.send.InstructionSender:sendInstruction()
+---[0.028225ms] org.springframework.util.CollectionUtils:isEmpty() #24
+---[0.01397ms] com.star.sms.model.product.core.business.OpenDevice:getOpenDeviceType() #27
+---[0.007413ms] com.star.sms.model.provision.enums.OpenDeviceType:name() #27
+---[0.005987ms] com.star.sms.model.product.core.business.OpenDevice:getCode() #27
+---[0.29422ms] org.apache.commons.logging.Log:info() #27
+---[0.01511ms] com.star.sms.business.provision.send.util.InstructionGroupUtil:groupInstructionByServiceStr() #29
+---[0.137417ms] com.star.sms.business.provision.send.InstructionSender:translate2Req() #30
+---[2.620605ms] com.star.sms.business.provision.send.InstructionSender:sendGroupByGroup() #31
| `---[2.603785ms] com.star.sms.business.provision.send.InstructionSender:sendGroupByGroup()
| `---[2.538497ms] com.star.sms.business.provision.send.InstructionSender:sendOneGroupReq() #52
+---[0.003421ms] com.star.sms.model.product.core.business.OpenDevice:getOpenDeviceType() #32
+---[0.003136ms] com.star.sms.model.provision.enums.OpenDeviceType:name() #32
+---[0.002851ms] com.star.sms.model.product.core.business.OpenDevice:getCode() #32
`---[0.118885ms] org.apache.commons.logging.Log:info() #32
5.线上 debug
watch命令检测函数入参返回值，抛出异常等信息
参数说明、
params : 表示所有参数数组 （因为不确定是几个参数）
returnObject:表示返回值
-b 方法调用之前观察
-e 方法异常后观察
-s 方法返回后观察
-f 方法结束后观察
-x 属性深度
例如：
watch com.star.sms.business.payment.gw.GwPayService payOrder returnObj 观察返回值
watch com.star.sms.business.payment.gw.GwPayService payOrder params 观察输入参数
watch com.star.sms.business.payment.gw.GwPayService payOrder "{params, returnObj}" -x 3
[arthas@10668]$ watch com.star.sms.business.payment.gw.GwPayService payOrder returnObj
Press Q or Ctrl+C to abort.
Affect(class count: 1 , method count: 1) cost in 694 ms, listenerId: 1
method=com.star.sms.business.payment.gw.GwPayService.payOrder location=AtExit
ts=2022-12-23 16:39:28; [cost=85.028278ms] result=@GwPayRecord[
serialVersionUID=@Long[6159375491528104642],
USER_ID=null,
groupNumber=null,
phoneNumber=@String[19200000078],
operationType=@GwPayOperaType[CALL_FEE_PAY],
payMethod=@DictionaryData[com.star.sms.model.system.basicparameter.DictionaryData@13d7],
payChannelType=@PayChannelType[SCAN_PAY],
payFee=@Money[锟? 10.00],
payDate=@Date[2022-12-23 00:00:00,000],
tradeNo=@String[202291392],
outTradeNo=@String[221223163927000244],
barCode=null,
buyerId=null,
thirdPayStatus=@PayStatus[SUCCESS],
bossPayStatus=@ScanPaymentSign[UNPAYED],
developOperator=null,
status=@Status[VALID],
acceptSheetId=null,
resp_code=@String[000000],
resp_desc=@String[璋冪敤鎴愬姛],
order_no=@String[12],
order_pay_status=@String[2],
pay_page_url=null,
scan_url=@String[http://www.baidu.com],
sign_info=@String[hhggr],
business_id=null,
gw_pos_trade_no=null,
prov_mer_order_no=null,
posPayStatus=null,
posRespDesc=null,
payNo=@String[13214],
id=@Long[893],
name=null,
createDate=@Timestamp[2022-12-23 16:39:27,292],
modifyDate=@Timestamp[2022-12-23 16:39:28,000],
createOperator=@Operator[com.star.sms.model.system.organization.Operator@5d],
modifyOperator=@Operator[com.star.sms.model.system.organization.Operator@5d],
remark=null,
operationTerminal=@Terminal[com.star.sms.model.system.organization.Terminal@2d660cb5],
operationSpot=@SaleChannel[com.star.sms.model.system.organization.SaleChannel@9],
version=@Integer[0],
]
[arthas@10668]$
[arthas@10668]$ watch com.star.sms.business.payment.gw.GwPayService payOrder "{params, returnObj}" -x 3
Press Q or Ctrl+C to abort.
Affect(class count: 1 , method count: 1) cost in 129 ms, listenerId: 4
method=com.star.sms.business.payment.gw.GwPayService.payOrder location=AtExit
ts=2022-12-26 13:56:10; [cost=43.709849ms] result=@ArrayList[
@Object[][
@GwPayRecord[
serialVersionUID=@Long[6159375491528104642],
USER_ID=null,
groupNumber=null,
phoneNumber=@String[19200007899],
operationType=@GwPayOperaType[CALL_FEE_PAY],
payMethod=@DictionaryData[com.star.sms.model.system.basicparameter.DictionaryData@13d7],
payChannelType=@PayChannelType[SCAN_PAY],
payFee=@Money[锟? 99.00],
payDate=@Date[2022-12-26 00:00:00,000],
tradeNo=@String[202299042],
outTradeNo=@String[221226135609002605],
barCode=null,
buyerId=null,
thirdPayStatus=@PayStatus[SUCCESS],
bossPayStatus=@ScanPaymentSign[UNPAYED],
developOperator=null,
status=@Status[VALID],
acceptSheetId=null,
resp_code=@String[000000],
resp_desc=@String[璋冪敤鎴愬姛],
order_no=@String[12],
order_pay_status=@String[2],
pay_page_url=null,
scan_url=@String[http://www.baidu.com],
sign_info=@String[hhggr],
business_id=null,
gw_pos_trade_no=null,
prov_mer_order_no=null,
posPayStatus=null,
posRespDesc=null,
payNo=@String[13214],
id=@Long[898],
name=null,
createDate=@Timestamp[2022-12-26 13:56:09,981],
modifyDate=@Timestamp[2022-12-26 13:56:10,000],
createOperator=@Operator[com.star.sms.model.system.organization.Operator@5d],
modifyOperator=@Operator[com.star.sms.model.system.organization.Operator@5d],
remark=@String[TRACE TEST],
operationTerminal=@Terminal[com.star.sms.model.system.organization.Terminal@7a05dcd9],
operationSpot=@SaleChannel[com.star.sms.model.system.organization.SaleChannel@9],
version=@Integer[0],
],
],
@GwPayRecord[
serialVersionUID=@Long[6159375491528104642],
USER_ID=null,
groupNumber=null,
phoneNumber=@String[19200007899],
operationType=@GwPayOperaType[
CALL_FEE_PAY=@GwPayOperaType[CALL_FEE_PAY],
BUSINESS_PAY=@GwPayOperaType[BUSINESS_PAY],
DAMAGES_PAY=@GwPayOperaType[DAMAGES_PAY],
$VALUES=@GwPayOperaType[][isEmpty=false;size=3],
name=@String[CALL_FEE_PAY],
ordinal=@Integer[0],
],
payMethod=@DictionaryData[
serialVersionUID=@Long[4048790182402733107],
code=@String[2800],
displayPos=@Integer[0],
type=@DictionaryDataType[com.star.sms.model.system.basicparameter.DictionaryDataType@1b5b],
status=@Status[VALID],
operateLevel=null,
saleAreas=null,
synchronizeStatus=@SynchronizeStatus[UNSYNCHRONIZED],
synchronizedt=null,
gw5GCorpOrgCode=null,
regionID=null,
regionExtId=null,
id=@Long[5079],
name=@String[鐜伴噾鏀粯],
createDate=@Timestamp[2022-03-30 12:13:49,000],
modifyDate=null,
createOperator=@Operator[com.star.sms.model.system.organization.Operator@710107e1],
modifyOperator=@Operator[com.star.sms.model.system.organization.Operator@7aac7ebc],
remark=null,
operationTerminal=@Terminal[com.star.sms.model.system.organization.Terminal@52b4a78f],
operationSpot=@SaleChannel[com.star.sms.model.system.organization.SaleChannel@5264c7b4],
version=@Integer[0],
],
payChannelType=@PayChannelType[
CARD_SWIP=@PayChannelType[CARD_SWIP],
SCAN_PAY=@PayChannelType[SCAN_PAY],
$VALUES=@PayChannelType[][isEmpty=false;size=2],
name=@String[SCAN_PAY],
ordinal=@Integer[1],
],
payFee=@Money[
MOVE_CODE=@Integer[32],
serialVersionUID=@Long[3256444694312726840],
CENTS=@int[][isEmpty=false;size=4],
HanDigiStr=@String[][isEmpty=false;size=10],
HanDiviStr=@String[][isEmpty=false;size=24],
ZERO=@Money[锟? 0.00],
CURRENCY_SYMBOL_POSTFIX=@String[ ],
amount=@Long[9900],
currency=@Currency[CNY],
currentCurrency=@Currency[CNY],
],
payDate=@Date[2022-12-26 00:00:00,000],
tradeNo=@String[202299042],
outTradeNo=@String[221226135609002605],
barCode=null,
buyerId=null,
thirdPayStatus=@PayStatus[
SUCCESS=@PayStatus[SUCCESS],
FAIL=@PayStatus[FAIL],
UNKOWN=@PayStatus[UNKOWN],
CANCEL=@PayStatus[CANCEL],
NOTPAY=@PayStatus[NOTPAY],
$VALUES=@PayStatus[][isEmpty=false;size=5],
name=@String[SUCCESS],
ordinal=@Integer[0],
],
bossPayStatus=@ScanPaymentSign[
UNPAYED=@ScanPaymentSign[UNPAYED],
PAYED=@ScanPaymentSign[PAYED],
$VALUES=@ScanPaymentSign[][isEmpty=false;size=2],
name=@String[UNPAYED],
ordinal=@Integer[0],
],
developOperator=null,
status=@Status[
INVALID=@Status[INVALID],
VALID=@Status[VALID],
$VALUES=@Status[][isEmpty=false;size=2],
name=@String[VALID],
ordinal=@Integer[1],
],
acceptSheetId=null,
resp_code=@String[000000],
resp_desc=@String[璋冪敤鎴愬姛],
order_no=@String[12],
order_pay_status=@String[2],
pay_page_url=null,
scan_url=@String[http://www.baidu.com],
sign_info=@String[hhggr],
business_id=null,
gw_pos_trade_no=null,
prov_mer_order_no=null,
posPayStatus=null,
posRespDesc=null,
payNo=@String[13214],
id=@Long[898],
name=null,
createDate=@Timestamp[2022-12-26 13:56:09,981],
modifyDate=@Timestamp[2022-12-26 13:56:10,000],
createOperator=@Operator[
PROPERTY_USERNAME=@String[code],
PROPERTY_PASSWORD=@String[password],
DYNAMIC_PASSWORD=@String[dynamicPassword],
serialVersionUID=@Long[3256443599062578999],
code=@String[0147],
password=null,
dynamicPassword=null,
type=@OperatorType[COMMON_OP],
tel=null,
familyAddress=null,
email=null,
validDateRange=null,
sex=null,
identityCard=null,
education=null,
department=@Department[com.star.sms.model.system.organization.Department@3],
level=@OperationLevel[com.star.sms.model.system.privilege.OperationLevel@1],
address=@Address[com.star.sms.model.system.resource.Address@1],
rayNode=null,
saleAreas=@ArrayList[isEmpty=false;size=13],
roles=@ArrayList[isEmpty=false;size=16],
saleChannels=@ArrayList[isEmpty=false;size=15],
channelBindings=@ArrayList[isEmpty=false;size=15],
segment=null,
synchronizeStatus=null,
synchronizedt=null,
gw5GOperCode=@String[OPER0001],
storeHouses=null,
terminals=null,
mustBindTerminal=@Boolean[false],
status=null,
selectedSaleChannel=null,
dynamicPasswordCheck=null,
isVIPOperator=@Boolean[false],
passwordLastModifyDate=@Date[2021-11-02 00:00:00,000],
imei=null,
jobType=null,
id=@Long[93],
name=@String[huangzw],
createDate=null,
modifyDate=null,
createOperator=null,
modifyOperator=null,
remark=null,
operationTerminal=null,
operationSpot=null,
version=@Integer[0],
],
//略
version=@Integer[0],
],
]
