JVM sandbox 实现热修复示例

JVM-SANDBOX简介

JVM-SANDBOX（沙箱）实现了一种在不重启、不侵入目标JVM应用的AOP解决方案。
GIT 地址 https://github.com/alibaba/jvm-sandbox

具有以下特性：
1）无侵入：目标应用无需重启也无需感知沙箱的存在
2）类隔离：沙箱以及沙箱的模块不会和目标应用的类相互干扰
3）可插拔：沙箱以及沙箱的模块可以随时加载和卸载，不会在目标应用留下痕迹
4）多租户：目标应用可以同时挂载不同租户下的沙箱并独立控制
5）高兼容：支持JDK[6,11]

本文将介绍利用 JVM-SANDBOX 自定义修复 exception 的方法。

故障模拟

原服务模拟抛出异常的代码如下：

import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@Slf4j
@RestController
public class SandboxController {

    @GetMapping("/test/void")
    public String testVoid() {
        error();
        return "success";
    }

    private void error() {
        throw new IllegalStateException("Illegal state!");
    }

}

此时，只要请求 /test/void 都会抛出异常。

修复示例

现在，我们提供一个热修复方案：

1. 安装jvm-sandbox。

# 下载
wget https://ompc.oss.aliyuncs.com/jvm-sandbox/release/sandbox-stable-bin.zip
# 解压
unzip sandbox-stable-bin.zip

2. 新建工程，编写修复代码

import com.alibaba.jvm.sandbox.api.Information;
import com.alibaba.jvm.sandbox.api.Module;
import com.alibaba.jvm.sandbox.api.ProcessController;
import com.alibaba.jvm.sandbox.api.annotation.Command;
import com.alibaba.jvm.sandbox.api.listener.ext.Advice;
import com.alibaba.jvm.sandbox.api.listener.ext.AdviceListener;
import com.alibaba.jvm.sandbox.api.listener.ext.EventWatchBuilder;
import com.alibaba.jvm.sandbox.api.resource.ModuleEventWatcher;
import org.kohsuke.MetaInfServices;

import javax.annotation.Resource;

/**
 * @author zhengqian
 * @date 2023.02.06
 */
@MetaInfServices(Module.class)
@Information(id = "exception-handler")
public class ExceptionHandlerModule implements Module {

    @Resource
    private ModuleEventWatcher moduleEventWatcher;

    @Command("repairExceptionVoid")
    public void repairExceptionVoid() {
        new EventWatchBuilder(moduleEventWatcher)
                .onClass("xxx.xxx.controller.SandboxController") //对应类名
                .onBehavior("error")  // 对应方法名
                .onWatch(new AdviceListener() {
                    /**
                     * 拦截指定方法，当这个方法抛出异常时将会被
                     * AdviceListener#afterThrowing()所拦截
                     */
                    @Override
                    protected void afterThrowing(Advice advice) throws Throwable {

                        // 在此，你可以通过ProcessController来改变原有方法的执行流程
                        // 这里的代码意义是：改变原方法抛出异常的行为，变更为立即返回；void返回值用null表示
                        ProcessController.returnImmediately(null);
                    }
                });
    }
}

工程依赖参考

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>jvm-sandbox-tool</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.deploy.skip>true</maven.deploy.skip>
        <maven.install.skip>true</maven.install.skip>
    </properties>

    <build>
        <finalName>${project.name}-${project.version}</finalName>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.0</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                    <encoding>UTF-8</encoding>
                    <showDeprecation>true</showDeprecation>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>attached</goal>
                        </goals>
                        <phase>package</phase>
                        <configuration>
                            <descriptorRefs>
                                <descriptorRef>jar-with-dependencies</descriptorRef>
                            </descriptorRefs>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <dependencies>

        <dependency>
            <groupId>com.alibaba.jvm.sandbox</groupId>
            <artifactId>sandbox-module-starter</artifactId>
            <version>1.4.0</version>
            <type>pom</type>
        </dependency>
        <dependency>
            <groupId>com.alibaba.jvm.sandbox</groupId>
            <artifactId>sandbox-api</artifactId>
            <version>1.4.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>com.alibaba.jvm.sandbox</groupId>
            <artifactId>sandbox-common-api</artifactId>
            <version>1.4.0</version>
        </dependency>
        <dependency>
            <groupId>org.kohsuke.metainf-services</groupId>
            <artifactId>metainf-services</artifactId>
            <version>1.9</version>
            <scope>provided</scope>
        </dependency>


        <!-- DEBUG工程公共依赖模块，SANDBOX父工程托管 -->
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.4</version>
        </dependency>
        <dependency>
            <groupId>commons-collections</groupId>
            <artifactId>commons-collections</artifactId>
            <version>3.2.2</version>
        </dependency>

        <!-- DEBUG工程独有依赖模块，所以单独配置并指定了版本 -->
        <dependency>
            <groupId>ognl</groupId>
            <artifactId>ognl</artifactId>
            <version>3.0.8</version>
        </dependency>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>18.0</version>
        </dependency>

        <!-- 给DEBUG模块单独配置一个日志 -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.24</version>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.2.1</version>
        </dependency>

        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>javax.servlet-api</artifactId>
            <version>4.0.1</version>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>com.alibaba.fastjson2</groupId>
            <artifactId>fastjson2</artifactId>
            <version>2.0.23</version>
        </dependency>
    </dependencies>

</project>

3. 打包，启动

# 打包
mvn clean package
# 将打包好的内容复制到第一步sandbox-module的目录位置，注意改成你自己的目录
cp target/jvm-sandbox-tool-1.0-SNAPSHOT-jar-with-dependencies.jar ~/install/jvm-sandbox/sandbox/sandbox-module/.

4. 启动

# 这里假设 27377 是目标进程号（也就是报异常的原服务）
./sandbox.sh -p 27377

5. 执行修复指令

./sandbox.sh -p 27377 -d 'exception-handler/repairExceptionVoid'

此时再请求 /test/void 会正常返回，不会报错。使用指令 ./sandbox.sh -p 27377 -S停用sandbox后，服务恢复异常，实现了热插拔。

改为可传参的通用版本

import com.alibaba.fastjson2.JSONObject;
import com.alibaba.jvm.sandbox.api.Information;
import com.alibaba.jvm.sandbox.api.Module;
import com.alibaba.jvm.sandbox.api.ProcessController;
import com.alibaba.jvm.sandbox.api.annotation.Command;
import com.alibaba.jvm.sandbox.api.http.printer.ConcurrentLinkedQueuePrinter;
import com.alibaba.jvm.sandbox.api.http.printer.Printer;
import com.alibaba.jvm.sandbox.api.listener.ext.Advice;
import com.alibaba.jvm.sandbox.api.listener.ext.AdviceListener;
import com.alibaba.jvm.sandbox.api.listener.ext.EventWatchBuilder;
import com.alibaba.jvm.sandbox.api.listener.ext.EventWatcher;
import com.alibaba.jvm.sandbox.api.resource.ModuleEventWatcher;
import org.apache.commons.lang3.StringUtils;
import org.kohsuke.MetaInfServices;

import javax.annotation.Resource;
import java.io.PrintWriter;
import java.util.ArrayList;
import java.util.Map;

@MetaInfServices(Module.class)
@Information(id = "exception-repair", version = "0.0.1", author = "zhengqian@rd.netease.com")
public class ExceptionRepairModule extends ParamSupported implements Module {

    @Resource
    private ModuleEventWatcher moduleEventWatcher;

    @Command("returnObject")
    public void returnObject(final Map<String, String> param, final PrintWriter writer) {
        final String cnPattern = getParameter(param, "class");
        final String mnPattern = getParameter(param, "method");
        final String rtPattern = getParameter(param, "return");
        final String rtString = getParameter(param, "returnString");

        final Printer printer = new ConcurrentLinkedQueuePrinter(writer);

        final EventWatcher watcher = new EventWatchBuilder(moduleEventWatcher)
                .onClass(cnPattern)
                .onBehavior(mnPattern)
                .onWatch(new AdviceListener() {
                    /**
                     * 拦截指定方法，当这个方法抛出异常时将会被
                     * AdviceListener#afterThrowing()所拦截
                     */
                    @Override
                    protected void afterThrowing(Advice advice) throws Throwable {

                        Class clazz = Class.forName(rtPattern);
                        Object object;
                        if (StringUtils.isEmpty(rtString)) {
                            object = clazz.newInstance();
                            printer.print("repair exception, return empty object");
                        } else {
                            object = JSONObject.parseObject(rtString, clazz);
                            printer.print("repair exception, return object: " + object.toString());
                        }

                        ProcessController.returnImmediately(object);
                    }
                });

        try {
            printer.println(String.format(
                    "tracing on [%s#%s].\nPress CTRL_C abort it!",
                    cnPattern,
                    mnPattern
            ));
            printer.waitingForBroken();
        } finally {
            watcher.onUnWatched();
        }
    }

    @Command("returnEmptyList")
    public void returnList(final Map<String, String> param, final PrintWriter writer) {
        final String cnPattern = getParameter(param, "class");
        final String mnPattern = getParameter(param, "method");
        final String rtPattern = getParameter(param, "return");

        final Printer printer = new ConcurrentLinkedQueuePrinter(writer);

        final EventWatcher watcher = new EventWatchBuilder(moduleEventWatcher)
                .onClass(cnPattern)
                .onBehavior(mnPattern)
                .onWatch(new AdviceListener() {
                    /**
                     * 拦截指定方法，当这个方法抛出异常时将会被
                     * AdviceListener#afterThrowing()所拦截
                     */
                    @Override
                    protected void afterThrowing(Advice advice) throws Throwable {
                        ProcessController.returnImmediately(new ArrayList<>());
                    }
                });

        try {
            printer.println(String.format(
                    "tracing on [%s#%s].\nPress CTRL_C abort it!",
                    cnPattern,
                    mnPattern
            ));
            printer.waitingForBroken();
        } finally {
            watcher.onUnWatched();
        }
    }
}

故障模拟

@Slf4j
@RestController
public class SandboxController {

    @GetMapping("/test/object")
    public SandboxReturnType testObject() {
        SandboxReturnType type = errorObject();
        log.info("type:{}", type);
        return type;
    }

    private SandboxReturnType errorObject() {
        throw new IllegalStateException("Illegal state!");
    }

}

@Data
public class SandboxReturnType {
    private String name;
}

执行指令，可以指定作用的类、方法、以及需要返回的object（传入的是json字段串，且这里的 returnString 做了urlencode）

./sandbox.sh -p 47620 -d 'exception-repair/returnObject?class=org.example.demo.controller.SandboxController&method=errorObject&return=org.example.demo.controller.SandboxReturnType&returnString=%7B%22name%22%3A%22111%22%7D'

此时，请求 /test/object 时可以正常返回传入的数据，实现热修复

{
  "name": 111
}

可传参的通用代码可以打包上传，需要时直接下载使用即可，不需要每次重复写代码、打包处理。