dremio 测试类SabotNode简单说明

实际上我以前简单说明过dremio 的一些测试类以及如何进行测试一般我们使用BaseTestQuery 就可以了
实际上对于测试dremio 包装了一个SabotNode 类，提供了不带ui 的测试框架能力(server 引擎能力)

SabotNode 提供的能力

SabotNode 是一个dremio 运行依赖的基本组件的包装，了解dremioSabotNode 依赖的组件之后可以更好的学习 dremio

 

 

基本测试使用BaseTestQuery 类

• 测试类

 

  @BeforeClass

  public static void setupDefaultTestCluster() throws Exception {

    config = SabotConfig.create(TEST_CONFIGURATIONS);

 

    Module module = Optional.ofNullable(SABOT_NODE_RULE.module)

      .map(m -> Modules.combine(m, getDefaultModule())).orElse(getDefaultModule());

 

    SABOT_NODE_RULE.register(module);

 

    openClient();

    localFs = HadoopFileSystem.getLocal(new Configuration());

 

    // turns on the verbose errors in tests

    // sever side stacktraces are added to the message before sending back to the client

    setSessionOption(ExecConstants.ENABLE_VERBOSE_ERRORS_KEY, "true");

  }

openClient 创建SabotNode 

protected static void openClient() throws Exception {

    clusterCoordinator = LocalClusterCoordinator.newRunningCoordinator();

 

    dfsTestTmpSchemaLocation = TestUtilities.createTempDir();

 

    nodes = new SabotNode[nodeCount];

    for(int i = 0; i < nodeCount; i++) {

      // first node has all roles, and all others are only executors

      nodes[i] = SABOT_NODE_RULE.newSabotNode(new SabotProviderConfig(i == 0));

      nodes[i].run();

      if(i == 0) {

        TestUtilities.addDefaultTestPlugins(nodes[i].getContext().getCatalogService(), dfsTestTmpSchemaLocation, true);

      }

    }

 

    client = QueryTestUtil.createClient(config,  clusterCoordinator, MAX_WIDTH_PER_NODE, defaultProperties);

 

    // turn off re-attempts, this needs to be set at the system level as many unit test will

    // reset the user session by restarting the client

    setEnableReAttempts(false);

  }

SabotModule 模块能力

SabotModule 属于一个guice 的标准模块，包含了依赖的模块，可以了解依赖的组件

 

 

测试参考使用

实际上官方测试案例代码还是比较全的，可以了解测试的使用，同时dremio 对于测试包也发布了maven 私服可以方便使用
比如简单source 扩展的测试（执行sql 操作的）

 

public class MyPluginTest extends BaseTestQuery {

    private CrateConf crateConf;

 

    @Before

    public  void initSource(){

        getSabotContext().getOptionManager().setOption(OptionValue.createLong(OptionValue.OptionType.SYSTEM, ExecConstants.ELASTIC_ACTION_RETRIES, 3));

        SourceConfig sc = new SourceConfig();

        sc.setName("cratedb");

        crateConf  = new CrateConf();

        crateConf.host="127.0.0.1";

        crateConf.port=5433;

        crateConf.username="crate";

        sc.setConnectionConf(crateConf);

        sc.setMetadataPolicy(CatalogService.DEFAULT_METADATA_POLICY);

        getSabotContext().getCatalogService().createSourceIfMissingWithThrow(sc);

    }

 

    @Test

    public  void test() throws Exception {

        String query  = "select * from cratedb.doc.demoapp";

        TestResult testResult=  testBuilder()

                .sqlQuery(query)

                .unOrdered()

                .baselineColumns("id", "name")

                .baselineValues(null,    null)

                .go();

    }

}

说明

dremio 组件比较多，代码量也不少，直接通过源码学习不是很好，了解测试类还是比较重要的，可以了解dremio 的运行机制，同时加速功能测试

参考资料

https://www.cnblogs.com/rongfengliang/p/14642625.html
https://www.cnblogs.com/rongfengliang/p/15966269.html
https://www.cnblogs.com/rongfengliang/p/15851639.html
https://github.com/dremio/dremio-oss/blob/d41cb52143b6b0289fc8ed4d970bfcf410a669e8/sabot/kernel/src/test/java/com/dremio/BaseTestQuery.java
https://github.com/dremio/dremio-oss/blob/d41cb52143b6b0289fc8ed4d970bfcf410a669e8/sabot/kernel/src/main/java/com/dremio/exec/ExecConstants.java
https://github.com/dremio/dremio-oss/blob/d41cb52143b6b0289fc8ed4d970bfcf410a669e8/sabot/kernel/src/test/java/com/dremio/exec/server/SabotNode.java