dremio 存储插件之StoragePluginRulesFactory 类简单说明

StoragePluginRulesFactory 是dremio 为了分离每个插件的规则，我们存储插件可以包含自己的规则工厂，具体使用到StoragePluginRulesFactory
的包含了CatalogServiceImpl，SqlHandlerConfig，实际都会到sql 执行的handler 中

系统内置实现

 

 

集成使用

主要是CatalogServiceImpl.java 中，其他执行部分通过间接引用

 
public RuleSet getStorageRules(OptimizerRulesContext context, PlannerPhase phase) {
    final ImmutableSet.Builder<RelOptRule> rules = ImmutableSet.builder();
    final Set<SourceType> types = new HashSet<>();
 
    try {
      for (ManagedStoragePlugin plugin : getPlugins().managed()) {
        // we want to check state without acquiring a read lock
        if (plugin.getState().getStatus() == SourceState.SourceStatus.bad) {
          // we shouldn't consider rules for misbehaving plugins.
          continue;
        }
 
        StoragePluginId pluginId;
        try {
          // getId has a check for plugin state
          pluginId = plugin.getId();
        } catch (UserException e) {
          if (e.getErrorType() == ErrorType.SOURCE_BAD_STATE) {
            // we shouldn't consider rules for misbehaving plugins.
            continue;
          }
          throw e;
        }
 
        StoragePluginRulesFactory factory = plugin.getRulesFactory();
        if(factory != null) {
          // add instance level rules.
          rules.addAll(factory.getRules(context, phase, pluginId));
 
          // add type level rules.
          if(types.add(pluginId.getType())) {
            rules.addAll(factory.getRules(context, phase, pluginId.getType()));
          }
        }
      }
    } catch (InstantiationException | IllegalAccessException e) {
      throw UserException.validationError(e).message("Failure getting plugin rules.").build(logger);
    }
 
    ImmutableSet<RelOptRule> rulesSet = rules.build();
    return RuleSets.ofList(rulesSet);
  }

SqlHandlerConfig 对于规则的使用 SqlHandlerConfig.java 间接也是通过执行上下文使用的CatalogService

 public RuleSet getRules(PlannerPhase phase) {
    return PlannerPhase.mergedRuleSets(
        context.getInjectedRules(phase),
        phase.getRules(context),
        context.getCatalogService().getStorageRules(context, phase));
  }

说明

dremio StoragePluginRulesFactory 还是比较强大的，扩展了dremio sql 处理能力，是一个很不错的扩展点 尤其我们需要开发自己的存储扩展的时候

参考资料

https://github.com/dremio/dremio-oss/blob/d41cb52143b6b0289fc8ed4d970bfcf410a669e8/sabot/kernel/src/main/java/com/dremio/exec/catalog/CatalogServiceImpl.java
https://github.com/dremio/dremio-oss/blob/d41cb52143b6b0289fc8ed4d970bfcf410a669e8/sabot/kernel/src/main/java/com/dremio/exec/planner/sql/handlers/SqlHandlerConfig.java
https://github.com/dremio/dremio-oss/blob/d41cb52143b6b0289fc8ed4d970bfcf410a669e8/sabot/kernel/src/main/java/com/dremio/exec/store/StoragePlugin.java

posted on 2022-10-21 12:52  荣锋亮  阅读(22)  评论(0)  编辑  收藏  举报

导航