dremio SchemaMutability 简单说明

dremio SchemaMutability 属于一个枚举，定义了schema 的可变性能力()

参考定义

public enum SchemaMutability {
  @Tag(1)
  ALL(true, true, true, true),
 
  @Tag(2)
  NONE(false, false, false, false),
 
  @Tag(3)
  SYSTEM_TABLE(false, true, false, false),
 
  @Tag(4)
  SYSTEM_TABLE_AND_VIEW(false, true, false, true),
 
  @Tag(5)
  SYSTEM_VIEW(false, false, false, true),
 
  @Tag(6)
  USER_TABLE(true, true, false, false),
 
  @Tag(7)
  USER_VIEW(false, false, true, true),
 
  @Tag(8)
  USER_TABLE_AND_VIEW(true, true, true, true);
 
  final boolean anyoneMutateTable;
  final boolean systemMutateTable;
  final boolean anyoneMutateView;
  final boolean systemMutateView;
 
  private SchemaMutability(
      boolean anyoneMutateTable,
      boolean systemMutateTable,
      boolean anyoneMutateView,
      boolean systemMutateView) {
    this.anyoneMutateTable = anyoneMutateTable;
    this.systemMutateTable = systemMutateTable;
    this.anyoneMutateView = anyoneMutateView;
    this.systemMutateView = systemMutateView;
  }
 
  public boolean hasMutationCapability(MutationType type, boolean isSystemUser) {
    if (isSystemUser) {
      switch (type) {
        case VIEW:
          return systemMutateView;
        case TABLE:
          return systemMutateTable;
        default:
          throw new IllegalStateException();
      }
    }
 
    switch (type) {
      case VIEW:
        return anyoneMutateView;
      case TABLE:
        return anyoneMutateTable;
      default:
        throw new IllegalStateException();
    }
  }
 
  public static enum MutationType {
    VIEW,
    TABLE
  }
}

使用

dremio 在一些系统存储插件中就进行了明确的能力定义

  • 比如home 的
createSafe(
    catalogService,
    ns,
    HomeFileConf.create(
        HomeFileSystemStoragePlugin.HOME_PLUGIN_NAME,
        uploadsPathConfig.getUri(),
        config.getThisNode(),
        SchemaMutability.USER_TABLE,
        CatalogService.NEVER_REFRESH_POLICY,
        enableAsyncForUploads,
        scratchPathConfig.getDataCredentials()),
    deferred);
  • FileSystemConf 抽象接口定义

相关实现上都会包含能力的定义

public abstract class FileSystemConf<C extends FileSystemConf<C, P>, P extends FileSystemPlugin<C>>
    extends ConnectionConf<C, P> implements AsyncStreamConf, MutablePluginConf {
  public abstract Path getPath();
 
  public abstract boolean isImpersonationEnabled();
 
  public abstract List<Property> getProperties();
 
  public abstract String getConnection();
 
  public abstract boolean isPartitionInferenceEnabled();
 
  public abstract SchemaMutability getSchemaMutability();
 

s3 文件系统扩展的处理

@Override
public SchemaMutability getSchemaMutability() {
  return SchemaMutability.USER_TABLE;
}
  • 处理

核心是内部的判定文件系统时候支持特定的操作，比如view，table，比如dropview 的处理

public void dropView(
    NamespaceKey tableSchemaPath, ViewOptions viewOptions, SchemaConfig schemaConfig)
    throws IOException {
  if (!Boolean.getBoolean(DremioConfig.LEGACY_STORE_VIEWS_ENABLED)) {
    throw UserException.parseError()
        .message("Unable to drop view. Filesystem views are not supported.")
        .build(logger);
  } else if (!getMutability()
      .hasMutationCapability(MutationType.VIEW, schemaConfig.isSystemUser())) {
    throw UserException.parseError()
        .message("Unable to drop view. Schema [%s] is immutable for this user.", this.name)
        .build(logger);
  }
 
  createFS(schemaConfig.getUserName())
      .delete(getViewPath(tableSchemaPath.getPathComponents()), false);
}

完整的如下图

说明

目前jdbc类的存储插件（arp）的没有此配置

参考资料

dac/backend/src/main/java/com/dremio/dac/daemon/SystemStoragePluginInitializer.java
common/legacy/src/main/java/com/dremio/exec/store/dfs/SchemaMutability.java
sabot/kernel/src/main/java/com/dremio/exec/store/dfs/FileSystemConf.java

posted on 2024-04-18 11:49  荣锋亮  阅读(4)  评论(0)  编辑  收藏  举报

导航