dremio SchemaMutability 简单说明

dremio SchemaMutability 属于一个枚举，定义了schema 的可变性能力()

参考定义

public enum SchemaMutability {

  @Tag(1)

  ALL(true, true, true, true),

 

  @Tag(2)

  NONE(false, false, false, false),

 

  @Tag(3)

  SYSTEM_TABLE(false, true, false, false),

 

  @Tag(4)

  SYSTEM_TABLE_AND_VIEW(false, true, false, true),

 

  @Tag(5)

  SYSTEM_VIEW(false, false, false, true),

 

  @Tag(6)

  USER_TABLE(true, true, false, false),

 

  @Tag(7)

  USER_VIEW(false, false, true, true),

 

  @Tag(8)

  USER_TABLE_AND_VIEW(true, true, true, true);

 

  final boolean anyoneMutateTable;

  final boolean systemMutateTable;

  final boolean anyoneMutateView;

  final boolean systemMutateView;

 

  private SchemaMutability(

      boolean anyoneMutateTable,

      boolean systemMutateTable,

      boolean anyoneMutateView,

      boolean systemMutateView) {

    this.anyoneMutateTable = anyoneMutateTable;

    this.systemMutateTable = systemMutateTable;

    this.anyoneMutateView = anyoneMutateView;

    this.systemMutateView = systemMutateView;

  }

 

  public boolean hasMutationCapability(MutationType type, boolean isSystemUser) {

    if (isSystemUser) {

      switch (type) {

        case VIEW:

          return systemMutateView;

        case TABLE:

          return systemMutateTable;

        default:

          throw new IllegalStateException();

      }

    }

 

    switch (type) {

      case VIEW:

        return anyoneMutateView;

      case TABLE:

        return anyoneMutateTable;

      default:

        throw new IllegalStateException();

    }

  }

 

  public static enum MutationType {

    VIEW,

    TABLE

  }

}

使用

dremio 在一些系统存储插件中就进行了明确的能力定义

• 比如home 的

createSafe(

    catalogService,

    ns,

    HomeFileConf.create(

        HomeFileSystemStoragePlugin.HOME_PLUGIN_NAME,

        uploadsPathConfig.getUri(),

        config.getThisNode(),

        SchemaMutability.USER_TABLE,

        CatalogService.NEVER_REFRESH_POLICY,

        enableAsyncForUploads,

        scratchPathConfig.getDataCredentials()),

    deferred);

• FileSystemConf 抽象接口定义

相关实现上都会包含能力的定义

public abstract class FileSystemConf<C extends FileSystemConf<C, P>, P extends FileSystemPlugin<C>>

    extends ConnectionConf<C, P> implements AsyncStreamConf, MutablePluginConf {

  public abstract Path getPath();

 

  public abstract boolean isImpersonationEnabled();

 

  public abstract List<Property> getProperties();

 

  public abstract String getConnection();

 

  public abstract boolean isPartitionInferenceEnabled();

 

  public abstract SchemaMutability getSchemaMutability();

 

s3 文件系统扩展的处理

@Override

public SchemaMutability getSchemaMutability() {

  return SchemaMutability.USER_TABLE;

}

• 处理

核心是内部的判定文件系统时候支持特定的操作，比如view，table，比如dropview 的处理

public void dropView(

    NamespaceKey tableSchemaPath, ViewOptions viewOptions, SchemaConfig schemaConfig)

    throws IOException {

  if (!Boolean.getBoolean(DremioConfig.LEGACY_STORE_VIEWS_ENABLED)) {

    throw UserException.parseError()

        .message("Unable to drop view. Filesystem views are not supported.")

        .build(logger);

  } else if (!getMutability()

      .hasMutationCapability(MutationType.VIEW, schemaConfig.isSystemUser())) {

    throw UserException.parseError()

        .message("Unable to drop view. Schema [%s] is immutable for this user.", this.name)

        .build(logger);

  }

 

  createFS(schemaConfig.getUserName())

      .delete(getViewPath(tableSchemaPath.getPathComponents()), false);

}

完整的如下图

说明

目前jdbc类的存储插件（arp）的没有此配置

参考资料

dac/backend/src/main/java/com/dremio/dac/daemon/SystemStoragePluginInitializer.java
common/legacy/src/main/java/com/dremio/exec/store/dfs/SchemaMutability.java
sabot/kernel/src/main/java/com/dremio/exec/store/dfs/FileSystemConf.java