dremio 的InformationSchemaCatalog 服务一
InformationSchemaCatalog 服务实现了模仿information_schema的能力，让我们可以更好的在bi 工具以及其他系统使用
接口定义
/**
* Facet of the catalog service that provides metadata with an information_schema like API.
* <p>
* The metadata provided may not satisfy the configured metadata policy requirement of various sources; the reads
* are directly from the KVStore without an inline refresh of the metadata.
*/
interface InformationSchemaCatalog {
/**
* List catalogs that satisfy the given search query.
*
* @param searchQuery search query, may be null
* @return iterator of catalogs
*/
Iterator<Catalog> listCatalogs(SearchQuery searchQuery);
/**
* List schemata that satisfy the given search query.
*
* @param searchQuery search query, may be null
* @return iterator of schemata
*/
Iterator<Schema> listSchemata(SearchQuery searchQuery);
/**
* List tables (datasets) that satisfy the given search query.
*
* @param searchQuery search query, may be null
* @return iterator of tables
*/
Iterator<Table> listTables(SearchQuery searchQuery);
/**
* List views that satisfy the given search query.
*
* @param searchQuery search query, may be null
* @return iterator of views
*/
Iterator<View> listViews(SearchQuery searchQuery);
/**
* List columns of all datasets that satisfy the given search query.
*
* @param searchQuery search query, may be null
* @return iterator of columns
*/
Iterator<TableSchema> listTableSchemata(SearchQuery searchQuery);
}
子类
从上图可以看出InformationSchemaCatalog 的直接实现是没有包含PrivilegeCatalog，catalog 的子类需要实现此接口，具体的实现InformationSchemaCatalogImpl
核心依赖NamespaceService（此服务比较重要，dremio 利用了key,value 存储解决数据状态管理）
比如数据表获取的
@Override
public Iterator<Table> listTables(SearchQuery searchQuery) {
// 基于NamespaceService进行处理
final Iterable<Map.Entry<NamespaceKey, NameSpaceContainer>> searchResults =
userNamespace.find(getCondition(searchQuery));
return StreamSupport.stream(searchResults.spliterator(), false)
.filter(IS_NOT_INTERNAL)
.filter(IS_DATASET)
.map(input -> {
final String sourceName = input.getKey().getRoot();
final TableType tableType;
if (input.getValue().getDataset().getType() == DatasetType.VIRTUAL_DATASET) {
tableType = TableType.VIEW;
} else if ("sys".equals(sourceName) || "INFORMATION_SCHEMA".equals(sourceName)) {
// 包含了sys 以及INFORMATION_SCHEMA 的处理，sys 数据dremio 内部的，INFORMATION_SCHEMA 属于标准的INFORMATION_SCHEMA
tableType = TableType.SYSTEM_TABLE;
} else {
tableType = TableType.TABLE;
}
return Table.newBuilder()
.setCatalogName(DEFAULT_CATALOG_NAME)
.setSchemaName(input.getKey().getParent().toUnescapedString())
.setTableName(input.getKey().getName())
.setTableType(tableType)
.build();
})
.iterator();
}
说明
InformationSchemaCatalog 的实现，完善了dremio 对于bi 以及jdbc 周边工具的集成，实现上不难，主要是利用了NamespaceService 服务
参考资料
sabot/kernel/src/main/java/com/dremio/exec/catalog/InformationSchemaCatalog.java
https://docs.dremio.com/software/sql-reference/information-schema/tables/
