dremio DatasetListingService 服务简单说明二
以前简单介绍过DatasetListingService，以下再说明下
创建
是通过DACDaemonModule 创建的
- 参考处理
final DatasetListingService localListing;
if (isDistributedMaster) {
// 默认实现是依赖NamespaceServiceImpl 的本地运行（master 节点）
localListing =
new DatasetListingServiceImpl(registry.provider(NamespaceService.Factory.class));
} else {
localListing = DatasetListingService.UNSUPPORTED;
}
final Provider<NodeEndpoint> searchEndPoint =
() -> {
// will return master endpoint if it's masterful mode
Optional<NodeEndpoint> serviceEndPoint =
registry
.provider(SchedulerService.class)
.get()
.getCurrentTaskOwner(SearchServiceImpl.LOCAL_TASK_LEADER_NAME);
return serviceEndPoint.orElse(null);
};
// this is the delegate service for localListing (calls start/close internally)
// DatasetListingInvoker 的实现会结合是否是master 使用不同的实现，如果是master 直接就是本地的请求，具体是NamespaceServiceImpl 实现，对于其他的会发起rpc 请求
registry.bind(
DatasetListingService.class,
new DatasetListingInvoker(
isDistributedMaster,
searchEndPoint,
registry.provider(FabricService.class),
bootstrap.getAllocator(),
localListing));
- DatasetListingInvoker rpc 处理
实际上的rpc 请求，实际上是向master 发起请求，具体的地址是master 的fabric 服务地址（45678）
一个参考处理
public SourceConfig getSource(String username, String sourcename) throws NamespaceException {
if (isMaster) { // RPC calls unless running on master
return datasetListing.getSource(username, sourcename);
}
final DLGetSourceRequest.Builder requestBuilder = DLGetSourceRequest.newBuilder();
requestBuilder.setUsername(username);
requestBuilder.setSourcename(sourcename);
final DLGetSourceResponse getSourceResponse;
try {
// 发起向master 的rpc 请求，内部实现是fabric-rpc 服务模块
getSourceResponse = newGetSourceEndpoint().send(requestBuilder.build()).getBody();
} catch (RpcException e) {
throw new RemoteNamespaceException("dataset listing failed: " + e.getMessage());
}
if (getSourceResponse.hasFailureMessage()) {
throw new RemoteNamespaceException(getSourceResponse.getFailureMessage());
}
final SourceConfig source = SourceConfig.getSchema().newMessage();
ProtobufIOUtil.mergeFrom(
getSourceResponse.getResponse().toByteArray(), source, SourceConfig.getSchema());
return source;
}
使用
对于DatasetListingService服务的使用地方还是比较多的，比如master 节点，执行节点，目前CatalogImpl，CatalogServiceImpl，PluginsManager，ContextService，SabotContext 这些实现都会使用到DatasetListingService
比如PluginsManager，是在CatalogServiceImpl 服务启动的时候创建的插件管理类，里边就会需要DatasetListingService 获取配置的数据源信息，这样不管是master 以及执行节点都能知道配置的数据源以及查询配置信息，我只简单说明下关于PluginsManager的，其他的参考源码分析下
- PluginsManager 的使用
public void start() throws NamespaceException {
// Encryption starts.
if ((sabotContext.isMaster()
|| (this.config.isMasterlessEnabled() && sabotContext.isCoordinator()))
&& this.optionManager.getOption(SOURCE_SECRETS_ENCRYPTION_ENABLED)) {
// dremio 25 开始的source 配置加密处理
migrateSourceSecrets();
}
// Encryption ends.
// Since this is run inside the system startup, no one should be able to interact with it until
// we've already
// started everything. Thus no locking is necessary.
ImmutableMap.Builder<String, CompletableFuture<SourceState>> futuresBuilder =
ImmutableMap.builder();
// DatasetListingService 服务获取配置的source， 同时通过配置创建ManagedStoragePlugin
for (SourceConfig source : datasetListing.getSources(SystemUser.SYSTEM_USERNAME)) {
ManagedStoragePlugin plugin = newPlugin(source);
// 存储插件的启动使用了startAsync 方法
futuresBuilder.put(source.getName(), plugin.startAsync());
plugins.put(c(source.getName()), plugin);
}
// 对于插件启动状态的监控（实际上就是sourcestates）
Map<String, CompletableFuture<SourceState>> futures = futuresBuilder.build();
final CompletableFuture<Void> futureWait =
CompletableFuture.allOf(futures.values().toArray(new CompletableFuture[futures.size()]));
try {
// wait STARTUP_WAIT_MILLIS or until all plugins have started/failed to start.
futureWait.get(startupWait, TimeUnit.MILLISECONDS);
} catch (Exception e) {
// ignore since we're going to evaluate individually below.
}
final StringBuilder sb = new StringBuilder();
int count = 0;
sb.append("Result of storage plugin startup: \n");
for (final ManagedStoragePlugin p : plugins.values()) {
count++;
String name = p.getName().getRoot();
final CompletableFuture<SourceState> future = futures.get(name);
Preconditions.checkNotNull(
future,
"Unexpected failure to retrieve source %s from available futures %s.",
name,
futures.keySet());
if (future.isDone()) {
try {
SourceState state = future.get();
String result =
state.getStatus() == SourceStatus.bad ? "started in bad state" : "success";
sb.append(
String.format("\t%s: %s (%dms). %s\n", name, result, p.getStartupTime(), state));
} catch (Exception ex) {
logger.error(
"Failure while starting plugin {} after {}ms.", p.getName(), p.getStartupTime(), ex);
sb.append(
String.format("\t%s: failed (%dms). %s\n", name, p.getStartupTime(), p.getState()));
p.initiateFixFailedStartTask();
}
} else {
// not finished, let's get a log entry later.
future.thenRun(Runnables.combo(new LateSourceRunnable(future, p)));
sb.append(String.format("\t%s: pending.\n", name));
}
}
// for coordinator, ensure catalog synchronization. Don't start this until the plugins manager
// is started.
if (sabotContext.getRoles().contains(Role.COORDINATOR)) {
refresher =
scheduler.schedule(
Schedule.Builder.everyMillis(CatalogServiceImpl.CATALOG_SYNC).build(),
Runnables.combo(new Refresher()));
}
if (count > 0) {
logger.info(sb.toString());
}
}
说明
DatasetListingService 从角色上来说主要是提供source 的获取能力，在master 以及执行节点中都有使用到，尤其是存储插件的启动
了解一些内部处理还是比较有用的
参考资料
dac/backend/src/main/java/com/dremio/dac/daemon/DACDaemonModule.java
services/namespace/src/main/java/com/dremio/service/listing/DatasetListingInvoker.java
services/namespace/src/main/java/com/dremio/service/listing/DatasetListingService.java
services/namespace/src/main/java/com/dremio/service/namespace/NamespaceService.java
services/fabric-rpc/src/main/java/com/dremio/services/fabric/simple/EndpointCreator.java
