dremio backend webserver 简单说明

dremio 的webserver 是基于jetty+jersey 开发的web server 同时也和标准的dremio 服务一样实现了service 接口方便服务的启动
具体是由DACDaemon 调用触发的启动

webserver start 服务

 
 @Override
  public void start() throws Exception {
   // 使用了DremioServer 服务进行实际的启动
    server.startDremioServer(
      registry,
      config,
      credentialsServiceProvider,
      uiType,
      this::registerEndpoints
    );
  }

jetty+jersey 服务的启动

DremioServer 类中

public void startDremioServer(
    SingletonRegistry registry,
    DACConfig config,
    Provider<CredentialsService> credentialsServiceProvider,
    String uiType,
    Consumer<ServletContextHandler> servletRegistrer
  ) throws Exception {
    this.credentialsServiceProvider = credentialsServiceProvider;
    try {
      if (!embeddedJetty.isRunning()) {
       // 创建jetty 的connector 
        createConnector(config);
      // 添加handlers，包含了不少dremio 内部的服务
        addHandlers();
      }
 
      if (config.verboseAccessLog) {
        accessLogFilter = new AccessLogFilter();
       // 此处servletContextHandler 是在addHandlers 中初始化的
        servletContextHandler.addFilter(
          new FilterHolder(accessLogFilter),
          "/*",
          EnumSet.of(DispatcherType.REQUEST));
      }
 
      if (config.serveUI) {
        final String basePath = "rest/dremio_static/";
        final String markerPath = String.format("META-INF/%s.properties", uiType);
       // dremio 自己的一个servlet
        final ServletHolder fallbackServletHolder = new ServletHolder("fallback-servlet", registry.lookup(DremioServlet.class));
        addStaticPath(fallbackServletHolder, basePath, markerPath);
        servletContextHandler.addServlet(fallbackServletHolder, "/*");
      }
 
      if (servletRegistrer != null) {
        servletRegistrer.accept(servletContextHandler);
      }
 
      if (!embeddedJetty.isRunning()) {
        embeddedJetty.start();
      }
 
      setPortFromConnector();
      logger.info("Started on {}://localhost:" + port, config.webSSLEnabled() ? "https" : "http");
 
      serviceStarted = true;
    } catch (Exception ex) {
      throw new ServerErrorException(ex);
    }
  }

其他api endpoint 的注册

WebServer 类中，基于了DremioServer 提供的一个函数接口，注册了不少dremio 的api 接口
使用了jetty 提供的ServletHolder 动态的servlet 注册接口，同时利用了ResourceConfig

 
protected void registerEndpoints(ServletContextHandler servletContextHandler) {
  // security header filters
  SecurityHeadersFilter securityHeadersFilter = new SecurityHeadersFilter(registry.provider(OptionManager.class));
  servletContextHandler.addFilter(new FilterHolder(securityHeadersFilter), "/*", EnumSet.of(DispatcherType.REQUEST));
 
  // Generic Response Headers filter for api responses
  servletContextHandler.addFilter(GenericResponseHeadersFilter.class.getName(), "/apiv2/*", EnumSet.of(DispatcherType.REQUEST));
  servletContextHandler.addFilter(GenericResponseHeadersFilter.class.getName(), "/api/*", EnumSet.of(DispatcherType.REQUEST));
 
  // add the font mime type.
  final MimeTypes mimeTypes = servletContextHandler.getMimeTypes();
  mimeTypes.addMimeMapping("woff2", "application/font-woff2; charset=utf-8");
  servletContextHandler.setMimeTypes(mimeTypes);
 
  // WebSocket API
  final SocketServlet servlet = new SocketServlet(registry.lookup(JobsService.class), registry.lookup(TokenManager.class),
    registry.provider(OptionManager.class));
  final ServletHolder wsHolder = new ServletHolder(servlet);
  wsHolder.setInitOrder(1);
  servletContextHandler.addServlet(wsHolder, "/apiv2/socket");
 
  // Rest API
  ResourceConfig restServer = restServerProvider.get();
 
  restServer.property(RestServerV2.ERROR_STACKTRACE_ENABLE, config.sendStackTraceToClient);
  restServer.property(RestServerV2.TEST_API_ENABLE, config.allowTestApis);
  restServer.property(RestServerV2.FIRST_TIME_API_ENABLE, isInternalUS);
 
  restServer.register(dremioBinder);
  restServer.register((DynamicFeature) (resourceInfo, context) -> context.register(DremioServer.TracingFilter.class));
 
  final ServletHolder restHolder = new ServletHolder(new ServletContainer(restServer));
  restHolder.setInitOrder(2);
  servletContextHandler.addServlet(restHolder, "/apiv2/*");
 
  // Public API
  ResourceConfig apiServer = apiServerProvider.get();
  apiServer.register(dremioBinder);
  apiServer.register((DynamicFeature) (resourceInfo, context) -> context.register(DremioServer.TracingFilter.class));
 
  final ServletHolder apiHolder = new ServletHolder(new ServletContainer(apiServer));
  apiHolder.setInitOrder(3);
  servletContextHandler.addServlet(apiHolder, "/api/v3/*");
  }

dremio 自己的实现

包含了APIServer以及RestServerV2

  • 参考类图

基于了ResourceConfig 进行了动态扩展

 

 


有一个灵活的设计是，dremio 基于了注解实现动态注册，比如api 以及rest 都包含了自己的注解扩展

 
protected void init(ScanResult result) {
    // FILTERS
    register(JSONPrettyPrintFilter.class);
    register(MediaTypeFilter.class);
 
    // RESOURCES
   // 基于APIResource 注解，通过调用register 动态注册接口服务
    for (Class<?> resource : result.getAnnotatedClasses(APIResource.class)) {
      register(resource);
    }
 
    // FEATURES
    register(DACAuthFilterFeature.class);
    register(DACJacksonJaxbJsonFeature.class);
    register(DACExceptionMapperFeature.class);
 
    // EXCEPTION MAPPERS
    register(JsonParseExceptionMapper.class);
    register(JsonMappingExceptionMapper.class);
 
    // PROPERTIES
    property(ServerProperties.RESPONSE_SET_STATUS_OVER_SEND_ERROR, "true");
    property(RestServerV2.TEST_API_ENABLE, "true");
 
    final String disableMoxy = PropertiesHelper.getPropertyNameForRuntime(CommonProperties.MOXY_JSON_FEATURE_DISABLE,
      getConfiguration().getRuntimeType());
    property(disableMoxy, true);
  }

说明

以上是一个简单的dremio webserver 服务介绍，基于此可以了解dremio api的运行机制，可以更好的进行扩展，实际上dremio 的api 部分还包含了
认证、日志以及一些其他服务，可以通过阅读源码查看学习

参考资料

dac/backend/src/main/java/com/dremio/dac/server/WebServer.java
dac/backend/src/main/java/com/dremio/dac/server/DremioServer.java
dac/backend/src/main/java/com/dremio/dac/server/DremioServlet.java
dac/backend/src/main/java/com/dremio/dac/daemon/DACDaemonModule.java

posted on 2023-01-20 22:43  荣锋亮  阅读(41)  评论(0)  编辑  收藏  举报

导航