dremio prometheus监控

dremio 提供了基于prometheus 的监控，同时也提供了opentracing 的支持（缺少文档说明）
对于prometheus 的支持也是需要进行编译的

编译扩展

• clone 代码

 

git clone https://github.com/dremio-hub/dremio-prometheus-exporter

• 修改
  修改为自己的版本

    <parent>

        <groupId>com.dremio</groupId>

        <artifactId>dremio-parent</artifactId>

        <version>13.0.0-202101272034330307-20fb9275</version>

    </parent>

• 构建

mvn clean package

使用

• copy jar 以及配置
  配置 dremio-telemetry.yaml

 

auto-reload:

  enabled: True

  period: 60

  unit: SECONDS

 

metrics:

  - name: prometheus_reporter

    comment: >

      Publish metrics on prometheus

    reporter:

      type: prometheus

      port: 12543

• 参考docker 镜像

FROM dremio/dremio-oss:13.0

COPY dremio-telemetry-prometheus-13.0.0-202101272034330307-20fb9275-shaded.jar /opt/dremio/jars/

COPY --from=hengyunabc/arthas:latest /opt/arthas /opt/arthas

COPY dremio-telemetry.yaml /opt/dremio/conf/

• 运行

version: "3"

services:

  zookeeper:

    image: zookeeper

    ports:

    - "2181:2181"

    - "8080:8080"

  dremio1:

    image: dalongrong/dremio-oss:13.0

    privileged: true

    cap_add: 

    - ALL

    environment:

     - DREMIO_JAVA_SERVER_EXTRA_OPTS=-Dsaffron.default.charset=UTF-16LE -Dsaffron.default.nationalcharset=UTF-16LE -Dsaffron.default.collation.name=UTF-16LE\$en_US

    volumes: 

    - "./dremio1.conf:/opt/dremio/conf/dremio.conf"

    - "./datas/data:/opt/dremio/data"

    ports:

      - "9047:9047"

      - "31010:31010"

      - "12543:12543"

  crate:

    image: crate

    ports:

    - "4200:4200"

    - "5433:5432"

  dremio2:

    image: dalongrong/dremio-oss:13.0

    privileged: true

    cap_add: 

    - ALL

    environment:

     - DREMIO_JAVA_SERVER_EXTRA_OPTS=-Dsaffron.default.charset=UTF-16LE -Dsaffron.default.nationalcharset=UTF-16LE -Dsaffron.default.collation.name=UTF-16LE\$en_US

    volumes: 

    - "./dremio3.conf:/opt/dremio/conf/dremio.conf"

    ports:

      - "9048:9047"

      - "31011:31010"

  pg:

    image: postgres:12

    environment:

      - "POSTGRES_PASSWORD=dalong"

    ports:

      - "5432:5432"

  mongo:

    image: mongo

    ports:

    - "27017:27017"

  minio: 

    image: minio/minio

    command: server /data

    ports: 

    - "9000:9000"

    environment:

      - "MINIO_ACCESS_KEY=minio"

      - "MINIO_SECRET_KEY=minio123"

• 效果

 

 

说明

目前缺少dashboard，后边有空了制作一个dashboard

参考资料

https://docs.dremio.com/advanced-administration/telemetry.html
https://github.com/dremio-hub/dremio-prometheus-exporter
https://docs.dremio.com/advanced-administration/dremio-conf.html