dremio prometheus监控

dremio 提供了基于prometheus 的监控，同时也提供了opentracing 的支持（缺少文档说明）
对于prometheus 的支持也是需要进行编译的

编译扩展

  • clone 代码
 
git clone https://github.com/dremio-hub/dremio-prometheus-exporter
  • 修改
    修改为自己的版本
    <parent>
        <groupId>com.dremio</groupId>
        <artifactId>dremio-parent</artifactId>
        <version>13.0.0-202101272034330307-20fb9275</version>
    </parent>
  • 构建
mvn clean package

使用

  • copy jar 以及配置
    配置 dremio-telemetry.yaml
 
auto-reload:
  enabled: True
  period: 60
  unit: SECONDS
 
metrics:
  - name: prometheus_reporter
    comment: >
      Publish metrics on prometheus
    reporter:
      type: prometheus
      port: 12543
  • 参考docker 镜像
FROM dremio/dremio-oss:13.0
COPY dremio-telemetry-prometheus-13.0.0-202101272034330307-20fb9275-shaded.jar /opt/dremio/jars/
COPY --from=hengyunabc/arthas:latest /opt/arthas /opt/arthas
COPY dremio-telemetry.yaml /opt/dremio/conf/
  • 运行
version: "3"
services:
  zookeeper:
    image: zookeeper
    ports:
    - "2181:2181"
    - "8080:8080"
  dremio1:
    image: dalongrong/dremio-oss:13.0
    privileged: true
    cap_add: 
    - ALL
    environment:
     - DREMIO_JAVA_SERVER_EXTRA_OPTS=-Dsaffron.default.charset=UTF-16LE -Dsaffron.default.nationalcharset=UTF-16LE -Dsaffron.default.collation.name=UTF-16LE\$en_US
    volumes: 
    - "./dremio1.conf:/opt/dremio/conf/dremio.conf"
    - "./datas/data:/opt/dremio/data"
    ports:
      - "9047:9047"
      - "31010:31010"
      - "12543:12543"
  crate:
    image: crate
    ports:
    - "4200:4200"
    - "5433:5432"
  dremio2:
    image: dalongrong/dremio-oss:13.0
    privileged: true
    cap_add: 
    - ALL
    environment:
     - DREMIO_JAVA_SERVER_EXTRA_OPTS=-Dsaffron.default.charset=UTF-16LE -Dsaffron.default.nationalcharset=UTF-16LE -Dsaffron.default.collation.name=UTF-16LE\$en_US
    volumes: 
    - "./dremio3.conf:/opt/dremio/conf/dremio.conf"
    ports:
      - "9048:9047"
      - "31011:31010"
  pg:
    image: postgres:12
    environment:
      - "POSTGRES_PASSWORD=dalong"
    ports:
      - "5432:5432"
  mongo:
    image: mongo
    ports:
    - "27017:27017"
  minio: 
    image: minio/minio
    command: server /data
    ports: 
    - "9000:9000"
    environment:
      - "MINIO_ACCESS_KEY=minio"
      - "MINIO_SECRET_KEY=minio123"
  • 效果

 

 

说明

目前缺少dashboard，后边有空了制作一个dashboard

参考资料

https://docs.dremio.com/advanced-administration/telemetry.html
https://github.com/dremio-hub/dremio-prometheus-exporter
https://docs.dremio.com/advanced-administration/dremio-conf.html

posted on 2021-02-16 22:22  荣锋亮  阅读(272)  评论(0)  编辑  收藏  举报

导航