cube.js prometheus 监控

这个问题是slack 中有人问到的，同时也是社区提的比较多的，基于prometheus 的监控方案是一个很不错的选择
因为cube.js 是基于express 开发的，同时官方也提供了插件扩展点的方法，我们可以直接使用现成的prometheus
express 扩展

参考配置

  • prometheus 模块
 
//const shrinkRay = require('shrink-ray-current');
const promMid = require('express-prometheus-middleware');
module.exports = function (app) {
    if (process.env.NODE_ENV == "production") {
        app.get("/", function (req, res) {
            res.send("oneservice api")
        })
    }
    // always use zip 
    //   app.use(shrinkRay())
    app.use(promMid({
        metricsPath: '/metrics',
        collectDefaultMetrics: true,
        requestDurationBuckets: [0.1, 0.5, 1, 1.5],
        requestLengthBuckets: [512, 1024, 5120, 10240, 51200, 102400],
        responseLengthBuckets: [512, 1024, 5120, 10240, 51200, 102400],
    }));
 
}
  • cube.js 使用插件
// Cube.js configuration options: https://cube.dev/docs/config
const { DremioDriver, DremioQuery } = require("@dalongrong/mydremio-driver")
const MyS3FileRepository = require("@dalongrong/cube-s3repository")
const { schemaVersion, checkAuth, homePage } = require('./libs/oneserviceapi')
const logger = require("./libs/logger")
const cubeConf  = require("./libs/cubeConf")
module.exports = {
    basePath: "/oneserviceapi/:projectid",
    schemaVersion: schemaVersion,
    dialectFactory: (dataSource) => {
        return DremioQuery
    },
    orchestratorOptions: cubeConf.orchestratorOptions,
    logger: (msg, params) => {
        if (process.env.NODE_ENV == "production") {
            logger.log({
                level: 'info',
                message: `${JSON.stringify({ msg, ...params })}`
            });
        } else {
            console.log(`${JSON.stringify({ msg, ...params })}`)
        }
    },
    telemetry: false,
    dbType: ({ dataSource } = {}) => {
        return "mydremio"
    },
    driverFactory: ({ dataSource } = {}) => {
        return new DremioDriver({})
    },
    repositoryFactory: ({ securityContext }) => {
        return new MyS3FileRepository.S3FileRepository({
            objectPrefix: securityContext.projectId,
            bucket: securityContext.tenantId,
        })
    },
    queryRewrite: (query, { securityContext }) => {
        return query;
      },
    checkAuth: checkAuth,
    contextToAppId: ({ securityContext }) => {
        return `ONESERVICEAPP_${securityContext.projectId}`
    },
    initApp: homePage
};
  • prometheus 环境准备
version: "3"
services: 
    grafana:
      image: grafana/grafana    
      ports:
      - "3000:3000"
    prometheus:
      image: prom/prometheus
      volumes:
      - "./prometheus.yml:/etc/prometheus/prometheus.yml"
      ports:
      - "9090:9090"

prometheus.yml

scrape_configs:
  - job_name: cubejs
    metrics_path: /metrics
    scrape_interval: 10s
    scrape_timeout: 10s
    static_configs:
      - targets: ['<hostip>:4000']

效果

  • 启动
yarn dev
  • 效果

 

 


监控

 

 

说明

以上是一个简单的集成，实际上还是很简单的，而且可以方便的解决我们日常系统监控的问题,注意对于集群模式的（node cluster ）监控指标会有问题，但是也是可以凑合能用

参考资料

https://www.npmjs.com/package/express-prometheus-middleware
https://www.npmjs.com/package/prometheus-api-metrics
https://grafana.com/grafana/dashboards/14565
https://grafana.com/grafana/dashboards/12230

posted on 2021-07-16 20:20  荣锋亮  阅读(221)  评论(0)  编辑  收藏  举报

导航