zeebe 集成elasticsearch exporter && 添加operate

zeebe 的operate是一个功能比较强大的管理工具，比simple-monitor 有好多方面上的改进

• 安全，支持用户账户的登陆
• 界面更友好，界面比较符合开团队工作流引擎的界面
• 系统监控更加强大，可以为我们提供比较全的系统指标

环境准备

还是老样子使用docker-compose 运行

• docker-compose 文件

version: "3"
services:
  db:
    image: oscarfonts/h2
    container_name: zeebe_db
    ports:
      - "1521:1521"
      - "81:81"
  operate:
    image: camunda/operate:latest
    ports: 
    - "8089:8080"
    volumes: 
    - "./application.yml:/usr/local/operate/config/application.yml"
  monitor:
    image: camunda/zeebe-simple-monitor
    environment:
      - spring.datasource.url=jdbc:h2:tcp://db:1521/zeebe-monitor
      - io.zeebe.monitor.connectionString=app:26500
      - io.zeebe.monitor.hazelcast.connection=app:5701
    ports:
    - "8080:8080"
  app:
    image: camunda/zeebe
    volumes:
      - ./zeebe-simple-monitor-exporter-0.13.0.jar:/usr/local/zeebe/lib/zeebe-simple-monitor-exporter-0.13.0.jar
      - ./zeebe-hazelcast-exporter-0.2.0.jar:/usr/local/zeebe/lib/zeebe-hazelcast-exporter-0.2.0.jar
      - ./zeebe.cfg.toml:/usr/local/zeebe/conf/zeebe.cfg.toml
    ports:
    - "26500:26500"
    - "26501:26501"
    - "26502:26502"
    - "26503:26503"
    - "26504:26504"
    - "5701:5701"
  elasticsearch:
    image: elasticsearch:6.7.1 
    container_name: elasticsearch
    environment:
    - "discovery.type=single-node"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - 9200:9200
      - 9300:9300 # required for Performance Analyzer

• 说明
  注意es 版本，目前测试6.7 是可以的，6.4 有问题（主要是对于operate的）
  application.yaml 配置

camunda.operate:
  # ELS instance to store Operate data
  elasticsearch:
    # Cluster name
    clusterName: docker-cluster
    # Host
    host: elasticsearch
    # Transport port
    port: 9200
  # Zeebe instance
  zeebe:
    # Broker contact point
    brokerContactPoint: app:26500
  # ELS instance to export Zeebe data to
  zeebeElasticsearch:
    # Cluster name
    clusterName: docker-cluster
    # Host
    host: elasticsearch
    # Transport port
    port: 9200
    # Index prefix, configured in Zeebe Elasticsearch exporter
    prefix: zeebe-record
logging:
  level:
    ROOT: INFO
    org.camunda.operate: DEBUG

#Spring Boot Actuator endpoints to be exposed
management.endpoints.web.exposure.include: health,info,conditions,configprops,prometheus

exporter 配置 zeebe.cfg.toml ,simple-monitor 以及operate 需要

[[exporters]]
id = "simple-monitor"
className = "io.zeebe.monitor.SimpleMonitorExporter"
[exporters.args]
  jdbcUrl = "jdbc:h2:tcp://db:1521/zeebe-monitor"
  driverName = "org.h2.Driver"
  userName = "sa"
  password = ""


[[exporters]]
 id = "hazelcast"
 className = "io.zeebe.hazelcast.exporter.HazelcastExporter"

[[exporters]]
id = "elasticsearch"
className = "io.zeebe.exporter.ElasticsearchExporter"
[exporters.args]
  url = "http://elasticsearch:9200"
  # The driver name of the jdbc driver implementation. Make sure that the implementation is
  # available in the exporter/broker classpath (add it to the broker lib folder).
  # The name is used to load the driver implementation like this
  # Class.forName(configuration.driverName);
  # To configure the amount of records, which has to be reached before the records are exported to
  # the database. Only counts the records which are in the end actually exported.
  #
  # batchSize = 100;

  # To configure the time in milliseconds, when the batch should be executed regardless whether the
  # batch size was reached or not.
  #
  #If the value is less then one, then no timer will be scheduled.
  #
  #batchTimerMilli = 1000

#id = "elasticsearch"
#className = "io.zeebe.exporter.ElasticsearchExporter"
#
# [exporters.args]
# url = "http://localhost:9200"
#
 [exporters.args.bulk]
 delay = 5
 size = 1_000

 [exporters.args.index]
  prefix = "zeebe-record"
  createTemplate = false
  command = false
  event = true
  rejection = false
  deployment = true
  incident = true
  job = true
  message = false
  messageSubscription = false
  raft = false
  workflowInstance = true
  workflowInstanceSubscription = false

启动&&测试

• 启动

docker-compose up -d

• 效果
  simple-monitor
  
  es 信息
  
  operate 界面：默认账户demo demo
  
• 使用
  添加一个流程，github 代码中包含一个简单的，我使用了simple monitor deploy
  
  operate 界面
  
  
• 问题
  目前operate 有一些异常日志, 解决的办法是注释zeebe.cfg.toml 关于[exporters.args.index] 的配置

 2019-04-30 02:34:04.263 INFO 6 --- [ Thread-3] o.c.o.z.ZeebeESImporter : Elasticsearch index for ValueType DEPLOYMENT was not found, alias zeebe-record-deployment. Skipping.
operate_1 | 2019-04-30 02:34:04.265 DEBUG 6 --- [ Thread-3] o.c.o.z.ZeebeESImporter : Latest loaded position for alias [zeebe-record-workflow-instance] and partitionId [0]: 0
operate_1 | 2019-04-30 02:34:04.266 INFO 6 --- [ Thread-3] o.c.o.z.ZeebeESImporter : Elasticsearch index for ValueType WORKFLOW_INSTANCE was not found, alias zeebe-record-workflow-instance. Skipping.
operate_1 | 2019-04-30 02:34:04.267 DEBUG 6 --- [ Thread-3] o.c.o.z.ZeebeESImporter : Latest loaded position for alias [zeebe-record-job] and partitionId [0]: 0
operate_1 | 2019-04-30 02:34:04.267 INFO 6 --- [ Thread-3] o.c.o.z.ZeebeESImporter : Elasticsearch index for ValueType JOB was not found, alias zeebe-record-job. Skipping.
operate_1 | 2019-04-30 02:34:04.268 DEBUG 6 --- [ Thread-3] o.c.o.z.ZeebeESImporter : Latest loaded position for alias [zeebe-record-incident] and partitionId [0]: 0
operate_1 | 2019-04-30 02:34:04.269 INFO 6 --- [ Thread-3] o.c.o.z.ZeebeESImporter : Elasticsearch index for ValueType INCIDENT was not found, alias zeebe-record-incident. Skipping.
operate_1 | 2019-04-30 02:34:04.270 DEBUG 6 --- [ Thread-3] o.c.o.z.ZeebeESImporter : Latest loaded position for alias [zeebe-record-variable] and partitionId [0]: 0
operate_1 | 2019-04-30 02:34:04.271 INFO 6 --- [ Thread-3] o.c.o.z.ZeebeESImporter : Elasticsearch index for ValueType VARIABLE was not found, alias zeebe-record-variable. Skipping.
operate_1 | 2019-04-30 02:34:05.623 DEBUG 6 --- [ Thread-2] o.c.o.e.w.BatchOperationWriter : 0 operations locked
operate_1 | 2019-04-30 02:34:0

说明

operate 是很强大，界面也比较友好，但是目前不是开源的，所以分析问题，可能有点费事。

参考资料

https://github.com/zeebe-io/docker-camunda-operate
https://github.com/rongfengliang/camunda-zeebe-es-exporter-demo-no-auth