Prometheus之钉钉告警

要实现Prometheus通过Alertmanager发送告警到钉钉，您可以按照以下步骤进行配置：

1、创建钉钉机器人：首先，您需要在钉钉群中添加一个自定义机器人，并获取机器人的Webhook地址。创建机器人时，您可以设置安全验证方式，如加签。创建完成后，保存好Webhook地址和加签后的秘钥（如果设置了加签）。

2、安装Prometheus和Alertmanager：在您的服务器上安装Prometheus和Alertmanager。安装完成后，启动服务并设置为开机自启。

3、安装并配置Webhook-dingtalk插件：下载并安装prometheus-webhook-dingtalk插件。安装后，需要配置config.yml文件，将钉钉机器人的Webhook地址填入相应的配置项中。您可能还需要配置消息模板，以自定义告警消息的格式。

https://github.com/timonwong/prometheus-webhook-dingtalk/releases/download/v2.1.0/prometheus-webhook-dingtalk-2.1.0.linux-amd64.tar.gz
mv prometheus-webhook-dingtalk-2.1.0.linux-amd64.tar.gz /data/apps/
tar zxf prometheus-webhook-dingtalk-2.1.0.linux-amd64.tar.gz
ln -sr prometheus-webhook-dingtalk-2.1.0.linux-amd64 prometheus-webhook-dingtalk
cp prometheus-webhook-dingtalk-2.1.0.linux-amd64/config.example.yml prometheus-webhook-dingtalk/config.yml

vi  /data/apps/prometheus-webhook-dingtalk/config.yml

## Request timeout
# timeout: 5s

## Uncomment following line in order to write template from scratch (be careful!)
#no_builtin_template: true

## Customizable templates path
templates:
  - contrib/templates/legacy/template.tmpl
  - contrib/templates/legacy/template1.tmpl

## You can also override default template using `default_message`
## The following example to use the 'legacy' template from v0.3.0
#default_message:
#  title: '{{ template "legacy.title" . }}'
#  text: '{{ template "legacy.content" . }}'

## Targets, previously was known as "profiles"
targets:
  alertname:
    url: https://oapi.dingtalk.com/robot/send?access_token=6bewrerf5c2d9f5b3e21ec4b26265a2a4bb9cacedftretretr194e862912be3e8d
    # secret for signature
    secret: Ssdfsd481a12863e134c7sdfsdf0d6730333a40daf45b10b77ff7sdfsd7b6447
#  webhook2:
#    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
#  webhook_legacy:
#    url: https://oapi.dingtalk.com/robot/send?access_token=6bewrerf5c2d9f5b3e21ec4b26265a2a4bb9cacedftretretr194e862912be3e8d
    # Customize template content
    message:
      # Use legacy template
#      title: '{{ template "legacy.title" . }}'
      text: '{{ template "dingding.to.message" . }}'
  webhook_mention_all:
    url: https://oapi.dingtalk.com/robot/send?access_token=6bewrerf5c2d9f5b3e21ec4b26265a2a4bb9cacedftretretr194e862912be3e8d
    mention:
      all: true
  webhook_mention_users:
    url: https://oapi.dingtalk.com/robot/send?access_token=6bewrerf5c2d9f5b3e21ec4b26265a2a4bb9cacedftretretr194e862912be3e8d
    mention:
      mobiles: ['156xxxx8827', '189xxxx8325']

4、配置Alertmanager：在Alertmanager的配置文件alertmanager.yml中，设置告警的路由规则，指定告警接收者为钉钉机器人。您需要将Webhook-dingtalk插件的监听地址（通常是http://<服务器IP>:8060/dingtalk/webhook1/send）作为接收器的URL。

5、配置Prometheus告警规则：在Prometheus的配置文件中，定义告警规则。这些规则将决定何时触发告警。

6、测试告警：为了测试告警是否能够成功发送到钉钉，您可以故意触发一个告警条件，比如停止某个服务，然后观察钉钉群中是否收到了告警消息。

7、错误排查：如果告警没有成功发送，检查Prometheus和Alertmanager的日志文件，查看是否有错误信息。同时，确认钉钉机器人的IP白名单设置是否正确，确保服务器的IP地址被允许发送消息到钉钉。

参考资料
Prometheus+Alertmanager+webhook-dingtalk实现钉钉告警

Prometheus+Alertmanager+webhook-dingtalk实现钉钉告警

Prometheus +alertmanager配置钉钉告警