监控工具 - Prometheus 的 Alertmanager

---

Alertmanager

• Docs https://prometheus.io/docs/alerting/
• Download https://prometheus.io/download/#alertmanager
• Routing tree editor https://www.prometheus.io/webtools/alerting/routing-tree-editor/
• Configuration https://prometheus.io/docs/alerting/latest/configuration/
• GitHub https://github.com/prometheus/alertmanager
• Docker https://hub.docker.com/r/prom/alertmanager/
• Command Help alertmanager -h

---

安装配置

环境信息

# uname -a
Linux node100 5.14.0-427.31.1.el9_4.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Aug 14 16:15:25 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
# cat /etc/system-release
Rocky Linux release 9.4 (Blue Onyx)
# 

# 二进制方式
tar -xvzf alertmanager-0.27.0.linux-amd64.tar.gz -C /opt
chown -R prometheus:prometheus /opt/alertmanager-0.27.0.linux-amd64
ln -sv /opt/alertmanager-0.27.0.linux-amd64 /opt/alertmanager
chmod -R 777 /opt/alertmanager
/opt/alertmanager/alertmanager --config.file=alertmanager.yml

# Docker方式
[root@node100 ~]# docker pull docker.m.daocloud.io/prom/alertmanager:v0.27.0
v0.27.0: Pulling from prom/alertmanager
9fa9226be034: Pull complete 
1617e25568b2: Pull complete 
e769c9462d1b: Pull complete 
697c2f9afe54: Pull complete 
27aac892fbe9: Pull complete 
5c5751833485: Pull complete 
4f4fb700ef54: Pull complete 
Digest: sha256:e13b6ed5cb929eeaee733479dce55e10eb3bc2e9c4586c705a4e8da41e5eacf5
Status: Downloaded newer image for docker.m.daocloud.io/prom/alertmanager:v0.27.0
docker.m.daocloud.io/prom/alertmanager:v0.27.0
[root@node100 ~]# 
[root@node100 ~]# mkdir -p /opt/alertmanager
[root@node100 ~]# docker run -itd --name alertmanager \
-p 9093:9093 \
-v /opt/alertmanager:/alertmanager \
docker.m.daocloud.io/prom/alertmanager:v0.27.0
4167b332cdfb8ac548c63e953f6655bab5719efa641c4ec15e076ff83f070f12
[root@node100 ~]# 
[root@node100 ~]# docker ps 
CONTAINER ID   IMAGE                                            COMMAND                  CREATED         STATUS         PORTS                                       NAMES
4167b332cdfb   docker.m.daocloud.io/prom/alertmanager:v0.27.0   "/bin/alertmanager -…"   4 seconds ago   Up 3 seconds   0.0.0.0:9093->9093/tcp, :::9093->9093/tcp   alertmanager
[root@node100 ~]# 
[root@node100 ~]# docker exec -it alertmanager sh
/alertmanager $ 
/alertmanager $ ls /bin/amtool 
/bin/amtool
/alertmanager $ /bin/amtool check-config /etc/alertmanager/alertmanager.yml 
Checking '/etc/alertmanager/alertmanager.yml'  SUCCESS
Found:
 - global config
 - route
 - 1 inhibit rules
 - 1 receivers
 - 0 templates
/alertmanager $ 
/alertmanager $ ls  /bin/alertmanager
/bin/alertmanager
/alertmanager $ 
/alertmanager $ cat /etc/alertmanager/alertmanager.yml 
route:
  group_by: ['alertname']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 1h
  receiver: 'web.hook'
receivers:
  - name: 'web.hook'
    webhook_configs:
      - url: 'http://127.0.0.1:5001/'
inhibit_rules:
  - source_match:
      severity: 'critical'
    target_match:
      severity: 'warning'
    equal: ['alertname', 'dev', 'instance']
/alertmanager $ 
/alertmanager $ exit
[root@node100 ~]# 

---

主要页面

• 主页面 http://localhost:9093/
• 运行状态 http://localhost:9093/#/status
• 告警 http://localhost:9093/#/alerts
• 静默 http://localhost:9093/#/silences
• 设置 http://localhost:9093/#/settings

---

与Prometheus关联

1. 在Prometheus服务器配置文件中添加相关配置并验证
2. 热加载新配置curl -X POST http://192.168.16.200:9090/-/reload
3. 页面查看 http://192.168.16.200:9090/targets 节点状态正常为UP

[root@node200 ~]# vim /opt/prometheus/prometheus.yml 
[root@node200 ~]# cat /opt/prometheus/prometheus.yml |grep -v "#"
global:

alerting:
  alertmanagers:
    - static_configs:
        - targets:
          - 192.168.16.100:9093

rule_files:

scrape_configs:
  - job_name: "prometheus"


    static_configs:
      - targets: ["192.168.16.200:9090"]
  - job_name: "alertmanager"
    static_configs:
      - targets: ["192.168.16.100:9093"]                
[root@node200 ~]#
[root@node200 ~]# /opt/prometheus/promtool check config /opt/prometheus/prometheus.yml 
Checking /opt/prometheus/prometheus.yml
 SUCCESS: /opt/prometheus/prometheus.yml is valid prometheus config file syntax
[root@node200 ~]#  
[root@node200 ~]# curl -X POST http://192.168.16.200:9090/-/reload
[root@node200 ~]# 

---

amtool

A cli tool for interacting with the Alertmanager API. It is bundled with all releases of Alertmanager.

• 文档 https://github.com/prometheus/alertmanager?tab=readme-ov-file#amtool
• 目录 alertmanager/cmd/amtool/
• 命令帮助 amtool -h

---