Alertmanager简介及机制_告警抑制
测试目标：客户端node_explore停止，抑制了blackbox探测邮件告警功能。
客户端安装blackbox_exporter服务
[root@promethus blackbox_exporter]# ll
-rwxr-xr-x 1 3434 3434 14925964 3月 15 21:32 blackbox_exporter
-rw-r--r-- 1 3434 3434 629 3月 15 21:42 blackbox.yml
配置好blackbox.yml文件
[root@promethus blackbox_exporter]# vim blackbox.yml
modules:
http_2xx:
prober: http
http_post_2xx:
prober: http
http:
method: POST
tcp_connect:
prober: tcp
pop3s_banner:
prober: tcp
tcp:
query_response:
- expect: "^+OK"
tls: true
tls_config:
insecure_skip_verify: false
icmp:
prober: icmp
服务端主配置文件。
- job_name: 'promeclient'
scrape_interval: 15s
file_sd_configs:
- files:
- /usr/local/prometheus/files/client.json
relabel_configs:
- source_labels: [__address__]
target_label: host_ip
- job_name: 'HTTP/TCP Health Check'
scrape_interval: 15s
metrics_path: /probe
params:
module: [http_2xx]
file_sd_configs:
- files:
- /usr/local/prometheus/files/service_blackbox.json
relabel_configs:
- source_labels: [__param_target]
target_label: instance
- target_label: scraper
replacement: blackbox-exporter
- target_label: __address__
replacement: 192.168.19.128:9115
- target_label: env
replacement: prod
rule_files:
- "/usr/local/prometheus/host.rule"
- "/usr/local/prometheus/blackbox.rules"
配置文件
[root@promethus files]# vim client.json
[
{
"targets": [
"hongk:9100"
],
"labels": {
"project": "HK测试服务器",
"app": "HK test",
"scraper": "telegraf",
"region": "SZD"
}
}
]
[root@promethus files]# vim service_blackbox.json
[
{
"targets": [
"http://192.168.19.128:9093",
"http://192.168.19.128:3000"
],
"labels": {
"app": "grafana_server/alertmanager",
"project": "pass-regisrty",
"scraper": "blackbox-export",
"region": "SZD",
"module": "http_2xx"
}
}
]
测试方式，正常情况下运行，先使得blackbox挂掉，然后node_explore挂掉。测试结果：当blackbox挂掉，正常显示blackbox相关url异常，当node_explore挂掉后，邮件只显示node_explore挂掉。达到了预期效果。
告警部分配置。
inhibit_rules:
- source_match:
job: 'promeclient'
target_match:
job: 'HTTP/TCP Health Check'
equal: ['region']
如上截图显示。
