blackbox+prometheus实现黑盒监控

目录

• blackbox+prometheus实现黑盒监控
  • 一、部署blackbox_exporter
  • 二、在prometheus中添加监控项
  • 三、常用监控项
  • 告警规则参考
  • blackbox_exporter配置文件详解

blackbox+prometheus实现黑盒监控

blackbox_exporter可以通过 HTTP、HTTPS、DNS、TCP、ICMP、gRPC对目标进行黑盒探测

一、部署blackbox_exporter

• 可使用二进制文件直接部署，或使用容器部署
• 若使用容器部署，建议将配置文件(/etc/blackbox/blackbox.yml)挂载出来;暴露服务端口9115
• 配置文件详解见文末

二、在prometheus中添加监控项

在prometheus.yml文件中添加如下配置

  - job_name: 'blackbox'
    metrics_path: /probe
    params:
      # blackbox_exporter中定义的模块
      module: [http_2xx]  # Look for a HTTP 200 response.
    static_configs:
      # 监控的目标
      - targets:
        - http://prometheus.io    # Target to probe with http.
          labels:
            app: cmoon
            dev: prod
        - https://prometheus.io   # Target to probe with https.
          labels:
            app: cmoon-test
        - http://example.com:8080 # Target to probe with http on port 8080.
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        # blackbox_exporter服务的连接地址
        replacement: 192.168.2.23:9115  # The blackbox exporter's real hostname:port.

三、常用监控项

监控项	详解
probe_http_status_code	HTTP状态码
probe_ssl_earliest_cert_expiry	证书到期时间
probe_success	连通性(==1为正常)

告警规则参考

告警规则参考

blackbox_exporter配置文件详解

# 定义各监控模块
modules:
  # http模块，get请求
  http_2xx:
    prober: http
  # http模块，get请求，添加了返回值判断
  http:
    valid_status_codes: []
    method: GET
    # 如果get的url地址返回的正文中有"fail",那么就会失败,则probe_success值为0
    fail_if_body_matches_regexp:
      - "#fail#"
    # 如果get的url地址返回的正文中没有"success",那么就会失败,则probe_success值为0
    fail_if_body_not_matches_regexp:
      - "#SUCCESS#"    
  # http模块，post请求
  http_post_2xx:
    prober: http
    http:
      method: POST
  # http模块，带有request body的post请求
  http_post_2xx_request:
    prober: http
    timeout: 5s
    http:
      method: POST
      headers:
        Content-Type: application/json
      body: '{"app": "c-moon"}'
  tcp_connect:
    prober: tcp
  pop3s_banner:
    prober: tcp
    tcp:
      query_response:
      - expect: "^+OK"
      tls: true
      tls_config:
        insecure_skip_verify: false
  ssh_banner:
    prober: tcp
    tcp:
      query_response:
      - expect: "^SSH-2.0-"
  irc_banner:
    prober: tcp
    tcp:
      query_response:
      - send: "NICK prober"
      - send: "USER prober prober prober :prober"
      - expect: "PING :([^ ]+)"
        send: "PONG ${1}"
      - expect: "^:[^ ]+ 001"
  icmp:
    prober: icmp
    timeout: 5s
    icmp:
      preferred_ip_protocol: "ip4"