elastalert搭建

elastalert搭建过程

在服务器上搭建python3.6环境

• 编译安装

# wget http://mirrors.sohu.com/python/3.6.0/Python-3.6.0.tgz

## 安装编译依赖包
# yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gcc make

## 编译安装
# tar -zxvf  Python-3.6.0
# cd Python-3.6.0
# ./configure --prefix=/usr/local/python36　　　　　　//编译存放路径至“/usr/local/python36”
# make && make install

• yum 安装

## 安装EPEL和IUS软件源
# yum install epel-release -y
# yum install https://centos7.iuscommunity.org/ius-release.rpm -y

## 安装Python3.6
# yum install python36u -y
# yum install python36u-devel -y

## 创建python3连接符
# ln -s /bin/python3.6 /bin/python3

## 安装pip3
# yum install python36u-pip -y

## 创建pip3链接符
# ln -s /bin/pip3.6 /bin/pip3

无法访问互联网情况下如何安装模块

• 使用上面的 编译安装 在内网主机和自建的虚拟机上安装python

• 自建虚拟机上下载模块

## 下载单个安装包
## pip3 download pakeage_namq -d /path/file/ -i https://pypi.tuna.tsinghua.edu.cn/simple/
# pip3 download xlwt pymysql -d ./pip_pakeage/ -i https://pypi.tuna.tsinghua.edu.cn/simple/

## 根据requirements下载安装包
## pip3 download -r requiremetns.txt -d /path/file/ -i https://pypi.tuna.tsinghua.edu.cn/simple/
# pip3 download -r requirements.txt -d ./pip_pakeage/ -i https://pypi.tuna.tsinghua.edu.cn/simple/

• 内网主机上安装模块

## 安装单个模块包
## pip3 install file:///path/filename
# pip3 install file:///tmp/pip_pakeage/xlwt-1.3.0-py2.py3-none-any.whl

## 安装 requirements 下载安装包
# pip3 install --no-index --find-links=/tmp/pip_pakeage/ -r /data/filename/requirements.txt

elastalert下载安装

在https://github.com/Yelp/elastalert上下载源码

# cd /opt/
# git clone https://github.com/Yelp/elastalert.git
# cd elastalert/
# python3 ./setup.py install --dry-run  ## 测试是否能直接安装成功
# python3 ./setup.py install

elastalert配置方法

• 配置config.yaml

# cp config.yaml.example config.yaml
# vim config.yaml
----------------------------------------------
rules_folder: /opt/elastalert/rules
run_every:
  seconds: 60
buffer_time:
  minutes: 3
es_host: 172.16.1.1
es_port: 9200
writeback_index: elastalert_status
writeback_alias: elastalert_alerts
alert_time_limit:
  days: 2

• 配置rule规则

# mkdir rules
# cp example_rules/example_frequency.yaml /opt/elastalert/rules/frequency.yaml
# vim /opt/elastalert/rules/frequency.yaml
-------------------------------------------------------------
name: API not 200
index: sg-access-*
type: frequency
num_events: 20
timeframe:
  minutes: 1
filter:
- query:
    query_string:
      query: "NOT statusCode: 200"
- query:
    query_string:
      query: "NOT statusCode: 302"
- query:
    query_string:
      query: "NOT directBackServer: 127.0.0.1"

alert:
  - command

command: ["python3", /opt/elastalert/weixin.py", "生产环境报警，报警:", "接口{orgPathName} 出现状态码{statusCode}频率高！","服务 IP: {directBackServer}; 服务端口：{port}"]

其他配置方式参考官网：https://elastalert.readthedocs.io/en/latest/

编写报警脚本

#!/usr/bin/env python3
# _*_coding:utf-8 _*_

import urllib.request
import json
import sys
import simplejson


def gettoken(corpid, corpsecret):
    gettoken_url = 'https://qyapi.weixin.qq.com/cgi-bin/gettoken?corpid=' + corpid + '&corpsecret=' +corpsecret
    print(gettoken_url)
    try:
        token_file = urllib.request.urlopen(gettoken_url)
    except urllib.request.HTTPError as e:
        print(e.code)
        print(e.read().decode("utf8"))
        sys.exit()
    token_data = token_file.read().decode('utf-8')
    token_json = json.loads(token_data)
    token_json.keys()
    token = token_json['access_token']
    return token


def senddata(access_token, subject, content, server):
    send_url = 'https://qyapi.weixin.qq.com/cgi-bin/message/send?access_token=' + access_token
    send_values = {
        "touser": "@all",          # 企业号中的用户帐号，在zabbix用户 Media中配置，如果配置不正常，将按部门发送。
        "toparty": "ID",           # 企业号中的部门id。
        "msgtype": "text",         # 消息类型。
        "agentid": "1000001",      # 企业号中的应用id。
        "text": {
            "content": str(subject + '\n\n' + content + '\n' + server)
        },
        "safe": "0",
    }
    send_data = simplejson.dumps(send_values, ensure_ascii=False).encode('utf-8')
    send_request = urllib.request.Request(send_url, send_data)
    response = json.loads(urllib.request.urlopen(send_request).read())
    print(str(response))


def senddata_report(subject, content, server):
    send_url = 'https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ'
    send_values = {
        "msgtype": "text",
        "text": {
            "content": str(subject + '\n\n' + content + '\n' + server)
        }
    }
    send_data = simplejson.dumps(send_values, ensure_ascii=False).encode('utf-8')
    send_request = urllib.request.Request(send_url, send_data)
    response = json.loads(urllib.request.urlopen(send_request).read())
    print(str(response))


if __name__ == '__main__':
    try:
        subject = str(sys.argv[1])
        content = str(sys.argv[2])
        server = str(sys.argv[3])
    except IndexError:
        print('需要传3个参数')
    else:
        corpid = 'XXXXXXXXXXXXXXXXXXXXXXXX'   # 企业号的标识
        corpsecret = 'YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY'    # 管理组凭证密钥
        accesstoken = gettoken(corpid, corpsecret)
        # senddata(accesstoken, subject, content, server)
        senddata_report(subject, content, server)

启动服务

• 调用接口向ES中创建索引

# elastalert-create-index elastalert-test-rule --config /opt/elastalert/config.yaml  /opt/elastalert/rules/frequency.yaml

• 启动服务前测试服务配置正常

# elastalert-test-rule --config /opt/elastalert/config.yaml  /opt/elastalert/rules/frequency.yaml

• 启动服务前测试报警功能正常

# elastalert-test-rule --config /opt/elastalert/config.yaml  /opt/elastalert/rules/frequency.yaml --alert

• 后台启动服务

nohup python -m elastalert.elastalert  --config /opt/elastalert/config.yaml --rule /opt/elastalert/rules/frequency.yaml >> /opt/elastalert/elastalert.log 2>&1 &