应用指标数据采集并录入Elasticsearch仓库 - 运维笔记
为了进行数字化IT治理,需要对一些应用进程相关指标进行采集并入库。收集到的应用指标数据最好要进行ES入仓,入到Kafka里面,并通过Kibana可视化展示。
下面是应用指标数据进行ES入仓的请求说明
测试区接口说明: 访问链接:http://192.168.10.10:10222/haha/heiheiAPI/test_1_kafka 请求header:Content-Type:application/json 请求方法:POST 请求参数,token中填入示例中的128位token即可,data的值是一个数组,数组中的每个元素是一个json对象(对应会插入ES中的1个document),给出具体信息: { "token": "bd1df2b11111111112222222222222222333333333333333333yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy", "data": [ { "type": "英文字符,最长不超过10位", "desc": "指标的中文描述, 可为空", "@timestamp": "时间戳,以标准unix时间戳为准,下面会给出shell和python的生成示例", "appname": "应用系统编码,以aiops上的信息为准", "hostname": "主机名", "value": "采集的指标值, 由于各系统需求不一致, 此处统一使用string类型, 否则可能会造成mapping冲突, 但某些功能需要int/float,再讨论", "checker": "发送数据的脚本路径或IP, 可为空" } ] } ============================================================================================== # python生成时间戳示例 import datetime timestamp = datetime.datetime.strftime(datetime.datetime.now(), '%s') ============================================================================================== # python请求示例 import urllib2 # 这里是python2,根据各人使用有不同,总之是构建http request import hashlib import json url = 'http://192.168.10.10:10222/haha/heiheiAPI/test_1_kafka' data = { 'token': 'bd1df2b11111111112222222222222222333333333333333333yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy', 'data': [ {} ] } headers = {'Content-Type': 'application/json'} req = urllib2.Request(url, json.dumps(data), headers = headers) res = urllib2.urlopen(req).read() ============================================================================================= # bash shell生成时间戳示例 date +'%s' # bash shell请求示例 curl -s -XPOST -H "Content-Type:application/json" -d 请求数据 http://192.168.10.10:10222/haha/heiheiAPI/test_1_kafka
这里使用了两个脚本:
1)shell脚本进行指标数据采集(最终使用脚本:20_app_process_index.sh)
2)python脚本进行ES入仓请求提交,请求方法:POST。 在python脚本里调用shell脚本,调取shell脚本执行结果。(最终使用脚本:20_app_process_index_es.py)
[root@kevin_test app_process_index]# pwd /root/100_app_index/app_process_index [root@kevin_test app_process_index]# ll total 120 -rwxr-xr-x 1 root root 783 Jun 20 18:16 20_app_process_index_es.py -rwxr-xr-x 1 root root 290 Jun 20 18:16 20_app_process_index.py2 -rwxr-xr-x 1 root root 15398 Jun 20 18:19 20_app_process_index.sh -rwxr-xr-x 1 root root 10414 Jun 20 18:16 20_app_process_index_shell.sh [root@kevin_test ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/vda1 20G 4.8G 14G 26% / tmpfs 2.9G 12K 2.9G 1% /dev/shm /dev/vdb2 9.8G 458M 8.8G 5% /software /dev/vdb3 30G 21G 7.2G 75% /opt/bobo /dev/vdb4 38G 6.9G 29G 20% /var/log/bobo 其中,/opt/cbd为应用程序所在空间,/var/log/bobo是日志所在空间
一、shell脚本 (执行结果要时JSON格式的,这里在shell脚本里拼出来的JSON样式)
[root@kevin_test app_process_index]# cat 20_app_process_index_shell.sh #!/bin/bash source /etc/profile hostName=$(hostname) IP=$(ifconfig eth0| sed -nr '2s/(^[^0-9]+)([0-9.]+)+( .*$)/\2/gp') appName=$(echo ${hostName:3:3}) collectTime=`date +'%s'` dataSource=$(echo "`pwd`/$0@${IP}") indexType="应用进程相关指标" echo "指标类型: ${indexType}" echo "指标采集时间: ${collectTime}" echo "主机名: ${hostName}" echo "应用名称: ${appName}" echo "发送数据源主机: ${dataSource}" echo -e "\n此次采集应用进程相关指标情况如下:" echo -e "---------------------------------------------------" ###########应用进程占内存百分比########### indexName1="应用进程占内存百分比" ps auxw|grep bxapp|awk '{if (NR>1){print $4}}' > /tmp/mem_list NUM1=`cat /tmp/mem_list|wc -l` if [ $NUM1 -eq 0 ];then echo "${indexName}为0" else MEM_PER_SUM=$(awk '{MEM_PER+=$1}END{print MEM_PER}' /tmp/mem_list) echo "${indexName1}:" ${MEM_PER_SUM}% fi echo -e "---------------------------------------------------" ###########应用进程占CPU百分比########### indexName2="应用进程占CPU百分比" ps auxw|grep bxapp|awk '{if (NR>1){print $3}}' > /tmp/cpu_list NUM2=`cat /tmp/cpu_list|wc -l` if [ $NUM2 -eq 0 ];then echo "${indexName2}为0" else CPU_PER_SUM=$(awk '{CPU_PER+=$1}END{print CPU_PER}' /tmp/cpu_list) echo "${indexName2}:" ${CPU_PER_SUM}% fi echo -e "---------------------------------------------------" ###########应用进程run最长时间及其进程########### indexName3="应用进程run最长时间" PID=$(ps -eo user,pid,etime|grep bxapp|sort -k 3 -rn|head -1|awk '{print $2}') RUN_TIME=$(ps -eo user,pid,etime|grep bxapp|sort -k 3 -rn|head -1|awk '{print $3}') PROCESS=$(ps -ef|grep ${PID}|grep -v grep|awk '{print $8}') echo -e "${indexName3}: ${RUN_TIME} \n进程: ${PROCESS}" echo -e "---------------------------------------------------" ###########应用进程TIME-WAIT连接数########### indexName4="应用进程TIME-WAIT连接数" NUM3=$(ss -a|grep -w "TIME_WAIT" |wc -l) echo "${indexName4}: ${NUM3}" echo -e "---------------------------------------------------" ###########应用进程ESTABLISHED连接数########### indexName5="应用进程ESTABLISHED连接数" NUM4=$(ss -a|grep -w "ESTAB" |wc -l) echo "${indexName5}: ${NUM4}" echo -e "---------------------------------------------------" ###########应用进程FIN_WAIT2连接数########### indexName6="应用进程FIN_WAIT2连接数" NUM5=$(ss -a|grep -w "FIN_WAIT2" |wc -l) echo "${indexName6}: ${NUM5}" echo -e "---------------------------------------------------" ###########日志空间使用率########### indexName7="日志空间使用率" LOG_SPACE_USER=$(df -h|grep "/var/log"|awk '{print $5}') echo "${indexName7}: ${LOG_SPACE_USER}" echo -e "---------------------------------------------------" ###########应用程序空间使用率########### indexName8="应用程序空间使用率" APP_SPACE_USER=$(df -h|grep "/opt"|awk '{print $5}') echo "${indexName8}: ${APP_SPACE_USER}" echo -e "---------------------------------------------------" ###########应用所在磁盘读速率(Blk/s)########### indexName9="应用所在磁盘读速率(Blk/s)" DISK=$(df -h|grep "/opt/${appName}"|awk '{print $1}') DEVICE=$(df -h|grep "/opt/${appName}"|awk '{print $1}'|awk -F"/" '{print $3}') READ=$(iostat ${DISK}|grep -w ${DEVICE}|awk '{print $3}') echo "${indexName9}: ${READ}" echo -e "---------------------------------------------------" ###########应用所在磁盘写速率(Blk/s)########### indexName10="应用所在磁盘写速率(Blk/s)" DISK=$(df -h|grep "/opt/${appName}"|awk '{print $1}') DEVICE=$(df -h|grep "/opt/${appName}"|awk '{print $1}'|awk -F"/" '{print $3}') WRITE=$(iostat ${DISK}|grep -w ${DEVICE}|awk '{print $4}') echo "${indexName10}: ${WRITE}" echo -e "---------------------------------------------------" ###########应用所在磁盘TPS########### indexName11="应用所在磁盘TPS" DISK=$(df -h|grep "/opt/${appName}"|awk '{print $1}') DEVICE=$(df -h|grep "/opt/${appName}"|awk '{print $1}'|awk -F"/" '{print $3}') TPS=$(iostat ${DISK}|grep -w ${DEVICE}|awk '{print $2}') echo "${indexName11}: ${TPS}" echo -e "---------------------------------------------------" ###########TOMCAT分配最小内存 ########### indexName12="TOMCAT分配最小内存" NUM6=$(ps -ef|grep -w tomcat|grep -v grep|wc -l) if [ ${NUM6} -eq 0 ];then echo "当前没有Tomcat进程,${indexName12}: 0" else for TOMCAT in $(find /opt/${appName}/ -name *tomcat* -type d) do NUM7=$(ps -ef|grep ${TOMCAT}|grep -v grep|wc -l) if [ ${NUM7} -eq 0 ];then echo "not ${TOMCAT}" >> /dev/null 2>&1 else XMS=$(ps -ef|grep ${TOMCAT}/|grep "\-Xms"|awk -F"-Xms" '{print $2}'|cut -d" " -f1) echo -e "${indexName12}: ${XMS} (实例名:${TOMCAT})" fi done fi echo -e "---------------------------------------------------" ###########TOMCAT分配最大内存 ########### indexName13="TOMCAT分配最大内存" NUM8=$(ps -ef|grep -w tomcat|grep -v grep|wc -l) if [ ${NUM8} -eq 0 ];then echo "当前没有Tomcat进程,${indexName13}: 0" else for TOMCAT in $(find /opt/${appName}/ -name *tomcat* -type d) do NUM9=$(ps -ef|grep ${TOMCAT}|grep -v grep|wc -l) if [ ${NUM9} -eq 0 ];then echo "not ${TOMCAT}" >> /dev/null 2>&1 else XMX=$(ps -ef|grep ${TOMCAT}/|grep "\-Xmx"|awk -F"-Xmx" '{print $2}'|cut -d" " -f1) echo -e "${indexName13}: ${XMX} (实例名:${TOMCAT})" fi done fi echo -e "---------------------------------------------------" ###########TOMCAT分配年轻代空间 ########### indexName14="TOMCAT分配年轻代空间" NUM10=$(ps -ef|grep -w tomcat|grep -v grep|wc -l) if [ ${NUM10} -eq 0 ];then echo "当前没有Tomcat进程,${indexName14}: 0" else for TOMCAT in $(find /opt/${appName}/ -name *tomcat* -type d) do NUM11=$(ps -ef|grep ${TOMCAT}|grep -v grep|wc -l) if [ ${NUM11} -eq 0 ];then echo "not ${TOMCAT}" >> /dev/null 2>&1 else XMN=$(ps -ef|grep ${TOMCAT}/|grep "\-Xmn"|awk -F"-Xmn" '{print $2}'|cut -d" " -f1) echo -e "${indexName14}: ${XMN} (实例名:${TOMCAT})" fi done fi echo -e "---------------------------------------------------" ###########TOMCAT每个线程的堆栈大小 ########### indexName15="TOMCAT每个线程的堆栈大小" NUM12=$(ps -ef|grep -w tomcat|grep -v grep|wc -l) if [ ${NUM12} -eq 0 ];then echo "当前没有Tomcat进程,${indexName15}: 0" else for TOMCAT in $(find /opt/${appName}/ -name *tomcat* -type d) do NUM13=$(ps -ef|grep ${TOMCAT}|grep -v grep|wc -l) if [ ${NUM13} -eq 0 ];then echo "not ${TOMCAT}" >> /dev/null 2>&1 else XSS=$(ps -ef|grep ${TOMCAT}/|grep "\-Xss"|awk -F"-Xss" '{print $2}'|cut -d" " -f1) echo -e "${indexName15}: ${XSS} (实例名:${TOMCAT})" fi done fi echo -e "---------------------------------------------------" ###########TOMCAT-GC时长占比 ########### indexName16="TOMCAT-GC时长占比" NUM14=$(ps -ef|grep -w tomcat|grep -v grep|wc -l) if [ ${NUM14} -eq 0 ];then echo "当前没有Tomcat进程,${indexName16}: 0" else for TOMCAT in $(find /opt/${appName}/ -name *tomcat* -type d) do NUM15=$(ps -ef|grep ${TOMCAT}|grep -v grep|wc -l) if [ ${NUM15} -eq 0 ];then echo "not ${TOMCAT}" >> /dev/null 2>&1 else GC=$(ps -ef|grep ${TOMCAT}/|grep "\GCTimeRatio"|awk -F"GCTimeRatio" '{print $2}'|cut -d" " -f1|awk -F"=" '{print $2}') echo -e "${indexName16}: ${GC} (实例名:${TOMCAT})" fi done fi echo -e "---------------------------------------------------" ###########Zookeeper分配最小内存 ########### indexName17="Zookeeper分配最小内存" #zookeeper安装时没有手动定义内存配置,默认为系统内存的四分之一 NUM16=$(ps -ef|grep zookeeper|grep -v grep|wc -l) if [ ${NUM16} -eq 0 ];then echo "当前没有zookeeper进程,${indexName17}: 0" else MEM_TOTAL=$(free -m|grep "Mem"|awk '{print $2}') ZK_MEM_MAX=`expr ${MEM_TOTAL} / 4` echo "${indexName17}: ${ZK_MEM_MAX}M" fi echo -e "---------------------------------------------------" ###########Zookeeper分配最大内存 ########### indexName18="Zookeeper分配最大内存" #zookeeper安装时没有手动定义内存配置,默认为系统内存的四分之一 NUM17=$(ps -ef|grep zookeeper|grep -v grep|wc -l) if [ ${NUM17} -eq 0 ];then echo "当前没有zookeeper进程,${indexName18}: 0" else MEM_TOTAL=$(free -m|grep "Mem"|awk '{print $2}') ZK_MEM_MAX=`expr ${MEM_TOTAL} / 4` echo "${indexName18}: ${ZK_MEM_MAX}M" fi echo -e "---------------------------------------------------" ###########KAFKA分配最小内存 ########### indexName19="KAFKA分配最小内存" NUM18=$(ps -ef|grep kafka|grep -v grep|wc -l) if [ ${NUM18} -eq 0 ];then echo "当前没有KAFKA进程,${indexName19}: 0" else cd /opt/${appName}/ for KAFKA in $(find /opt/${appName}/ -name *kafka* -type d) do NUM19=$(ps -ef|grep ${KAFKA}|grep -v grep|wc -l) if [ ${NUM19} -eq 0 ];then echo "not this ${KAFKA}" >/dev/null 2>&1 else FILE=${KAFKA}/bin/kafka-server-start.sh KAFKA_MEM_MIN=$(cat ${FILE}|grep "Xmx"|awk -F'"' '{print $2}'|awk '{print $2}'|awk -F"Xms" '{print $2}') echo "${indexName19}: ${KAFKA_MEM_MIN}" fi done fi echo -e "---------------------------------------------------" ###########KAFKA分配最大内存 ########### indexName20="KAFKA分配最大内存" NUM20=$(ps -ef|grep kafka|grep -v grep|wc -l) if [ ${NUM20} -eq 0 ];then echo "当前没有KAFKA进程,${indexName20}: 0" else cd /opt/${appName}/ for KAFKA in $(find /opt/${appName}/ -name *kafka* -type d) do NUM21=$(ps -ef|grep ${KAFKA}|grep -v grep|wc -l) if [ ${NUM21} -eq 0 ];then echo "not this ${KAFKA}" >/dev/null 2>&1 else FILE=${KAFKA}/bin/kafka-server-start.sh KAFKA_MEM_MAX=$(cat ${FILE}|grep "Xmx"|awk -F'"' '{print $2}'|awk '{print $1}'|awk -F"Xmx" '{print $2}') echo "${indexName20}: ${KAFKA_MEM_MAX}" fi done fi echo -e "---------------------------------------------------" ###########应用系统进程数量 ########### indexName21="应用系统进程数量" NUM22=$(ps -eo user,pid,cmd|awk '{if($1=="bxapp")print}'|wc -l) echo "${indexName21}:"${NUM22}
执行这个脚本:
[root@kevin_test app_process_index]# sh 20_app_process_index_shell.sh 指标类型: 应用进程相关指标 指标采集时间: 1561029337 主机名: kevin_test 应用名称: bobo 发送数据源主机: /root/100_app_index/app_process_index/20_app_process_index_shell.sh@192.168.10.99 此次采集应用进程相关指标情况如下: --------------------------------------------------- 应用进程占内存百分比: 69.7% --------------------------------------------------- 应用进程占CPU百分比: 13.8% --------------------------------------------------- 应用进程run最长时间: 609-08:33:26 进程: ../src/redis-sentinel --------------------------------------------------- 应用进程TIME-WAIT连接数: 0 --------------------------------------------------- 应用进程ESTABLISHED连接数: 455 --------------------------------------------------- 应用进程FIN_WAIT2连接数: 0 --------------------------------------------------- 日志空间使用率: 20% --------------------------------------------------- 应用程序空间使用率: 75% --------------------------------------------------- 应用所在磁盘读速率(Blk/s): 2.30 --------------------------------------------------- 应用所在磁盘写速率(Blk/s): 16.41 --------------------------------------------------- 应用所在磁盘TPS: 1.31 --------------------------------------------------- 当前没有Tomcat进程,TOMCAT分配最小内存: 0 --------------------------------------------------- 当前没有Tomcat进程,TOMCAT分配最大内存: 0 --------------------------------------------------- 当前没有Tomcat进程,TOMCAT分配年轻代空间: 0 --------------------------------------------------- 当前没有Tomcat进程,TOMCAT每个线程的堆栈大小: 0 --------------------------------------------------- 当前没有Tomcat进程,TOMCAT-GC时长占比: 0 --------------------------------------------------- Zookeeper分配最小内存: 1459M --------------------------------------------------- Zookeeper分配最大内存: 1459M --------------------------------------------------- 当前没有KAFKA进程,KAFKA分配最小内存: 0 --------------------------------------------------- 当前没有KAFKA进程,KAFKA分配最大内存: 0 --------------------------------------------------- 应用系统进程数量:10
注意:上面是还没有转化JSON格式的脚本。下面是转化为JSON格式后的脚本(实际用的就是下面这个脚本)
[root@kevin_test app_process_index]# cat 20_app_process_index.sh #!/bin/bash source /etc/profile hostName=$(hostname) IP=$(ifconfig eth0| sed -nr '2s/(^[^0-9]+)([0-9.]+)+( .*$)/\2/gp') appName=$(echo ${hostName:3:3}) collectTime=$(date +'%s') dataSource=$(echo "$0@${IP}") indexType="app_process" ###########应用进程占内存百分比########### ps auxw|grep bxapp|awk '{if (NR>1){print $4}}' > /tmp/mem_list NUM1=`cat /tmp/mem_list|wc -l` if [ $NUM1 -eq 0 ];then echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用进程占内存百分比\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"当前没有进程\", \"checker\":\"${dataSource}\" }+" else MEM_PER_SUM=$(awk '{MEM_PER+=$1}END{print MEM_PER}' /tmp/mem_list) echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用进程占内存百分比\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${MEM_PER_SUM}%\", \"checker\":\"${dataSource}\" }+" fi ###########应用进程占CPU百分比########### ps auxw|grep bxapp|awk '{if (NR>1){print $3}}' > /tmp/cpu_list NUM2=`cat /tmp/cpu_list|wc -l` if [ $NUM2 -eq 0 ];then echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用进程占CPU百分比\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"当前没有进程\", \"checker\":\"${dataSource}\" }+" else CPU_PER_SUM=$(awk '{CPU_PER+=$1}END{print CPU_PER}' /tmp/cpu_list) echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用进程占CPU百分比\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${CPU_PER_SUM}%\", \"checker\":\"${dataSource}\" }+" fi ###########应用进程run最长时间及其进程########### PID=$(ps -eo user,pid,etime|grep bxapp|sort -k 3 -rn|head -1|awk '{print $2}') RUN_TIME=$(ps -eo user,pid,etime|grep bxapp|sort -k 3 -rn|head -1|awk '{print $3}') PROCESS=$(ps -ef|grep ${PID}|grep -v grep|awk '{print $8}') echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用进程run最长时间及其进程\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${RUN_TIME}, ${PROCESS}\", \"checker\":\"${dataSource}\" }+" ###########应用进程TIME-WAIT连接数########### NUM3=$(ss -a|grep -w "TIME_WAIT" |wc -l) echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用进程TIME-WAIT连接数\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${NUM3}\", \"checker\":\"${dataSource}\" }+" ###########应用进程ESTABLISHED连接数########### NUM4=$(ss -a|grep -w "ESTAB" |wc -l) echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用进程ESTABLISHED连接数\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${NUM4}\", \"checker\":\"${dataSource}\" }+" ###########应用进程FIN_WAIT2连接数########### NUM5=$(ss -a|grep -w "FIN_WAIT2" |wc -l) echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用进程FIN_WAIT2连接数\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${NUM5}\", \"checker\":\"${dataSource}\" }+" ###########日志空间使用率########### LOG_SPACE_USER=$(df -h|grep "/var/log"|awk '{print $5}') echo -e "{ \"type\":\"${indexType}\", \"desc\":\"日志空间使用率\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${LOG_SPACE_USER}\", \"checker\":\"${dataSource}\" }+" ###########应用程序空间使用率########### APP_SPACE_USER=$(df -h|grep "/opt"|awk '{print $5}') echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用程序空间使用率\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${APP_SPACE_USER}\", \"checker\":\"${dataSource}\" }+" ###########应用所在磁盘读速率(Blk/s)########### DISK=$(df -h|grep "/opt/${appName}"|awk '{print $1}') DEVICE=$(df -h|grep "/opt/${appName}"|awk '{print $1}'|awk -F"/" '{print $3}') READ=$(iostat ${DISK}|grep -w ${DEVICE}|awk '{print $3}') echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用所在磁盘读速率(Blk/s)\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${READ}\", \"checker\":\"${dataSource}\" }+" ###########应用所在磁盘写速率(Blk/s)########### DISK=$(df -h|grep "/opt/${appName}"|awk '{print $1}') DEVICE=$(df -h|grep "/opt/${appName}"|awk '{print $1}'|awk -F"/" '{print $3}') WRITE=$(iostat ${DISK}|grep -w ${DEVICE}|awk '{print $4}') echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用所在磁盘写速率(Blk/s)\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${WRITE}\", \"checker\":\"${dataSource}\" }+" ###########应用所在磁盘TPS########### DISK=$(df -h|grep "/opt/${appName}"|awk '{print $1}') DEVICE=$(df -h|grep "/opt/${appName}"|awk '{print $1}'|awk -F"/" '{print $3}') TPS=$(iostat ${DISK}|grep -w ${DEVICE}|awk '{print $2}') echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用所在磁盘TPS\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${TPS}\", \"checker\":\"${dataSource}\" }+" ###########TOMCAT分配最小内存 ########### NUM6=$(ps -ef|grep -w tomcat|grep -v grep|wc -l) if [ ${NUM6} -eq 0 ];then echo -e "{ \"type\":\"${indexType}\", \"desc\":\"TOMCAT分配最小内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"当前没有Tomcat进程\", \"checker\":\"${dataSource}\" }+" else for TOMCAT in $(find /opt/${appName}/ -name *tomcat* -type d) do NUM7=$(ps -ef|grep ${TOMCAT}|grep -v grep|wc -l) if [ ${NUM7} -eq 0 ];then echo "not ${TOMCAT}" >> /dev/null 2>&1 else XMS=$(ps -ef|grep ${TOMCAT}/|grep "\-Xms"|awk -F"-Xms" '{print $2}'|cut -d" " -f1) echo -e "{ \"type\":\"${indexType}\", \"desc\":\"TOMCAT分配最小内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${XMS} (实例名:${TOMCAT})\", \"checker\":\"${dataSource}\" }+" fi done fi ###########TOMCAT分配最大内存 ########### NUM8=$(ps -ef|grep -w tomcat|grep -v grep|wc -l) if [ ${NUM8} -eq 0 ];then echo -e "{ \"type\":\"${indexType}\", \"desc\":\"TOMCAT分配最大内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"当前没有Tomcat进程\", \"checker\":\"${dataSource}\" }+" else for TOMCAT in $(find /opt/${appName}/ -name *tomcat* -type d) do NUM9=$(ps -ef|grep ${TOMCAT}|grep -v grep|wc -l) if [ ${NUM9} -eq 0 ];then echo "not ${TOMCAT}" >> /dev/null 2>&1 else XMX=$(ps -ef|grep ${TOMCAT}/|grep "\-Xmx"|awk -F"-Xmx" '{print $2}'|cut -d" " -f1) echo -e "{ \"type\":\"${indexType}\", \"desc\":\"TOMCAT分配最大内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${XMX} (实例名:${TOMCAT})\", \"checker\":\"${dataSource}\" }+" fi done fi ###########TOMCAT分配年轻代空间 ########### NUM10=$(ps -ef|grep -w tomcat|grep -v grep|wc -l) if [ ${NUM10} -eq 0 ];then echo -e "{ \"type\":\"${indexType}\", \"desc\":\"TOMCAT分配年轻代空间\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"当前没有Tomcat进程\", \"checker\":\"${dataSource}\" }+" else for TOMCAT in $(find /opt/${appName}/ -name *tomcat* -type d) do NUM11=$(ps -ef|grep ${TOMCAT}|grep -v grep|wc -l) if [ ${NUM11} -eq 0 ];then echo "not ${TOMCAT}" >> /dev/null 2>&1 else XMN=$(ps -ef|grep ${TOMCAT}/|grep "\-Xmn"|awk -F"-Xmn" '{print $2}'|cut -d" " -f1) echo -e "{ \"type\":\"${indexType}\", \"desc\":\"TOMCAT分配年轻代空间\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${XMN} (实例名:${TOMCAT})\", \"checker\":\"${dataSource}\" }+" fi done fi ###########TOMCAT每个线程的堆栈大小 ########### NUM12=$(ps -ef|grep -w tomcat|grep -v grep|wc -l) if [ ${NUM12} -eq 0 ];then echo -e "{ \"type\":\"${indexType}\", \"desc\":\"TOMCAT每个线程的堆栈大小\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"当前没有Tomcat进程\", \"checker\":\"${dataSource}\" }+" else for TOMCAT in $(find /opt/${appName}/ -name *tomcat* -type d) do NUM13=$(ps -ef|grep ${TOMCAT}|grep -v grep|wc -l) if [ ${NUM13} -eq 0 ];then echo "not ${TOMCAT}" >> /dev/null 2>&1 else XSS=$(ps -ef|grep ${TOMCAT}/|grep "\-Xss"|awk -F"-Xss" '{print $2}'|cut -d" " -f1) echo -e "{ \"type\":\"${indexType}\", \"desc\":\"TOMCAT每个线程的堆栈大小\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${XSS} (实例名:${TOMCAT})\", \"checker\":\"${dataSource}\" }+" fi done fi ###########TOMCAT-GC时长占比 ########### NUM14=$(ps -ef|grep -w tomcat|grep -v grep|wc -l) if [ ${NUM14} -eq 0 ];then echo -e "{ \"type\":\"${indexType}\", \"desc\":\"TOMCAT-GC时长占比\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"当前没有Tomcat进程\", \"checker\":\"${dataSource}\" }+" else for TOMCAT in $(find /opt/${appName}/ -name *tomcat* -type d) do NUM15=$(ps -ef|grep ${TOMCAT}|grep -v grep|wc -l) if [ ${NUM15} -eq 0 ];then echo "not ${TOMCAT}" >> /dev/null 2>&1 else GC=$(ps -ef|grep ${TOMCAT}/|grep "\GCTimeRatio"|awk -F"GCTimeRatio" '{print $2}'|cut -d" " -f1|awk -F"=" '{print $2}') echo -e "{ \"type\":\"${indexType}\", \"desc\":\"TOMCAT-GC时长占比\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${GC} (实例名:${TOMCAT})\", \"checker\":\"${dataSource}\" }+" fi done fi ###########Zookeeper分配最小内存########### #zookeeper安装时没有手动定义内存配置,默认为系统内存的四分之一 NUM16=$(ps -ef|grep zookeeper|grep -v grep|wc -l) if [ ${NUM16} -eq 0 ];then echo -e "{ \"type\":\"${indexType}\", \"desc\":\"Zookeeper分配最小内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"当前没有Zookeeper进程\", \"checker\":\"${dataSource}\" }+" else MEM_TOTAL=$(free -m|grep "Mem"|awk '{print $2}') ZK_MEM_MIN=`expr ${MEM_TOTAL} / 4` echo -e "{ \"type\":\"${indexType}\", \"desc\":\"Zookeeper分配最小内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${ZK_MEM_MIN}M\", \"checker\":\"${dataSource}\" }+" fi ###########Zookeeper分配最大内存########### #zookeeper安装时没有手动定义内存配置,默认为系统内存的四分之一 NUM17=$(ps -ef|grep zookeeper|grep -v grep|wc -l) if [ ${NUM17} -eq 0 ];then echo -e "{ \"type\":\"${indexType}\", \"desc\":\"Zookeeper分配最大内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"当前没有Zookeeper进程\", \"checker\":\"${dataSource}\" }+" else MEM_TOTAL=$(free -m|grep "Mem"|awk '{print $2}') ZK_MEM_MAX=`expr ${MEM_TOTAL} / 4` echo -e "{ \"type\":\"${indexType}\", \"desc\":\"Zookeeper分配最大内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${ZK_MEM_MAX}M\", \"checker\":\"${dataSource}\" }+" fi ###########KAFKA分配最小内存 ########### NUM18=$(ps -ef|grep kafka|grep -v grep|wc -l) if [ ${NUM18} -eq 0 ];then echo -e "{ \"type\":\"${indexType}\", \"desc\":\"KAFKA分配最小内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"当前没有KAFKA进程\", \"checker\":\"${dataSource}\" }+" else cd /opt/${appName}/ for KAFKA in $(find /opt/${appName}/ -name *kafka* -type d) do NUM19=$(ps -ef|grep ${KAFKA}|grep -v grep|wc -l) if [ ${NUM19} -eq 0 ];then echo "not this ${KAFKA}" >/dev/null 2>&1 else FILE=${KAFKA}/bin/kafka-server-start.sh KAFKA_MEM_MIN=$(cat ${FILE}|grep "Xmx"|awk -F'"' '{print $2}'|awk '{print $2}'|awk -F"Xms" '{print $2}') echo -e "{ \"type\":\"${indexType}\", \"desc\":\"KAFKA分配最小内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${KAFKA_MEM_MIN}\", \"checker\":\"${dataSource}\" }+" fi done fi ###########KAFKA分配最大内存 ########### NUM20=$(ps -ef|grep kafka|grep -v grep|wc -l) if [ ${NUM20} -eq 0 ];then echo -e "{ \"type\":\"${indexType}\", \"desc\":\"KAFKA分配最大内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"当前没有KAFKA进程\", \"checker\":\"${dataSource}\" }+" else cd /opt/${appName}/ for KAFKA in $(find /opt/${appName}/ -name *kafka* -type d) do NUM21=$(ps -ef|grep ${KAFKA}|grep -v grep|wc -l) if [ ${NUM21} -eq 0 ];then echo "not this ${KAFKA}" >/dev/null 2>&1 else FILE=${KAFKA}/bin/kafka-server-start.sh KAFKA_MEM_MAX=$(cat ${FILE}|grep "Xmx"|awk -F'"' '{print $2}'|awk '{print $1}'|awk -F"Xmx" '{print $2}') echo -e "{ \"type\":\"${indexType}\", \"desc\":\"KAFKA分配最大内存\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${KAFKA_MEM_MAX}\", \"checker\":\"${dataSource}\" }+" fi done fi ###########应用系统进程数量 ########### NUM22=$(ps -eo user,pid,cmd|awk '{if($1=="bxapp")print}'|wc -l) echo -e "{ \"type\":\"${indexType}\", \"desc\":\"应用系统进程数量\", \"@timestamp\":\"${collectTime}\", \"appname\":\"${appName}\", \"hostname\":\"${hostName}\", \"value\":\"${NUM22}\", \"checker\":\"${dataSource}\" }"
执行这个脚本(JSON字段跟上面ES入仓要求的字段匹配):
[root@kevin_test app_process_index]# sh 20_app_process_index.sh { "type":"app_process", "desc":"应用进程占内存百分比", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"69.7%", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"应用进程占CPU百分比", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"13.8%", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"应用进程run最长时间及其进程", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"609-08:36:42, ../src/redis-sentinel", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"应用进程TIME-WAIT连接数", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"0", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"应用进程ESTABLISHED连接数", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"454", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"应用进程FIN_WAIT2连接数", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"0", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"日志空间使用率", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"20%", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"应用程序空间使用率", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"75%", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"应用所在磁盘读速率(Blk/s)", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"2.30", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"应用所在磁盘写速率(Blk/s)", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"16.41", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"应用所在磁盘TPS", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"1.31", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"TOMCAT分配最小内存", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"TOMCAT分配最大内存", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"TOMCAT分配年轻代空间", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"TOMCAT每个线程的堆栈大小", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"TOMCAT-GC时长占比", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"Zookeeper分配最小内存", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"1459M", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"Zookeeper分配最大内存", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"1459M", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"KAFKA分配最小内存", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有KAFKA进程", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"KAFKA分配最大内存", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有KAFKA进程", "checker":"20_app_process_index.sh@192.168.10.99" }+ { "type":"app_process", "desc":"应用系统进程数量", "@timestamp":"1561029533", "appname":"bobo", "hostname":"kevin_test", "value":"10", "checker":"20_app_process_index.sh@192.168.10.99" }
二、然后使用python调用shell脚本,调取shell脚本的执行结果
下面三个脚本都可以使用,区别在于:
第一个脚本:默认就是在当前路径下调用shell脚本,shell和python两个脚本是放在统一目录下的
第二个脚本和第三个脚本:先获取当前脚本所在目录路径,然后在python脚本里调用shell脚本时,使用的是shell脚本全路径
这里推荐使用第三个脚本,但是注意:
使用第二或第三个脚本时,shell脚本里的dataSource要写成:dataSource=$(echo "$0@${IP}")
而使用第一个脚本时,shell脚本里的dataSource要写成:dataSource=$(echo "`pwd`/$0@${IP}")
区别就是:
第一或第二个脚本,在输出的JSON格式的结果中,checker不会带上`pwd`
第三个脚本,在输出的JSON格式的结果中,checker会带上`pwd`,所以就不需要在shell脚本里再加上`pwd`,否则路径就多了~
第一个脚本================================================================== [root@kevin_test app_process_index]# cat 20_app_process_index.py #!/usr/bin/python # -*- coding: UTF-8 -*- import os import json #默认就在当前路径下执行20_app_process_index.sh json_set = os.popen("sh 20_app_process_index.sh").read().split('+') for json_str in json_set: print json_str 第二个脚本================================================================== [root@kevin_test app_process_index]# cat 20_app_process_index.py1 #!/usr/bin/python # -*- coding: UTF-8 -*- import os import json #获取当前路径,调用shell脚本, 执行时跟全路径 now_path=os.path.abspath('.') index_script="sh %s/20_app_process_index.sh" % (now_path) json_set = os.popen(index_script).read().split('+') for json_str in json_set: print json_str 第三个脚本================================================================ [root@kevin_test app_process_index]# cat 20_app_process_index.py2 #!/usr/bin/python # -*- coding: UTF-8 -*- import os import json #获取当前路径,调用shell脚本, 执行时跟全路径 now_path=os.path.abspath(__file__) now_path=os.path.dirname(now_path) index_script="sh %s/20_app_process_index.sh" % (now_path) json_set = os.popen(index_script).read().split('+') for json_str in json_set: print json_str
来看看几个脚本结果:
如下第一个脚本的"checker" 字段,由于shell脚本里的"dataSource"没有带"pwd",所以"checker"字段中的脚本路径不全
[root@kevin_test app_process_index]# python 20_app_process_index.py { "type":"app_process", "desc":"应用进程占内存百分比", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"69.7%", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用进程占CPU百分比", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"13.8%", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用进程run最长时间及其进程", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"609-08:46:50, ../src/redis-sentinel", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用进程TIME-WAIT连接数", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"0", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用进程ESTABLISHED连接数", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"455", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用进程FIN_WAIT2连接数", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"0", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"日志空间使用率", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"20%", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用程序空间使用率", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"75%", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用所在磁盘读速率(Blk/s)", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"2.30", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用所在磁盘写速率(Blk/s)", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"16.41", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用所在磁盘TPS", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"1.31", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"TOMCAT分配最小内存", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"TOMCAT分配最大内存", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"TOMCAT分配年轻代空间", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"TOMCAT每个线程的堆栈大小", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"TOMCAT-GC时长占比", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"Zookeeper分配最小内存", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"1459M", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"Zookeeper分配最大内存", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"1459M", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"KAFKA分配最小内存", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有KAFKA进程", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"KAFKA分配最大内存", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有KAFKA进程", "checker":"20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用系统进程数量", "@timestamp":"1561030141", "appname":"bobo", "hostname":"kevin_test", "value":"11", "checker":"20_app_process_index.sh@192.168.10.99" }
第二或第三个脚本执行结果一样,"checker"字段中的脚本路径是完整路径
[root@kevin_test app_process_index]# python 20_app_process_index.py { "type":"app_process", "desc":"应用进程占内存百分比", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"70.8%", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用进程占CPU百分比", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"144.8%", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用进程run最长时间及其进程", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"609-08:51:53, ../src/redis-sentinel", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用进程TIME-WAIT连接数", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"0", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用进程ESTABLISHED连接数", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"456", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用进程FIN_WAIT2连接数", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"0", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"日志空间使用率", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"20%", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用程序空间使用率", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"75%", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用所在磁盘读速率(Blk/s)", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"2.30", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用所在磁盘写速率(Blk/s)", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"16.41", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用所在磁盘TPS", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"1.31", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"TOMCAT分配最小内存", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"TOMCAT分配最大内存", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"TOMCAT分配年轻代空间", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"TOMCAT每个线程的堆栈大小", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"TOMCAT-GC时长占比", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有Tomcat进程", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"Zookeeper分配最小内存", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"1459M", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"Zookeeper分配最大内存", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"1459M", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"KAFKA分配最小内存", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有KAFKA进程", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"KAFKA分配最大内存", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"当前没有KAFKA进程", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" } { "type":"app_process", "desc":"应用系统进程数量", "@timestamp":"1561030444", "appname":"bobo", "hostname":"kevin_test", "value":"10", "checker":"/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99" }
最后将上面的第三个脚本整合到"20_app_process_index_es.py"脚本里,作为ES入仓请求的脚本
[root@kevin_test app_process_index]# cat 20_app_process_index_es.py #!/usr/bin/python # -*- coding: UTF-8 -*- import urllib2 import hashlib import json import os now_path=os.path.abspath(__file__) now_path=os.path.dirname(now_path) index_script="sh %s/20_app_process_index.sh" % (now_path) json_set = os.popen(index_script).read().split('+') app_index = [] for json_str in json_set: app_index.append(json.loads(json_str)) url = 'http://192.168.10.10:10222/haha/heiheiAPI/test_1_kafka' data = { 'token': 'bd1df2b11111111112222222222222222333333333333333333yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy', 'data': app_index } headers = {'Content-Type': 'application/json'} req = urllib2.Request(url, json.dumps(data), headers = headers) res = urllib2.urlopen(req).read() print res #print app_index
执行这个请求脚本
[root@kevin_test app_process_index]# python 20_app_process_index_es.py {"retCode": 200} 将脚本中的最后一行注释打开(即"#print app_index"),测试下入仓的数据: [root@kevin_test app_process_index]# python 20_app_process_index_es.py {"retCode": 200} [{u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'69.7%', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u' \u5e94\u7528\u8fdb\u7a0b\u5360\u5185\u5b58\u767e\u5206\u6bd4'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'13.8%', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'\u5e94\u7528\u8fdb\u7a0b\u5360CPU\u767e\u5206\u6bd4'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'609-08:54:24, ../src/redis-sentinel', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'\u5e94\u7528\u8fdb\u7a0brun\u6700\u957f\u65f6\u95f4\u53ca\u5176\u8fdb\u7a0b'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'0', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'\u5e94\u7528\u8fdb\u7a0bTIME-WAIT\u8fde\u63a5\u6570'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'458', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'\u5e94\u7528\u8fdb\u7a0bESTABLISHED\u8fde\u63a5\u6570'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'0', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'\u5e94\u7528\u8fdb\u7a0bFIN_WAIT2\u8fde\u63a5\u6570'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'20%', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'\u65e5\u5fd7\u7a7a\u95f4\u4f7f\u7528\u7387'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'75%', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'\u5e94\u7528\u7a0b\u5e8f\u7a7a\u95f4\u4f7f\u7528\u7387'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'2.30', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'\u5e94\u7528\u6240\u5728\u78c1\u76d8\u8bfb\u901f\u7387\uff08Blk/s\uff09'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'16.41', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'\u5e94\u7528\u6240\u5728\u78c1\u76d8\u5199\u901f\u7387\uff08Blk/s\uff09'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'1.31', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'\u5e94\u7528\u6240\u5728\u78c1\u76d8TPS'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'\u5f53\u524d\u6ca1\u6709Tomcat\u8fdb\u7a0b', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'TOMCAT\u5206\u914d\u6700\u5c0f\u5185\u5b58'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'\u5f53\u524d\u6ca1\u6709Tomcat\u8fdb\u7a0b', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'TOMCAT\u5206\u914d\u6700\u5927\u5185\u5b58'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'\u5f53\u524d\u6ca1\u6709Tomcat\u8fdb\u7a0b', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99', u'type': u'app_process', u'desc': u'TOMCAT\u5206\u914d\u5e74\u8f7b\u4ee3\u7a7a\u95f4'}, {u'appname': u'bobo', u'@timestamp': u'1561030595', u'hostname': u'kevin_test', u'value': u'\u5f53\u524d\u6ca1\u6709Tomcat\u8fdb\u7a0b', u'checker': u'/root/100_app_index/app_process_index/20_app_process_index.sh@192.168.10.99',
======================================================
上面涉及到python的当前路径调用问题(即上面讨论的第一、二、三的python脚本)
需要注意: 上面20_app_process_index_es.py脚本中的"json_set = os.popen(index_script).read().split('+')"说明: 20_app_process_index.sh脚本执行结果为json格式,并通过"+"分割,那么传给列表app_index里面,每个元素 就是这个shell脚本执行结果中的一组json内容。 注意:如果shell执行结果的内容中有"+",则执行20_app_process_index_es.py脚本就会失败,python解析失败! 可以在20_app_process_index_es.py脚本中"app_index.append(json.loads(json_str))"这一行的上面添加" print json_str",即打印app_index列表内容进行测试。 如果shell脚本执行结果中有"+",跟设置的分隔符一样, 就可以测试到执行python脚本失败的原因了! 可以将shell脚本中的分割符号"+"改写其他的,比如"+++++"
这里对python获取当前目录路径和文件的方法做个记录:
一、获取当前路径
1)使用sys.argv[0] import sys print sys.argv[0] #输出 #本地路径 2)os模块 import os print os.getcwd() #获取当前工作目录路径 print os.path.abspath('.') #获取当前工作目录路径 print os.path.abspath('test.txt') #获取当前目录文件下的工作目录路径 print os.path.abspath('..') #获取当前工作的父目录 !注意是父目录路径 print os.path.abspath(os.curdir) #获取当前工作目录路径 3)改变当前目录 使用: os.chdir(path)。 4)组合路径返回 os.path.join('file1','file2','file3') 合并得到路径 file1/file2/file3
二、获得当前目录下所有文件
1)os.walk() 用于在目录树种游走输出目录中的文件名字,向上或下; 语法 os.walk(top[, topdown=True[, onerror=None[, followlinks=False]]]) 参数: top -- 根目录下的每一个文件夹(包含它自己), 产生3-元组 (dirpath, dirnames, filenames)【文件夹路径, 文件夹名字, 文件名】。 topdown --可选,为True或者没有指定, 一个目录的的3-元组将比它的任何子文件夹的3-元组先产生 (目录自上而下)。 如果topdown为 False, 一个目录的3-元组将比它的任何子文件夹的3-元组后产生 (目录自下而上)。 onerror -- 可选,是一个函数; 它调用时有一个参数, 一个OSError实例。报告这错误后,继续walk,或者抛出exception终止walk。 followlinks -- 设置为 true,则通过软链接访问目录。 2) import os root = os.getcwd() def file_name(file_dir): for root, dirs, files in os.walk(file_dir): print "-----------" print root #os.walk()所在目录 print dirs #os.walk()所在目录的所有目录名 print files #os.walk()所在目录的所有非目录文件名 print " " file_name(root)