vc-mysql-sniffer统计MySQL的SQL分布
有时候我们需要统计线上的SQL执行情况，比如想知道哪条SQL执行最频繁，我们可以开启general_log，然后进行统计，但是general_log开启非常损耗性能，那么我们可以使用vc-mysql-sniffer来代替，该工具是编译好的二进制，下载即可使用。下载地址：
https://www.vividcortex.com/resources/network-analyzer-for-mysql
我这里修改了网上的一个脚本，分析vc-mysql-sniffer抓取到的结果统计频繁执行的SQL。
#!/usr/bin/python #coding:utf8 # python analysis-vc-log.py 3315 | sort | uniq -c | sort -nr |head -n 10 import re import sys import os import commands vc_sniffer_time=5 port=sys.argv[1] vc_cmd=""" /usr/bin/timeout %s /data/software/vc-mysql-sniffer -binding="[::]:%s" > /tmp/tmp_vc_mysql_%s.txt """ % (vc_sniffer_time,port,port) outtext = commands.getoutput(vc_cmd) cmd=""" grep -Ev '# Time:|# User@Host' /tmp/tmp_vc_mysql_%s.txt |sed 's/# Query_time.*/myxxxxx/g' |awk BEGIN{RS=EOF}'{gsub(/\\n/," ");print}'|awk BEGIN{RS=EOF}'{gsub(/myxxxxx/,"\\n");print}' >/tmp/vc_mysql_%s.txt""" % (port,port) outtext = commands.getoutput(cmd) file="/tmp/vc_mysql_%s.txt" % (port) logFo = open(file) for line in logFo: line = re.sub(r"\n","",line) lineMatch = re.match(r".*",line) if lineMatch: lineTmp = lineMatch.group(0) lineTmp = lineTmp.lower() # remove extra space lineTmp = re.sub(r"\s+", " ",lineTmp) # replace values (value) to values (x) lineTmp = re.sub(r"values\s*\(.*?\)", "values (x)",lineTmp) # replace filed = 'value' to filed = 'x' lineTmp = re.sub(r"(=|>|<|>=|<=)\s*('|\").*?\2","\\1 'x'",lineTmp) # replace filed = value to filed = x lineTmp = re.sub(r"(=|>|<|>=|<=)\s*[0-9]+","\\1 x",lineTmp) # replace like 'value' to like 'x' lineTmp = re.sub(r"like\s+('|\").*?\1","like 'x'",lineTmp) # replace in (value) to in (x) lineTmp = re.sub(r"in\s+\(.*?\)","in (x)",lineTmp) # replace between '...' and '...' to between 'x' and 'x' lineTmp = re.sub(r"between\s+('|\").*?\1\s+and\s+\1.*?\1","between 'x' and 'x' ",lineTmp) # replace between ... and ... to between x and x lineTmp = re.sub(r"between\s+[0-9]+\s+and\s+[0-9]+","between x and x ",lineTmp) # replace limit x,y to limit lineTmp = re.sub(r"limit.*","limit",lineTmp) print lineTmp logFo.close()
python analysis-vc-log.py 3310 | sort | uniq -c | sort -nr |head -n 10
抓取mysql 3310端口，统计执行最频繁的10条SQL。结果我这里就不贴上来了。有兴趣的同学自己测试。
参考资料：
