GROK 表达式
GROK 表达式
常用表达式
标识：USERNAME 或 USER
正则：[a-zA-Z0-9._-]+
名称：用户名
描述：由数字、大小写及特殊字符(._-)组成的字符串
例子：1234、Bob、Alex.Wong
标识：EMAILLOCALPART
正则：[a-zA-Z][a-zA-Z0-9_.+-=:]+
名称：用户名
描述：首位由大小写字母组成，其他位由数字、大小写及特殊字符(_.+-=:)组成的字符串
例子：windcoder、windcoder_com、abc-123
标识：EMAILADDRESS
正则：
[a-zA-Z][a-zA-Z0-9_.+-=:]+@\b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*(.?|\b)
名称：电子邮件
描述：首位由大小写字母组成，其他位由数字、大小写及特殊字符(_.+-=:)组成的字符串，不能匹配qq邮箱
例子：windcoder@abc.com、windcoder_com@gmail.com、abc-123@163.com
标识：INT
正则：(?:[+-]?(?:[0-9]+))
名称：整数
描述：匹配0和正负整数
例子：0、-123、43987
标识：BASE10NUM 或 NUMBER
正则：(?:[+-]?(?:[0-9]+))
名称：十进制数字
描述：包括整数和小数
例子：0、18、5.23
标识：BASE16NUM
正则：(?<![0-9A-Fa-f])(?:[+-]?(?:0x)?(?:[0-9A-Fa-f]+))
名称：十六进制数字
描述：整数
例子：0x0045fa2d、-0x3F8709
标识：WORD
正则：(?<![0-9A-Fa-f])(?:[+-]?(?:0x)?(?:[0-9A-Fa-f]+))
名称：字符串
描述：包括数字和大小写字母
例子：String、65754、ILoveYou
标识：NOTSPACE
正则：\S+
名称：非空格字符串
描述：匹配非空格、至少一次
例子：String、65754、ILoveYou
标识：SPACE
正则：\s*
名称：空格字符串
描述：匹配空格、0次和多次
例子：String、65754、ILoveYou
标识：QUOTEDSTRING 或 QS
正则：(?>(?<!\)(?>"(?>\.|[\"]+)+"|""|(?>'(?>\.|[\']+)+')|''|(?>(?>\\.|[^\\]+)+`)|``))
名称：带引号的字符串
描述：带引号的字符串
例子："This is an apple"、'What is your name?'
标识：UUID
正则：[A-Fa-f0-9]{8}-(?:[A-Fa-f0-9]{4}-){3}[A-Fa-f0-9]{12}
名称：标准UUID
描述：标准UUID 32位
例子：23717d73-0ca5-44fd-a542-7d51ee72e21d
标识：MAC
正则：(?😦?:[A-Fa-f0-9]{2}-){5}[A-Fa-f0-9]{2})
名称：MAC地址
描述：可以是思科（Cisco）设备里的MAC地址，也可以是通用或者Windows系统的MAC地址
例子：23717d73-0ca5-44fd-a542-7d51ee72e21d
标识：IP
正则：(?:%{IPV6}|%{IPV4})
IPV4
?<![0-9])(?:(?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5]))(?![0-9])
IPV6 ((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?
名称：IP地址
描述：IPv4或IPv6地址
例子：127.0.0.1、FE80:0000:0000:0000:AAAA:0000:00C2:0002
标识：HOSTNAME
正则：\b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*(.?|\b)
名称：IP或者主机名称
例子：PC-20210723HTPJ，192.168.0.2
标识：HOSTPORT
正则：%{IPORHOST}:%{POSINT}
名称：主机名(IP)+端口
例子：PC-20210723HTPJ:8080，192.168.0.2:808
标识：PATH
正则：(?:%{UNIXPATH}|%{WINPATH})
名称：路径
描述：Unix系统或者Windows系统里的路径格式
例子：d:\aa\bb\cc 或 /aa/bb/cc
标识：URIPROTO
正则：A-Za-z+
名称：URI协议
例子：http ftp
标识：URIHOST
正则：%{IPORHOST}(?::%{POSINT:port})?
名称：URI主机
例子：windcoder.com、10.0.0.1:22
标识：URIPATH
正则：(?😕[A-Za-z0-9$.+!'(){},~:;=@#%&_-])+
名称：URI路径
例子：//windcoder.com/abc/、/api.php
日期表达式
标识：MONTH
正则：
\b(?:[Jj]an(?:uary|uar)?|[Ff]eb(?:ruary|ruar)?|Mm?r(?:ch|z)?|[Aa]pr(?:il)?|[Mm]a(?:y|i)?|[Jj]un(?:e|i)?|[Jj]ul(?:y|i)?|[Aa]ug(?:ust)?|[Ss]ep(?:tember)?|Oo?t(?:ober)?|[Nn]ov(?:ember)?|[Dd]e(?:c|z)(?:ember)?)\b
名称：月份
描述：英文格式月份
例子：January, Feb, December
标识：MONTHNUM
正则：(?:0?[1-9]|1[0-2])
名称：月份
描述：数字格式月份
例子：1, 2, 12
标识：MONTHDAY
正则：(?:0?[1-9]|1[0-2])
名称：日期
描述：数字格式日期
例子：03、9、31
标识：DAY
正则：
(?:Mon(?:day)?|Tue(?:sday)?|Wed(?:nesday)?|Thu(?:rsday)?|Fri(?:day)?|Sat(?:urday)?|Sun(?:day)?)
名称：星期几名称
描述：英文格式星期几
例子： Monday, Tue, Thu
标识：YEAR
正则：(?>\d\d){1,2}
名称：年份数字
例子： 21、22、23
标识：HOUR
正则：(?:2[0123]|[01]?[0-9])
名称：小时数字
例子： 13、14、23
标识：MINUTE
正则：(?:[0-5][0-9])
名称：分钟数字
例子： 13、14、43
标识：SECOND
正则：(?😦?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)
名称：秒数字
例子： 13、14、43
标识：TIME
正则：
(?!<[0-9])(?:2[0123]|[01]?[0-9])😦?:[0-5][0-9])(?:😦?😦?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))(?![0-9])
名称：时间
例子： 12:32:00
标识：DATE_US
名称：美国时间
例子： 10-01-1892、10/01/1892/
标识：DATE_EU
正则：
(?😦?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])./-./-
名称：欧洲日期格式
例子：01-10-1892、01/10/1882、01.10.1892
标识：ISO8601_TIMEZONE
正则：
(?:Z|[+-]%{HOUR}(?::?%{MINUTE}))
名称：ISO8601时间格式
例子：+10:23、-1023
标识：TIMESTAMP_ISO8601
正则：
%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?
名称：ISO8601时间戳格式
例子：2016-07-03T00:34:06+08:00
标识：DATE
正则：
%{DATE_US}|%{DATE_EU}
名称：美国日期或欧洲日期
例子：10-01-1892、10/01/1892/ 或 01-10-1892、01/10/1882、01.10.1892
标识：DATESTAMP
正则：
%{DATE_US}|%{DATE_EU}
名称：完整日期+时间
例子：07-03-2016 00:34:06
标识：HTTPDATE
正则：
%{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME} %{INT}
名称：http默认日期格式
例子：03/Jul/2016:00:36:53 +0800
预定义完整正则表达式
USERNAME [a-zA-Z0-9._-]+
USER %{USERNAME}
EMAILLOCALPART [a-zA-Z][a-zA-Z0-9_.+-=:]+
EMAILADDRESS %{EMAILLOCALPART}@%{HOSTNAME}
INT (?:[+-]?(?:[0-9]+))
BASE10NUM (?<![0-9.+-])(?>[+-]?(?😦?:[0-9]+(?:.[0-9]+)?)|(?:.[0-9]+)))
NUMBER (?:%{BASE10NUM})
BASE16NUM (?<![0-9A-Fa-f])(?:[+-]?(?:0x)?(?:[0-9A-Fa-f]+))
BASE16FLOAT \b(?<![0-9A-Fa-f.])(?:[+-]?(?:0x)?(?😦?:[0-9A-Fa-f]+(?:.[0-9A-Fa-f]*)?)|(?:.[0-9A-Fa-f]+)))\b
POSINT \b(?:[1-9][0-9]*)\b
NONNEGINT \b(?:[0-9]+)\b
WORD \b\w+\b
NOTSPACE \S+
SPACE \s*
DATA .*?
GREEDYDATA .*
QUOTEDSTRING (?>(?<!\)(?>"(?>\.|[\"]+)+"|""|(?>'(?>\.|[\']+)+')|''|(?>(?>\\.|[^\\]+)+`)|``))
UUID [A-Fa-f0-9]{8}-(?:[A-Fa-f0-9]{4}-){3}[A-Fa-f0-9]{12}
URN, allowing use of RFC 2141 section 2.3 reserved characters
URN urn:[0-9A-Za-z][0-9A-Za-z-]{0,31}😦?:%[0-9a-fA-F]{2}|[0-9A-Za-z()+,.:=@;$_!*'/?#-])+
Networking
MAC (?:%{CISCOMAC}|%{WINDOWSMAC}|%{COMMONMAC})
CISCOMAC (?😦?:[A-Fa-f0-9]{4}.){2}[A-Fa-f0-9]{4})
WINDOWSMAC (?😦?:[A-Fa-f0-9]{2}-){5}[A-Fa-f0-9]{2})
COMMONMAC (?😦?:[A-Fa-f0-9]{2}😃{5}[A-Fa-f0-9]{2})
IPV6 ((([0-9A-Fa-f]{1,4}😃{7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}😃{6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}😃{5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}😃{4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?😦(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}😃{3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}😦(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}😃{2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}😦(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}😃{1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}😦(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(😦((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}😦(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?
IPV4 (?<![0-9])(?😦?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])...)(?![0-9])
IP (?:%{IPV6}|%{IPV4})
HOSTNAME \b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*(.?|\b)
IPORHOST (?:%{IP}|%{HOSTNAME})
HOSTPORT %{IPORHOST}:%{POSINT}
paths (only absolute paths are matched)
PATH (?:%{UNIXPATH}|%{WINPATH})
UNIXPATH (/[[[:alnum:]]_%!$@:.,+~-]*)+
TTY (?:/dev/(pts|tty([pq])?)(\w+)?/?(?:[0-9]+))
WINPATH (?>[A-Za-z]+:|\)(?:\[^\?])+
URIPROTO A-Za-z+
URIHOST %{IPORHOST}(?::%{POSINT:port})?
uripath comes loosely from RFC1738, but mostly from what Firefox
doesn't turn into %XX
URIPATH (?😕[A-Za-z0-9$.+!'(){},~:;=@#%&_-])+
#URIPARAM ?(?:[A-Za-z0-9]+(?:=(?:[&]*))?(?:&(?:[A-Za-z0-9]+(?:=(?:[&]))?)?))?
URIPARAM ?[A-Za-z0-9$.+!'|(){},~@#%&/=:;_?-[]<>]
URIPATHPARAM %{URIPATH}(?:%{URIPARAM})?
URI %{URIPROTO}😕/(?:%{USER}(?::[^@]*)?@)?(?:%{URIHOST})?(?:%{URIPATHPARAM})?
Months: January, Feb, 3, 03, 12, December
MONTH \b(?:[Jj]an(?:uary|uar)?|[Ff]eb(?:ruary|ruar)?|Mm?r(?:ch|z)?|[Aa]pr(?:il)?|[Mm]a(?:y|i)?|[Jj]un(?:e|i)?|[Jj]ul(?:y|i)?|[Aa]ug(?:ust)?|[Ss]ep(?:tember)?|Oo?t(?:ober)?|[Nn]ov(?:ember)?|[Dd]e(?:c|z)(?:ember)?)\b
MONTHNUM (?:0?[1-9]|1[0-2])
MONTHNUM2 (?:0[1-9]|1[0-2])
MONTHDAY (?😦?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])
Days: Monday, Tue, Thu, etc...
DAY (?:Mon(?:day)?|Tue(?:sday)?|Wed(?:nesday)?|Thu(?:rsday)?|Fri(?:day)?|Sat(?:urday)?|Sun(?:day)?)
Years?
YEAR (?>\d\d){1,2}
HOUR (?:2[0123]|[01]?[0-9])
MINUTE (?:[0-5][0-9])
'60' is a leap second in most time standards and thus is valid.
SECOND (?😦?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)
TIME (?!<[0-9])%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9])
datestamp is YYYY/MM/DD-HH:MM:SS.UUUU (or something like it)
DATE_US %{MONTHNUM}[/-]%{MONTHDAY}[/-]%{YEAR}
DATE_EU %{MONTHDAY}[./-]%{MONTHNUM}[./-]%{YEAR}
ISO8601_TIMEZONE (?:Z|[+-]%{HOUR}(?::?%{MINUTE}))
ISO8601_SECOND (?:%{SECOND}|60)
TIMESTAMP_ISO8601 %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?
DATE %{DATE_US}|%{DATE_EU}
DATESTAMP %{DATE}[- ]%{TIME}
TZ (?:[APMCE][SD]T|UTC)
DATESTAMP_RFC822 %{DAY} %{MONTH} %{MONTHDAY} %{YEAR} %{TIME} %{TZ}
DATESTAMP_RFC2822 %{DAY}, %{MONTHDAY} %{MONTH} %{YEAR} %{TIME} %{ISO8601_TIMEZONE}
DATESTAMP_OTHER %{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{TZ} %{YEAR}
DATESTAMP_EVENTLOG %{YEAR}%{MONTHNUM2}%{MONTHDAY}%{HOUR}%{MINUTE}%{SECOND}
Syslog Dates: Month Day HH:MM:SS
SYSLOGTIMESTAMP %{MONTH} +%{MONTHDAY} %{TIME}
PROG [\x21-\x5a\x5c\x5e-\x7e]+
SYSLOGPROG %{PROG:program}(?:[%{POSINT:pid}])?
SYSLOGHOST %{IPORHOST}
SYSLOGFACILITY <%{NONNEGINT:facility}.%{NONNEGINT:priority}>
HTTPDATE %{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME} %{INT}
Shortcuts
QS %{QUOTEDSTRING}
日志格式
SYSLOGBASE %{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:
日志级别
LOGLEVEL ([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo?(?:rmation)?|INFO?(?:RMATION)?|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)
