logstash定义表达式

例如： echo '[2018/02/02 08:34:43.032]' >> tomcat_catalina.out

过滤方式

"message" => "\[(?<log_time>%{NOTSPACE}%{SPACE}%{NOTSPACE})\]"

 

{
"log_time" => "2018/02/02 08:34:43.032",
"message" => "[2018/02/02 08:34:43.032]",
"prospector" => {
"type" => "log"
},
"source" => "/ali/logs/tomcat_catalina.out",
"offset" => 2313,
"@version" => "1",
"@timestamp" => 2018-02-03T07:06:48.983Z,
"fields" => {
"tag" => "log1"
},
"beat" => {
"name" => "iZwz9amcsywc3lxhketqdpZ",
"hostname" => "iZwz9amcsywc3lxhketqdpZ",
"version" => "6.1.2"
}
}

。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。

自定义grok表达式

语法：(?<field_name>the pattern here)

eg：

grok{
　　match=>{
　　　　"message"=>"%{IP:clientip}\s+(?<mypattern>[A-Z]+)"
　　}
}

rs:

{
　　"message" => "12.12.12.12 ABC",
　　"@version" => "1",
　　"@timestamp" => "2016-03-30T03:22:04.466Z",
　　"host" => "master",
　　"clientip" => "12.12.12.12",
　　"mypattern" => "ABC"
}