tomcat日志采集

1、 采集tomcat确实比之前的需求复杂很多，我在搭建了一个tomcat的环境，然后产生如下报错先贴出来：

Jan 05, 2017 10:53:35 AM org.apache.catalina.core.AprLifecycleListener lifecycleEvent

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

Jan 05, 2017 10:53:35 AM org.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler ["http-bio-8088"]

Jan 05, 2017 10:53:35 AM org.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler ["ajp-bio-8009"]

Jan 05, 2017 10:53:35 AM org.apache.coyote.AbstractProtocol init

SEVERE: Failed to initialize end point associated with ProtocolHandler ["ajp-bio-8009"]

java.net.BindException: Address already in use (Bind failed) <null>:8009

        at org.apache.tomcat.util.net.JIoEndpoint.bind(JIoEndpoint.java:413)

        at org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:665)

        at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:452)

        at org.apache.catalina.startup.Catalina.load(Catalina.java:667)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at org.apache.tomcat.util.net.JIoEndpoint.bind(JIoEndpoint.java:400)

        ... 16 more

 

 

2、 分析我们需要的结构：

通过上面的分析，我们需要的数据有:时间戳、类名、日志信息。

我们需要的操作就是先把相同时间和的多行日志数据合并到同一个事件里面再分析。

 

###提示，因为tomcat日志比较困难，我们可以参考默认的日志结构：

 

[root@monitor patterns]# pwd

/test/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/logstash-patterns-core-4.0.2/patterns

[root@monitor patterns]# cat java

JAVACLASS (?:[a-zA-Z$_][a-zA-Z$_0-9]*\.)*[a-zA-Z$_][a-zA-Z$_0-9]*

#Space is an allowed character to match special cases like 'Native Method' or 'Unknown Source'

JAVAFILE (?:[A-Za-z0-9_. -]+)

#Allow special <init>, <clinit> methods

JAVAMETHOD (?:(<(?:cl)?init>)|[a-zA-Z$_][a-zA-Z$_0-9]*)

#Line number is optional in special cases 'Native method' or 'Unknown source'

JAVASTACKTRACEPART %{SPACE}at %{JAVACLASS:class}\.%{JAVAMETHOD:method}\(%{JAVAFILE:file}(?::%{NUMBER:line})?\)

# Java Logs

JAVATHREAD (?:[A-Z]{2}-Processor[\d]+)

JAVACLASS (?:[a-zA-Z0-9-]+\.)+[A-Za-z0-9$]+

JAVAFILE (?:[A-Za-z0-9_.-]+)

JAVASTACKTRACEPART at %{JAVACLASS:class}\.%{WORD:method}\(%{JAVAFILE:file}:%{NUMBER:line}\)

JAVALOGMESSAGE (.*)

# MMM dd, yyyy HH:mm:ss eg: Jan 9, 2014 7:13:13 AM

CATALINA_DATESTAMP %{MONTH} %{MONTHDAY}, 20%{YEAR} %{HOUR}:?%{MINUTE}(?::?%{SECOND}) (?:AM|PM)

# yyyy-MM-dd HH:mm:ss,SSS ZZZ eg: 2014-01-09 17:32:25,527 -0800

TOMCAT_DATESTAMP 20%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:?%{MINUTE}(?::?%{SECOND}) %{ISO8601_TIMEZONE}

CATALINALOG %{CATALINA_DATESTAMP:timestamp} %{JAVACLASS:class} %{JAVALOGMESSAGE:logmessage}

# 2014-01-09 20:03:28,269 -0800 | ERROR | com.example.service.ExampleService - something compeletely unexpected happened...

TOMCATLOG %{TOMCAT_DATESTAMP:timestamp} \| %{LOGLEVEL:level} \| %{JAVACLASS:class} - %{JAVALOGMESSAGE:logmessage}

 

通过对比我们可以很简单的先把日志相同时间的合并：

[root@controller etc]# cat tomcat.conf

input{stdin{}}

filter {

        multiline {

            pattern => "(^%{CATALINA_DATESTAMP})"

                negate => true

                what => "previous"

        }

        if "_grokparsefailure" in [tags] {

            drop { }

        }

         grok {

                  match => [ "message", "%{CATALINALOG}" ]

        }

        date {

                  match => [ "timestamp", "yyyy-MM-dd HH:mm:ss,SSS Z", "MMM dd, yyyy HH:mm:ss a" ]

        }

    }

output{stdout{codec=>rubydebug}}

 

 

 

##先看测试数据，要比较小一点的：

Jan 05, 2017 10:53:35 AM org.apache.catalina.startup.Catalina load

INFO: Initialization processed in 728 ms

Jan 05, 2017 10:53:35 AM org.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

Jan 05, 2017 10:53:35 AM org.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: Apache Tomcat/7.0.73

测试效果：

    "@timestamp" => 2017-01-05T03:45:46.749Z,

      "@version" => "1",

          "host" => "controller",

       "message" => "Jan 05, 2017 10:53:35 AM org.apache.catalina.startup.Catalina load\nINFO: Initialization processed in 728 ms",

          "tags" => [

        [0] "multiline"

    ]

}

{

    "@timestamp" => 2017-01-05T03:45:46.760Z,

      "@version" => "1",

          "host" => "controller",

       "message" => "Jan 05, 2017 10:53:35 AM org.apache.catalina.core.StandardService startInternal\nINFO: Starting service Catalina",

          "tags" => [

        [0] "multiline"

    ]

}

{

    "@timestamp" => 2017-01-05T03:45:46.780Z,

      "@version" => "1",

          "host" => "controller",

       "message" => "Jan 05, 2017 10:53:35 AM org.apache.catalina.core.StandardEngine startInternal\nINFO: Starting Servlet Engine: Apache Tomcat/7.0.73",

          "tags" => [

        [0] "multiline"

    ]

}

 

 

3、 之前用的都是系统默认的catalina文件管理日志，通过简化的方式我们可以使用log4j的方式。

 

1、 安装log4j:

1、下载与Tomcat相应版本的tomcat-juli.jar 和 tomcat-juli-adapters.jar，及log4j-1.2.17.jar，放在tomcat/lib目录中  附上网址：http://archive.apache.org/dist/tomcat/tomcat-7/v7.0.73/bin/extras/  下载时注意你的 TOMCAT 版本

再将tomcat-juli.jar 复制到tomcat/bin目录中，替换掉原来的

 

2、修改 Tomcat 的 conf/context.xml 文件，将<Context>改为<Context swallowOutput="true">这步很重要。很多人会忘。

 

3、创建log4j.properties放在tomcat/lib中

 

[root@controller lib]# cat log4j.properties

log4j.rootLogger=info,Console,R  

log4j.appender.Console=org.apache.log4j.ConsoleAppender 

log4j.appender.Console.layout=org.apache.log4j.PatternLayout 

#log4j.appender.Console.layout.ConversionPattern=%d [%t] %-5p %c - %m%n 

log4j.appender.Console.layout.ConversionPattern=%d{yy-MM-dd HH:mm:ss} %5p %c{1}:%L - %m%n  

log4j.appender.R=org.apache.log4j.DailyRollingFileAppender 

log4j.appender.R.File=${catalina.home}/logs/tomcat.log  

log4j.appender.R.layout=org.apache.log4j.PatternLayout 

log4j.appender.R.layout.ConversionPattern=%d{yyyy.MM.dd HH:mm:ss} %5p %c{1}(%L):? %m%n     

log4j.logger.org.apache=info, R 

log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost]=DEBUG, R  

log4j.logger.org.apache.catalina.core=info, R 

log4j.logger.org.apache.catalina.session=info, R 

 

4、 重启看到log目录下生成tomcat.log文件说明已经安装成功了。

 

5、 log4j当然可以指定生成日志文件的格式：

log4j.appender.R.layout.ConversionPattern={"debug_level":"%p","debug_timestamp":"%d{ISO8601}","debug_thread":"%t","debug_file":"%F", "debug_line":"%L","debug_message":"%m"}%n

 

##生成日志之后直接解析成json即可。

 

6、 当然也有一个比较优秀的插件，也是我们推荐的方式：log4j-jsonevent-layout：

这玩意儿的作用相当于我们在nginx中干的事儿，直接将log4j的日志格式定义成json的，有助于性能提升~

 

7、安装：

先上传一下几个包，已经从官方打包了几个jar包，确实的话很容易失败和报错：

commons-lang-2.6.jar

jsonevent-layout-1.8-SNAPSHOT.jar

json-smart-1.1.1.jar

 

 

7、 修改log4j.properties，直接把日志发送到Logstash：

 

[root@controller lib]# cat log4j.properties

log4j.rootCategory=info, RollingLog    ###为了方便出日志我们用Info，线上大家可以用WARN

log4j.appender.RollingLog=org.apache.log4j.DailyRollingFileAppender

log4j.appender.RollingLog.Threshold=TRACE

log4j.appender.RollingLog.File=${catalina.home}/logs/api.log

log4j.appender.RollingLog.DatePattern=.yyyy-MM-dd

log4j.appender.RollingLog.layout=net.logstash.log4j.JSONEventLayoutV1

 

###备注：重启后我们生成了相关日志在api.log下面，下面我们用json格式可以直接解析他了。

看一下我们需要做的匹配文件：

[root@controller etc]# cat tomcat_log4j_layout.conf

input {

  file {

    codec => json

    path => "/usr/local/src/apache-tomcat-7.0.73/logs/api.log"

    type => "log4j"

    start_position => "beginning"

    sincedb_path => "/dev/null"

  }

}

output{

   if[type] == "log4j"{

    redis {

      host => "192.168.0.46"

      port => 6379

      data_type => "list"

      key => "logstash:log4j"

    }

  }

}

本文出自：http://www.roncoo.com/course/view/3c0710458fe347c2a0b31135bbbcb57b