elk收集tomcat的日志
logstash收集tomcat的日志 不要修改下tomcat中server.xml的日志格式，否则tomcat无法启动，试过多次，不行，就用自带的日志让logstash去收集 首先给tomcat日志授权，否则logstash无权读取日志文件 [root@bogon bin]# ls -l /usr/local/tomcat/logs/ total 208 -rw-r----- 1 root root 101390 Feb 12 02:42 catalina.2019-02-12.log -rw-r----- 1 root root 101500 Feb 12 02:42 catalina.out -rw-r----- 1 root root 0 Feb 12 01:40 host-manager.2019-02-12.log -rw-r----- 1 root root 2454 Feb 12 02:42 localhost.2019-02-12.log -rw-r----- 1 root root 2421 Feb 12 03:07 localhost_access_log.2019-02-12.txt -rw-r----- 1 root root 0 Feb 12 01:40 manager.2019-02-12.log [root@bogon bin]# chmod 777 /usr/local/tomcat/logs/* [root@bogon bin]# ls -l /usr/local/tomcat/logs/ total 208 -rwxrwxrwx 1 root root 101390 Feb 12 02:42 catalina.2019-02-12.log -rwxrwxrwx 1 root root 101500 Feb 12 02:42 catalina.out -rwxrwxrwx 1 root root 0 Feb 12 01:40 host-manager.2019-02-12.log -rwxrwxrwx 1 root root 2454 Feb 12 02:42 localhost.2019-02-12.log -rwxrwxrwx 1 root root 2421 Feb 12 03:07 localhost_access_log.2019-02-12.txt -rwxrwxrwx 1 root root 0 Feb 12 01:40 manager.2019-02-12.log 现在要收集localhost_access_log.2019-02-12.txt日志内容，步骤如下： 配置logstash的语法规则，来收集tomcat日志： cat> /home/logstash-6.3.0/config/tomcat_test.conf<<EOF input { file { path => ["/usr/local/tomcat/logs/localhost_access_log.2019-02-12.txt"] type => "tomcat_log" start_position => "beginning" codec => json } } filter { date { match => [ "timestamp" , "YYYY-MM-dd HH:mm:ss" ] } } output { elasticsearch { hosts => ["192.168.0.91:9200"] index => "tomcat-pc-%{+YYYY.MM.dd}" } stdout { codec => rubydebug } } EOF 然后测试下有没有语法错误 /home/logstash-6.3.0/bin/logstash -t -f /home/logstash-6.3.0/config/tomcat_test.conf --config.test_and_exit 开始启动 nohup /home/logstash-6.3.0/bin/logstash -f /home/logstash-6.3.0/config/tomcat_test.conf & 启动非常慢，需要几分钟 查看logstash进程 [root@bogon tomcat]# ps -ef |grep logstash root 22853 5194 99 03:03 pts/1 00:02:02 /bin/java -Xms1g -Xmx1g -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.compile.invokedynamic=true -Djruby.jit.threshold=0 -XX:+HeapDumpOnOutOfMemoryError -Djava.security.egd=file:/dev/urandom -cp /home/logstash-6.3.0/logstash-core/lib/jars/commons-compiler-3.0.8.jar:/home/logstash-6.3.0/logstash-core/lib/jars/google-java-format-1.1.jar:/home/logstash-6.3.0/logstash-core/lib/jars/guava-19.0.jar:/home/logstash-6.3.0/logstash-core/lib/jars/jackson-annotations-2.9.5.jar:/home/logstash-6.3.0/logstash-core/lib/jars/jackson-core-2.9.5.jar:/home/logstash-6.3.0/logstash-core/lib/jars/jackson-databind-2.9.5.jar:/home/logstash-6.3.0/logstash-core/lib/jars/jackson-dataformat-cbor-2.9.5.jar:/home/logstash-6.3.0/logstash-core/lib/jars/janino-3.0.8.jar:/home/logstash-6.3.0/logstash-core/lib/jars/jruby-complete-9.1.13.0.jar:/home/logstash-6.3.0/logstash-core/lib/jars/log4j-api-2.9.1.jar:/home/logstash-6.3.0/logstash-core/lib/jars/log4j-core-2.9.1.jar:/home/logstash-6.3.0/logstash-core/lib/jars/log4j-slf4j-impl-2.9.1.jar:/home/logstash-6.3.0/logstash-core/lib/jars/logstash-core.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.core.commands-3.6.0.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.core.contenttype-3.4.100.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.core.expressions-3.4.300.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.core.filesystem-1.3.100.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.core.jobs-3.5.100.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.core.resources-3.7.100.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.core.runtime-3.7.0.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.equinox.app-1.3.100.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.equinox.common-3.6.0.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.equinox.preferences-3.4.1.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.equinox.registry-3.5.101.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.jdt.core-3.10.0.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.osgi-3.7.1.jar:/home/logstash-6.3.0/logstash-core/lib/jars/org.eclipse.text-3.5.101.jar:/home/logstash-6.3.0/logstash-core/lib/jars/slf4j-api-1.7.25.jar org.logstash.Logstash -t -f /home/logstash-6.3.0/config/tomcat_test.conf --config.test_and_exit root 22916 18873 0 03:04 pts/3 00:00:00 grep logstash 查看elk端口 [root@bogon tomcat]# netstat -tnlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 192.168.0.91:5601 0.0.0.0:* LISTEN 15053/node tcp 0 0 192.168.0.91:9100 0.0.0.0:* LISTEN 14232/grunt tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 943/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1173/master tcp6 0 0 127.0.0.1:9600 :::* LISTEN 23343/java tcp6 0 0 127.0.0.1:8005 :::* LISTEN 21582/java tcp6 0 0 :::8009 :::* LISTEN 21582/java tcp6 0 0 :::8080 :::* LISTEN 21582/java tcp6 0 0 192.168.0.91:9200 :::* LISTEN 14105/java tcp6 0 0 192.168.0.91:9300 :::* LISTEN 14105/java tcp6 0 0 :::22 :::* LISTEN 943/sshd tcp6 0 0 ::1:25 :::* LISTEN 1173/master 可以看到logstash启动后随机生成了一个9600端口
elasticsearch的head插件查看
先查看概览：
再查看数据浏览：
配置kibana查看日志
首先查看索引：
cat /home/logstash-6.3.0/config/tomcat_test.conf
input {
file {
path => ["/usr/local/tomcat/logs/localhost_access_log.2019-02-12.txt"]
type => "tomcat_log"
start_position => "beginning"
codec => json
}
}
filter {
date {
match => [ "timestamp" , "YYYY-MM-dd HH:mm:ss" ]
}
}
output {
elasticsearch {
hosts => ["192.168.0.91:9200"]
index => "tomcat-pc-%{+YYYY.MM.dd}"
}
stdout {
codec => rubydebug
}
}
里面的index => "tomcat-pc-%{+YYYY.MM.dd}" 就是索引，这个索引一会配置kibana时候需要用到。
配置Kibana获得日志
参照文档： https://www.cnblogs.com/cjsblog/p/9476813.html https://blog.csdn.net/ZHANG_H_A/article/details/53129565 http://blog.51cto.com/jinlong/2055379
