使用filebeat收集日志
目录
一、初始化环境
1.1 环境准备
| 系统版本 | 主机名 | IP地址 | 服务 |
|---|---|---|---|
| Centos 7.5 | node | 192.168.1.1 | es、kibana |
| Centos 7.5 | test | 192.168.1.2 | filebeat |
1.2 安装es
$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.0.rpm
$ yum -y install elasticsearch-6.6.0.rpm
$ egrep -v '#|^$' /etc/elasticsearch/elasticsearch.yml
node.name: node
path.data: /elk/data
path.logs: /elk/log
network.host: 192.168.1.1
http.port: 9200
$ mkdir -p /elk/{data,log}
$ chown elasticsearch.elasticsearch /elk -R
$ systemctl start elasticsearch
$ ss -lnt | grep 9200
LISTEN 0 128 ::ffff:192.168.1.1:9200 :::*
1.3 安装Kibana
$ wget https://artifacts.elastic.co/downloads/kibana/kibana-6.6.0-x86_64.rpm
$ yum -y install kibana-6.6.0-x86_64.rpm
$ egrep -v '#|^$' /etc/kibana/kibana.yml
server.port: 5601
server.host: "192.168.1.1"
server.name: "node"
elasticsearch.hosts: ["http://192.168.1.1:9200"]
kibana.index: ".kibana"
$ systemctl start kibana
$ ss -lnt | grep 5601
LISTEN 0 128 192.168.1.1:5601 *:*
二、收集nginx日志
由于nginx的日志格式不是json的，收集起来也无法立即定位到关键信息，所以就直接转为json格式并进行拆分！
$ vim /etc/yum.repos.d/nginx.repo
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=1
$ yum -y install nginx httpd-tools
$ vim /etc/nginx/nginx.conf
#添加以下内容将其日志格式转换为json格式
log_format json '{ "@time_local": "$time_local", '
'"remote_addr": "$remote_addr", '
'"referer": "$http_referer", '
'"request": "$request", '
'"status": $status, '
'"bytes": $body_bytes_sent, '
'"agent": "$http_user_agent", '
'"x_forwarded": "$http_x_forwarded_for", '
'"up_addr": "$upstream_addr",'
'"up_host": "$upstream_http_host",'
'"up_resp_time": "$upstream_response_time",'
'"request_time": "$request_time"'
' }';
access_log /var/log/nginx/access.log json;
$ nginx -t
$ nginx
$ wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.6.0-x86_64.rpm
$ yum -y install filebeat-6.6.0-x86_64.rpm
$ rm -rf /etc/filebeat/filebeat.yml
$ vim /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- /var/log/nginx/access.log
json.keys_under_root: true
json.overwrite_keys: true
tags: ["access"]
- type: log
enabled: true
paths:
- /var/log/nginx/error.log
tags: ["error"]
output.elasticsearch:
hosts: ["192.168.1.1:9200"]
indices:
- index: "nginx-acess-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
tags: "access"
- index: "nginx-error-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
tags: "error"
setup.template.name: "nginx"
setup.template.pattern: "nginx-*"
setup.template.enabled: false
setup.template.overwrite: true
$ systemctl start filebeat
$ ab -n 1000 -c 100 http://192.168.1.2/
$ ab -n 1000 -c 100 http://192.168.1.2/test
kibana自行添加索引：
这样就可以将日志拆分成好几个字段，便于查找关键信息！
三、收集tomcat日志
tomcat日志默认情况下虽然是json格式，但是并没有进行拆分，所以，需要进行以下配置进行拆分！
$ yum -y install tomcat tomcat-webapps tomcat-admin-webapps tomcat-docs-webapp tomcat-javadoc
$ vim /etc/tomcat/server.xml
#139行原本的删除，添加以下内容：
pattern="{"client":"%h", "client user":"%l", "authenticated":"%u", "access time":"%t", "method":"%r", "status":"%s", "send bytes":"%b", "Query?string":"%q", "partner":"%{Referer}i", "Agent version":"%{User-Agent}i"}"/>
$ systemctl start tomcat
$ vim /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- /var/log/tomcat/localhost_access_log.*.txt
json.keys_under_root: true
json.overwrite_keys: true
tags: ["tomcat"]
output.elasticsearch:
hosts: ["192.168.1.1:9200"]
indices:
- index: "tomcat-access-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
tags: "tomcat"
setup.template.name: "tomcat"
setup.template.pattern: "tomcat-*"
setup.template.enabled: false
setup.template.overwrite: true
$ systemctl restart filebeat
自行访问tomcat，使其产生日志！
自行添加tomcat索引！
四、收集ES日志
因为ES的日志有点不同，需要用到多行匹配模式！直接在node主机上安装filebeat进行操作！
$ yum -y install filebeat-6.6.0-x86_64.rpm
$ vim /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- /elk/log/elasticsearch.log
tags: ["es"]
multiline.pattern: '^\['
multiline.negate: true
multiline.match: after
output.elasticsearch:
hosts: ["192.168.1.1:9200"]
indices:
- index: "es-java-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
tags: "es"
setup.template.name: "es"
setup.template.pattern: "es-java-*"
setup.template.enabled: false
setup.template.overwrite: true
$ systemctl start filebeat
想办法让ES产生一些错误日志！
自行创建索引。
这就是ES错误的特点，所以需要使用以上多行合并技术！
五、收集docker容器日志
如果需要实现安装docker、docker-compose！
$ yum install -y yum-utils device-mapper-persistent-data lvm2
$ yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
$ yum install -y docker-ce-18.09.0-3.el7 docker-ce-cli-18.09.0-3.el7 containerd.io-1.2.0-3.el7
$ systemctl daemon-reload && systemctl start docker
$ curl -L https://github.com/docker/compose/releases/download/1.25.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
$ chmod +x /usr/local/bin/docker-compose
$ mkdir compose && cd compose
[root@test compose]# vim docker-compose.yaml
version: '3'
services:
nginx:
image: nginx
labels:
service: nginx
logging:
options:
labels: "service"
ports:
- "80:80"
db:
image: nginx
labels:
service: db
logging:
options:
labels: "service"
ports:
- "3306:80"
#使用一个nginx镜像模拟两种服务
[root@test compose]# docker-compose up
$ vim /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
paths:
- /var/lib/docker/containers/*/*-json.log
json.keys_under_root: true
json.overwrite_keys: true
output.elasticsearch:
hosts: ["192.168.1.1:9200"]
indices:
- index: "docker-nginx-access-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
attrs.service: "nginx"
stream: "stdout"
- index: "docker-nginx-error-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
attrs.service: "nginx"
stream: "stderr"
- index: "docker-db-access-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
attrs.service: "db"
stream: "stdout"
- index: "docker-db-error-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
attrs.service: "db"
stream: "stderr"
setup.template.name: "docker"
setup.template.pattern: "docker-*"
setup.template.enabled: false
setup.template.overwrite: true
$ systemctl restart filebeat
访问容器中的服务，使其产生日志！
自行添加索引！
六、使用filebeat自带模块进行监控
filebeat自带了很多模块，这里以nginx为例！
$ vim /etc/filebeat/filebeat.yml
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
reload.period: 10s
output.elasticsearch:
hosts: ["192.168.1.1:9200"]
indices:
- index: "nginx-access-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
fileset.name: "access"
- index: "nginx-error-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
fileset.name: "error"
setup.template.name: "nginx"
setup.template.pattern: "nginx-*"
setup.template.enabled: false
setup.template.overwrite: true
$ filebeat modules enable nginx
Enabled nginx
$ vim /etc/filebeat/modules.d/nginx.yml
- module: nginx
# Access logs
access:
enabled: true
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
var.paths: ["/var/log/nginx/access.log"]
# Error logs
error:
enabled: true
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
var.paths: ["/var/log/nginx/error.log"]
$ yum -y install nginx
$ nginx
ES服务器需要安装以下两个插件才支持此功能！
$ /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-user-agent
$ /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-geoip
$ systemctl restart elasticsearch
安装完成之后：
$ filebeat setup
$ systemctl restart filebeat
$ ab -c 100 -n 100 http://192.168.1.2/
添加错误日志索引：
七、Kibana的x-pack监控
*************** 当你发现自己的才华撑不起野心时，就请安静下来学习吧！***************
