ELK单机部署

 

##################################### 7.16版本###########################################################

 

 

 

ELK + filebeat 日志可视化搭建

系统： centos7
内存： 16G

一 配置yum源

[elasticsearch]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md

[kibana-7.x]
name=Kibana repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

[logstash-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

[elastic-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

 

二 安装ELK filebeat

yum install elasticsearch logstash kibana filebeat -y

三 修改配置文件

filebeat elasticsearch kibana 这三个配置文件修改主要的就可以了

重点logstash的配置
vim /etc/logstash/conf.d/logstash.conf

# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
beats {
port => 5044
}
}

output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
#user => "elastic"
#password => "changeme"
}
}

 

 

 

 

 

filebeat logstash 自定义索引配置

 

filebeat.yml

 

filebeat.inputs:
- type: log
enabled: true
paths:
- /var/log/messages
fields:
service: message
- type: log
enabled: true
paths:
- /var/log/yum.log
fields:
service: yum
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
setup.template.settings:
index.number_of_shards: 1
setup.kibana:
output.logstash:
hosts: ["localhost:5044"]
processors:
- add_host_metadata:
when.not.contains.tags: forwarded
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~

 

 

 

 

 

logstash.conf

input {
beats {
# index => "syslog"
port => 5044
}
}

#filter {
# grok { match => [ "message", "%{HTTPDATE:[@metadata][timestamp]}" ] }
# date { match => [ "[@metadata][timestamp]", "dd/MMM/yyyy:HH:mm:ss Z" ] }
#}

output {
if [fields][service] == "message" {
elasticsearch {
hosts => ["localhost:9200"]
index => "test-message-%{+YYYY.MM}"
}
} else if [fields][service] == "yum" {
elasticsearch {
hosts => ["localhost:9200"]
index => "test-yum-%{+YYYY.MM}"
}
}
}