ELK6.6.0+filebeat6.6.0部署
elastic不能用root用户去启动，否则会报错，所以创建elastic用户ES集群部署
1.创建elastic用户
$ useradd elastic $ passwd elastic
2..部署JDK环境
$ tar xvf jdk-8u191-linux-x64.tar.gz -C /usr/local/ $ mv /usr/local/jdk1.8.0_191/ /usr/local/java $ vim /etc/profile.d/elk.sh export JAVA_HOME=/usr/local/java export PATH=$JAVA_HOME:$PATH $ source /etc/profile
3.下载elastic源码包
$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.0.tar.gz
4.解压elastic
$ tar xvf elasticsearch-6.6.0.tar.gz -C /usr/local/ $ mv /usr/local/elasticsearch-6.6.0/ /usr/local/elastic $ chown -R elastic /usr/local/elastic/
5.修改elastic内存配置
elasticsearch6.6.0默认内存需要1G，如果没有1G内存可能会报错，如果内存不够则需要修改配置文件
$ vim /usr/local/elastic/config/jvm.options -Xms512m -Xmx512m
6.修改elastic配置文件
$ mkdir /data/es-data -p $ mkdir /var/log/elastic/ $ vim /usr/local/elastic/config/elasticsearch.yml # 组名自定义，但是同一个组，组名必须一致 cluster.name: my-application # 节点名称，建议和主机名一致 node.name: elastic # 数据存放目录 path.data: /data/es-data # 日志存放路径 path.logs: /var/log/elastic # 锁住内存，bubei 使用到交换分区去 bootstrap.memory_lock: true # 由于只部署两个节点，因此设置为1，否则当master宕机，将无法重新选取master discovery.zen.minimum_master_nodes: 1 # 网络设置 network.host: 0.0.0.0 # 端口 http.port: 9200 # 从节点配置 # 关闭多播 discovery.zen.ping.unicast.enabled: false # 发单播，ip地址是master和自己 discovery.zen.ping.unicast.hosts: ["192.168.1.131", "192.168.1.164"]
6.启动elastic
$ chown -R elastic /data/ $ chown -R elastic /var/log/elastic/ $ su - elastic $ /usr/local/elastic/bin/elasticsearch
7.测试
8.报错
[1]: max file descriptors [65535] for elasticsearch process is too low, increase to at least [65536]
#切换回root $ vim /etc/security/limits.conf # elastic是用户 elastic soft nofile 65536 elastic hard nofile 65536 # 登录elastic查看 $ ulimit -Hn 65536
[2]: memory locking requested for elasticsearch process but memory is not locked
$ vim /etc/security/limits.conf elastic - memlock unlimited
[3]: max number of threads [3802] for user [elastic] is too low, increase to at least [4096]
$ vim /etc/security/limits.d/20-nproc.conf elastic - nproc 4096
[4]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
$ vim /etc/sysctl.conf vm.max_map_count=655360 $ sysctl -p
插件安装
elastic5.0之后，head插件需要独立安装
1.head插件
# 安装NodeJS
$ wget https://npm.taobao.org/mirrors/node/latest-v4.x/node-v4.5.0-linux-x64.tar.gz
$ tar -zxvf node-v4.5.0-linux-x64.tar.gz -C /usr/local/
$ mv /usr/local/nodenode-v4.5.0-linux-x64 /usr/local/node
$ vim /etc/profile
export NODE_HOME=/usr/local/node
export PATH=$PATH:$NODE_HOME/bin/
export NODE_PATH=$NODE_HOME/lib/node_modules
$ source /etc/profile
# 安装npm
$ npm install -g cnpm --registry=https://registry.npm.taobao.org
# 安装grunt
$ npm install -g grunt
$ npm install -g grunt-cli --registry=https://registry.npm.taobao.org --no-proxy
# 确认版本
node -v
v9.5.0
$ npm -v
5.6.0
$ grunt -version
grunt-cli v1.3.2
grunt v1.0.1
$ wget https://github.com/mobz/elasticsearch-head/archive/master.zip
$ unzip master.zip
$ cd elasticsearch-head-master/
# npm install -g cnpm --registry=https://registry.npm.taobao.org
$ npm install
# 修改es的配置文件
# head插件可以访问es
$ vim /usr/local/elasticsearch-6.6.0/config/elasticsearch.yml
http.cors.enabled: true
http.cors.allow-origin: "*"
# 修改head插件配置文件
$ vim Gruntfile.js
# 增加一行hostname
connect: {
server: {
options: {
hostname: '0.0.0.0',
port: 9100,
base: '.',
keepalive: true
}
}
}
$ vim elasticsearch-head-master/_site/app.js
# 修改localhost为es的ip地址
# this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://localhost:9200";
this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://192.168.1.126:9200";
# 重启es
$ grunt server
Filebeat+Logstash部署
1.Filebeat部署
a.下载filebeat源码包
$ wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.6.0-linux-x86_64.tar.gz
b.解压源码包
$ tar xvf filebeat-6.6.0-linux-x86_64.tar.gz -C /usr/local/ $ mv /usr/local/filebeat-6.6.0-linux-x86_64/ /usr/local/filebeat
c.修改配置文件
$ vim /usr/local/filebeat/filebeat.yml
#=========================== Filebeat inputs =============================
filebeat.inputs:
- type: log
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
# 指定读取文件的位置
paths:
- /var/log/*.log
# 只发送包含ERR,WARN字样的日志
# include_lines: ['^ERR', '^WARN']
# 不发送包含OK字样的日志
# exclude_lines: ["^OK"]
# 定义写到ES时的type值
# document_type: "test"
# 输出的位置，直接输出到elastic的话，选第一个，输出到logstash的话，选第二个
#output.elasticsearch:
# Array of hosts to connect to.
#hosts: ["localhost:9200"]
output.logstash:
# The Logstash hosts
hosts: ["localhost:5044"]
d.启动filebeat
$ vim /etc/profile.d/elk.sh export PATH=$PATH:/usr/local/filebeat/ $ source /etc/profile $ filebeat -e -c /usr/local/filebeat/filebeat.yml
2.Logstash部署
a.部署JDK环境
$ tar xvf jdk-8u191-linux-x64.tar.gz -C /usr/local/ $ mv /usr/local/jdk1.8.0_191/ /usr/local/java $ vim /etc/profile.d/elk.sh export JAVA_HOME=/usr/local/java export PATH=$JAVA_HOME:$PATH:/usr/local/filebeat/ $ source /etc/profile
b.下载Logstash源码包
$ wget https://artifacts.elastic.co/downloads/logstash/logstash-6.6.0.tar.gz
c.解压源码包
$ tar xvf logstash-6.6.0.tar.gz -C /usr/local/ $ mv /usr/local/logstash-6.6.0/ /usr/local/logstash
d.修改配置文件
input {
beats {
port => 5044
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => ["http://192.168.1.126:9200"]
index => "test"
}
}
Kibana部署
1. 下载Kibana
$ wget https://artifacts.elastic.co/downloads/kibana/kibana-6.6.0-linux-x86_64.tar.gz
2. 解压源码包
$ tar xvf kibana-6.6.0-linux-x86_64.tar.gz -C /usr/local/ $ mv /usr/local/kibana-6.6.0-linux-x86_64/ /usr/local/kibana
3. 修改配置文件
$ vim /usr/local/kibana/config/kibana.yml server.host: "192.168.1.130" elasticsearch.hosts: ["http://192.168.1.126:9200"]
4.启动Kibana
$ /usr/local/kibana/bin/kibana
访问 192.168.1.130:5601
PS：如有错误，欢迎指正
