[operator]ELK6的安装
找了很久才找到一个博客写得比较全面的，FrankDeng
系统环境：CentOS7
相关软件：node-v10.9.0.tar.gz、kibana-6.4.0-linux-x86_64.tar.gz、logstash-6.4.0.zip、elasticsearch-6.4.0.tar.gz，下载地址
先准备三个节点，我这里在每个系统里的/etc/hosts里都添加下了以下内容，在后面的elasticsearch（es）集群的配置里会用到
192.168.133.177 CentOS7-ELk-node21
192.168.133.178 CentOS7-ELk-node22
192.168.133.179 CentOS7-ELk-node23
这里在node21里进行集群的配置
[root@CentOS7-ELk-node21 ~]# grep -v "^#" /opt/module/elk/config/elasticsearch.yml cluster.name: mycluster node.name: CentOS7-ELk-node21 bootstrap.memory_lock: false network.host: 192.168.133.177 discovery.zen.ping.unicast.hosts: ["192.168.133.177", "192.168.133.178", "192.168.133.179"] discovery.zen.minimum_master_nodes: 2
在node22里进行的配置
[root@CentOS7-ELK-node22 ~]# !grep grep -v "^#" /opt/module/elk/config/elasticsearch.yml cluster.name: mycluster node.name: CentOS7-ELk-node22 network.host: 192.168.133.178 discovery.zen.ping.unicast.hosts: ["192.168.133.177", "192.168.133.178", "192.168.133.179"] discovery.zen.minimum_master_nodes: 2
在node23里进行的配置
[root@CentOS7-ELK-node23 ~]# !grep grep -v "^#" /opt/module/elk/config/elasticsearch.yml cluster.name: mycluster node.name: CentOS7-ELk-node23 network.host: 192.168.133.179 discovery.zen.ping.unicast.hosts: ["192.168.133.177", "192.168.133.178", "192.168.133.179"] discovery.zen.minimum_master_nodes: 2
在node22和node23的节点的elasticsearch.yml的文件里将node.name的值改成自动对应的内容然后直接启动es
[root@CentOS7-ELk-node21 ~]# /opt/module/elk/bin/elasticsearch
[root@CentOS7-ELk-node22 ~]# /opt/module/elk/bin/elasticsearch
[root@CentOS7-ELk-node23 ~]# /opt/module/elk/bin/elasticsearch
使用curl命令进行访问看到如下内容说明启动成功
[root@CentOS7-ELk-node21 ~]# curl http://CentOS7-ELk-node21:9200 { "name" : "CentOS7-ELk-node21", "cluster_name" : "mycluster", "cluster_uuid" : "_na_", "version" : { "number" : "6.4.0", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "595516e", "build_date" : "2018-08-17T23:18:47.308994Z", "build_snapshot" : false, "lucene_version" : "7.4.0", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0" }, "tagline" : "You Know, for Search" }
安装个最常用的head插件，下载地址，因为它实际上是一个node.js工作，所以这先来安装个node.js
[root@CentOS7-ELK-node21 ~]# cd /opt/module/node-v10.9.0/
[root@CentOS7-ELK-node21 node-v10.9.0]# ./configure --prefix=/opt/module/node
[root@CentOS7-ELK-node21 node-v10.9.0]# make && make install
经过一次刺激战场经典局鸡屁股的时间。。。
装完成了之后将如下配置添加到/etc/profie里
export NODE_HOME=/opt/module/node
export PATH=$NODE_HOME/bin:$PATH
生效
[root@CentOS7-ELk-node21 node]# source /etc/profile
到elasticsearch-head压缩包所有的目录，解压它后需要安装grunt，先来设置个国内源后再执行
[root@CentOS7-ELk-node21 elasticsearch-head-master]# npm config set registry https://registry.npm.taobao.org
[root@CentOS7-ELk-node21 elasticsearch-head-master]# npm install -g grunt
/opt/module/node/bin/grunt -> /opt/module/node/lib/node_modules/grunt/bin/grunt
+ grunt@1.0.3
added 96 packages from 60 contributors in 5.424s
[root@CentOS7-ELk-node21 elasticsearch-head-master]# npm install
npm WARN deprecated http2@3.3.7: Use the built-in module in node 9.0.0 or newer, instead
npm WARN deprecated coffee-script@1.10.0: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
> phantomjs-prebuilt@2.1.16 install /usr/local/elasticsearch-head-master/node_modules/phantomjs-prebuilt
> node install.js
PhantomJS not found on PATH
Download already available at /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
skipped....
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
added 66 packages from 69 contributors in 8.049s
版本确认一下
[root@CentOS7-ELk-node21 elasticsearch-head-master]# node -v v10.9.0 [root@CentOS7-ELk-node21 elasticsearch-head-master]# npm -v 6.2.0 [root@CentOS7-ELk-node21 elasticsearch-head-master]# grunt -version grunt-cli v1.2.0 grunt v1.0.1
修改head的源码文件
[root@CentOS7-ELk-node21 elasticsearch-head-master]# vim Gruntfile.js
connect: { server: { options: { hostname: '0.0.0.0', port: 9100, base: '.', keepalive: true } } }
修改es的配置文件
[root@CentOS7-ELk-node21 elk]# tail -2 config/elasticsearch.yml http.cors.enabled: true http.cors.allow-origin: "*"
重启es,启动head
[root@CentOS7-ELk-node21 elasticsearch-head-master]# grunt server & [1] 55189 [root@CentOS7-ELk-node21 elasticsearch-head-master]# (node:55189) ExperimentalWarning: The http2 module is an experimental API. Running "connect:server" (connect) task Waiting forever... Started connect web server on http://localhost:9100
在启动中间会遇到类bootstrap checkd failed的报错，在这个错误下面会有错误的详细信息，常见的有两种
ERROR: [2] bootstrap checks failed [1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解决方案：
编辑 /etc/security/limits.conf，追加以下内容； * soft nofile 65536 * hard nofile 65536 此文件修改后需要重新登录用户，才会生效
另外一种：
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解决方案：
编辑 /etc/sysctl.conf，追加以下内容： vm.max_map_count=655360 保存后，执行： sysctl -p 重新启动
启动好之后访问node21的9100端口
再来一个bigdesk插件
[root@CentOS7-ELk-node21 local]# wget https://github.com/hlstudio/bigdesk/archive/master.zip [root@CentOS7-ELk-node21 local]# yum install httpd -y [root@CentOS7-ELk-node21 local]# systemctl start httpd [root@CentOS7-ELk-node21 local]# systemctl enable httpd
bigdesk是一个master.zip的包，解压后将它移动到/var/www/html下面即可
[root@CentOS7-ELk-node21 local]# mv bigdesk-master/ /var/www/html/
使用浏览器访问：http://192.168.133.177/bigdesk-master/_site/#nodes
可以说相当好看了，再来一发IK的分词
[root@CentOS7-ELk-node21 local]# mv bigdesk-master/ /var/www/html/ [root@CentOS7-ELk-node21 local]# cd /opt/module/elk/bin/ [root@CentOS7-ELk-node21 bin]# su es [es@CentOS7-ELk-node21 bin]$ ./elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.4.0/elasticsearch-analysis-ik-6.4.0.zip
本人在安装的时候第一次出现Exception in thread "main" java.net.ConnectException: Connection refused (Connection refused),可怜我的上进心，再来一次
-> Downloading https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.4.0/elasticsearch-analysis-ik-6.4.0.zip [=================================================] 100% @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: plugin requires additional permissions @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ * java.net.SocketPermission * connect,resolve See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html for descriptions of what these permissions allow and the associated risks. Continue with installation? [y/N]y -> Installed analysis-ik
重启es后,测试IK，
[es@CentOS7-ELk-node21 bin]$ curl -XPUT http://192.168.133.177:9200/index
logstash的安装
[root@CentOS7-ELk-node21 local]# unzip filebeat-6.4.0-linux-x86_64.tar.gz [root@CentOS7-ELk-node21 local]# cd logstash-6.4.0
修改logstach的配置文件
[root@CentOS7-ELk-node21 logstash-6.4.0]# vim logstash-simple.conf input { stdin { } } output { elasticsearch { hosts => ["192.168.133.177:9200"] } }
一般在企业中，filebeat做为input，然后output到es中，这里得先构建离线插件
[root@CentOS7-ELk-node21 bin]# ./logstash-plugin prepare-offline-pack logstash-input-beats [root@CentOS7-ELk-node21 bin]# ./logstash-plugin install file:///usr/local/logstash-6.4.0/logstash-offline-plugins-6.4.0.zip
修改配置文件
[root@CentOS7-ELk-node21 config]# vim logstash-sample.conf input { beats { port => 5044 } } output { elasticsearch { hosts => ["http://192.168.133.177:9200"] #stdout { codec => rubydebug } index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" manage_template => false document_type => "%{[@metadata][type]}" #user => "elastic" #password => "changeme" } }
启动logstash
[root@CentOS7-ELk-node21 logstash-6.4.0]# bin/logstash -f config/logstash-sample.conf &
logstash只是一个日志引擎，日志其实都是通过filebeat上报的，我这里在node22上安装filebeat，解压后进行filebeat目录后，修改配置文件
[root@CentOS7-ELK-node22 filebeat-6.4.0-linux-x86_64]# vim filebeat.yml - type: log enabled: true paths: - /var/log/nginx/*.log //如果没有安装nignx的可以这写这一段，这里有设置了上报日志的参数，我的目的只要有日志生成就可以了 - type: log # Change to true to enable this input configuration. enabled: true # Paths that should be crawled and fetched. Glob based paths. paths: - /var/log/*.log output.logstash: # The Logstash hosts hosts: ["192.168.133.177:5044"] //注意这里
启动filebeat
[root@CentOS7-ELK-node22 filebeat-6.4.0-linux-x86_64]# ./filebeat -e -c filebeat.yml
好的，再安装个kibana，这就全部结束了
[root@CentOS7-ELk-node21 kibana-6.4.0-linux-x86_64]# cd /usr/local/kibana-6.4.0-linux-x86_64 [root@CentOS7-ELk-node21 kibana-6.4.0-linux-x86_64]# grep -v "^#\|^$" config/kibana.yml server.host: "192.168.133.177" elasticsearch.url: "http://192.168.133.177:9200"
启动kibana
[root@CentOS7-ELk-node21 kibana-6.4.0-linux-x86_64]# bin/kibana &
看到端口占用5601即可使用浏览器浏览
在第一次浏览的时候会要我们设置索引的关键字，在设置栏的下面也会有相应的提示
设置好之后再回到Discover的菜单就可以看到你的日志了
