ELK安装部署
一、ELK简介
ELK是Elasticsearch、Logstash、Kibana的简称，这三者是核心套件，但并非全部。
Elasticsearch是实时全文搜索和分析引擎，提供搜集、分析、存储数据三大功能；是一套开放REST和JAVA API等结构提供高效搜索功能，可扩展的分布式系统。它构建于Apache Lucene搜索引擎库之上。
Logstash是一个用来搜集、分析、过滤日志的工具。它支持几乎任何类型的日志，包括系统日志、错误日志和自定义应用程序日志。它可以从许多来源接收日志，这些来源包括 syslog、消息传递（例如 RabbitMQ）和JMX，它能够以多种方式输出数据，包括电子邮件、websockets和Elasticsearch。
Kibana是一个基于Web的图形界面，用于搜索、分析和可视化存储在 Elasticsearch指标中的日志数据。它利用Elasticsearch的REST接口来检索数据，不仅允许用户创建他们自己的数据的定制仪表板视图，还允许他们以特殊的方式查询和过滤数据。
Beats：filebeat:日志文件，metricbeat:类似cpu/内存度量数据，packetbeat:网络数据，winlogbeat:windows数据，heartbeat：健康检查
官网地址：https://www.elastic.co/cn/
二、ELK安装环境
| 服务器IP地址 | 操作系统版本 | 节点名称 | 备注 | ||
| 192.168.8.120 | RHEL6.6_X86_64 | kibana节点 | elasticsearch | ||
| 192.168.8.121 | RHEL6.6_X86_64 | logstash节点 | elasticsearch | ||
| 192.168.8.122 | RHEL6.6_X86_64 | elasticsearch | |||
在主机hosts文件中配置主机IP解析
192.168.8.120 master-node
192.168.8.121 data-node1
192.168.8.122 data-node2
1.软件下载
elasticsearch下载：https://www.elastic.co/cn/downloads/elasticsearch
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.0-linux-x86_64.tar.gz
logstash下载：https://www.elastic.co/cn/downloads/logstash
kibana下载：https://www.elastic.co/cn/downloads/kibana
https://artifacts.elastic.co/downloads/kibana/kibana-7.5.0-linux-x86_64.tar.gz
JDK下载：https://download.oracle.com/otn/java/jdk/8u231-b11/
2.基础架构图
3.普通用户创建
[root@Linux7 soft]# useradd elkuser
[root@Linux7 soft]# passwd elkuser
Changing password for user elkuser.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
4.jdk安装
[root@Linux7 soft]# rpm -ivh jdk-8u231-linux-x64.rpm
5.操作系统环境配置
[root@Linux7 ~]# vi /etc/security/limits.conf
* soft nproc 204800
* hard nproc 204800
* soft nofile 655360
* hard nofile 655360
* soft memlock unlimited
* hard memlock unlimited
[root@Linux7 ~]# vi /etc/sysctl.conf
vm.max_map_count=1024000
###RHEL6.0:[root@Linux6 data]# vi /etc/security/limits.d/90-nproc.conf 增加配置内容：* soft nproc 65535
###RHEL7.0：[root@Linux7 ~]# vi /etc/security/limits.d/20-nproc.conf 增加配置内容：* soft nproc 65535
[root@Linux7 ~]# sysctl -p
6.elasticsearch安装
[root@Linux7 soft]# tar -xzvf elasticsearch-7.5.0-linux-x86_64.tar.gz
[root@Linux7 soft]# mv elasticsearch-7.5.0 /usr/local/elasticsearch7.5
[root@Linux7 soft]# chown -R elkuser.elkuser /usr/local/elasticsearch7.5
elasticsearch 参数配置：
[elkuser@Linux7 elasticsearch7.5]$ vi config/elasticsearch.yml
network.host: 192.168.8.120
cluster.initial_master_nodes: ["node-1"]
elasticsearch 启动：
[root@Linux7 elasticsearch7.5]# su - elkuser
[elkuser@Linux7 ~]$ cd /usr/local/elasticsearch7.5/
[elkuser@Linux7 elasticsearch7.5]$ bin/elasticsearch
elasticsearch 启动状态查看：
[2019-12-13T11:03:45,087][INFO ][o.e.h.AbstractHttpServerTransport] [Linux7] publish_address {10.231.35.131:9200}, bound_addresses {10.231.35.131:9200}
[2019-12-13T11:03:45,088][INFO ][o.e.n.Node ] [Linux7] started
[2019-12-13T11:03:45,336][INFO ][o.e.l.LicenseService ] [Linux7] license [149d87ef-d8ca-498a-aef8-5f79f95df570] mode [basic] - valid
[2019-12-13T11:03:45,337][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [Linux7] Active license is now [BASIC]; Security is disabled
[2019-12-13T11:03:45,353][INFO ][o.e.g.GatewayService ] [Linux7] recovered [0] indices into cluster_state
elasticsearch 访问：
浏览器地址栏输入：192.168.8.120:9200
7.kibana安装
[root@Linux7 soft]# tar -xzvf kibana-7.5.0-linux-x86_64.tar.gz
[root@Linux7 soft]# mv kibana-7.5.0-linux-x86_64 /usr/local/kibana7.5
[root@Linux7 soft]# chown -R elkuser:elkuser /usr/local/kibana7.5/
kibana配置修改：
[elkuser@Linux7 kibana7.5]$ vi config/kibana.yml
server.host: "192.168.8.120"
elasticsearch.hosts: ["http://192.168.8.120:9200"]
kibana启动：
[root@Linux7 soft]# su - elkuser
[elkuser@Linux7 kibana7.5]$ bin/kibana
kibana访问：
http://192.168.8.120:5601
三、总结
1.安装elastic stack的需要jdk1.8版本环境
2.运行elastic stack的用户需要非root
如果是不root模式运行，报错信息如下
[root@Linux7 elasticsearch7.5]# bin/elasticsearch
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2019-12-13T10:34:41,302][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [Linux7] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.5.0.jar:7.5.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.5.0.jar:7.5.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.5.0.jar:7.5.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.5.0.jar:7.5.0]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.5.0.jar:7.5.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.5.0.jar:7.5.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.5.0.jar:7.5.0]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:105) ~[elasticsearch-7.5.0.jar:7.5.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) ~[elasticsearch-7.5.0.jar:7.5.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.5.0.jar:7.5.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.5.0.jar:7.5.0]
... 6 more
3.elastic stack分为生产（配置network.host为非localhost）和非生产模式，其中生产模式要对运行环境做检测，比如用户的会话限制、cluster.initail_master_nodes等参数。
修改config/elasticsearch.yml配置文件network.host为非localhost模式后，启动报错信息如下
ERROR: [2] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[2019-12-13T10:46:44,400][INFO ][o.e.n.Node ] [Linux7] stopping ...
[2019-12-13T10:46:44,427][INFO ][o.e.n.Node ] [Linux7] stopped
[2019-12-13T10:46:44,428][INFO ][o.e.n.Node ] [Linux7] closing ...
[2019-12-13T10:46:44,471][INFO ][o.e.n.Node ] [Linux7] closed
[2019-12-13T10:46:44,476][INFO ][o.e.x.m.p.NativeController] [Linux7] Native controller process has stopped - no new native processes can be started
