Elasticsearch：负责日志检索和分析，它的特点有：分布式，零配置，自动发现，索引自动分片，索引副本机制，restful风格接口，多数据源，自动搜索负载等
Logstash：对日志进行收集、过滤，并将其存储供以后使用（如，搜索日志）
Kibana：为日志分析提供友好的Web界面，可以帮助汇总、分析和搜索重要数据日志
客户端软件：
在需要收集日志的所有服务上部署logstash，作为logstash agent（logstash shipper）用于监控并过滤收集日志，将过滤后的内容发送到logstash indexer，logstash indexer将日志收集在一起交给全文搜索服务ElasticSearch，可以用ElasticSearch进行自定义搜索，然后通过Kibana来结合自定义搜索进行页面展示。
借用网络上别人家的图
下载地址
cd /home
mkdir elk
cd elk
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.zip
unzip elasticsearch-6.2.4.zip
mv elasticsearch-6.2.4 elasticsearch
特别注意:elasticsearch5.0以上版本不支持root用户启动，现在增加一个用户
在增加用户之前，需要下载log4j 我安装的过程报了下面的错误 所以安装log4j
yum install -y unzip zip
yum install -y log4j* ###安装log4j包
OpenJDK 64-Bit Server VM warning: Cannot open file logs/gc.log due to Permission denied 2018-12-27 13:44:07,424 main ERROR Unable to invoke factory method in class org.apache.logging.log4j.core.appender.RollingFileAppender for element RollingFile: java.lang.IllegalStateException: No factory method found for class org.apache.logging.log4j.core.appender.RollingFileAppender java.lang.IllegalStateException: 2018-12-27 13:44:07,428 main ERROR RollingFileManager (/home/elk/elasticsearch/logs/elasticsearch_deprecation.log) java.io.FileNotFoundException:/home/elk/elasticsearch/logs/elasticsearch_deprecation.log (Permission denied) java.io.FileNotFoundException:
#groupadd esuser
#useradd -g esuser esuser
将data和logs目录的属主和属组改为esuser
#chown esuser.esuser /home/elk/elasticsearch -R 把目录/elasticsearch及其下的所有文件和子目录的属主改成esuser，属组改成esuser 第二个是属于哪个组。
标红的地方也要注意，要给绝对路径的用户
[root@insure elk]# sudo chown -R esuser.root /home/elk/elasticsearch [root@insure elk]# ls -ltr total 233724 drwxr-xr-x 8 esuser root 4096 Apr 12 2018 elasticsearch -rw-r--r-- 1 root root 153969993 May 15 2018 logstash-6.2.4.zip -rw-r--r-- 1 root root 85348919 Oct 11 13:49 kibana-6.2.4-linux-x86_64.tar.gz drwxr-xr-x 2 root root 4096 Dec 27 10:03 back [root@insure elk]# sudo chown -R esuser.esuser /home/elk/elasticsearch [root@insure elk]# ls -ltr total 233724 drwxr-xr-x 8 esuser esuser 4096 Apr 12 2018 elasticsearch
启动elasticsearch &或者-d 可以后台启动
[esuser@insure elasticsearch]$ ./bin/elasticsearch [2018-12-27T10:33:51,435][INFO ][o.e.n.Node ] [] initializing ... [2018-12-27T10:33:51,492][INFO ][o.e.e.NodeEnvironment ] [LGHRC-d] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [439.6gb], net total_space [492gb], types [rootfs] [2018-12-27T10:33:51,492][INFO ][o.e.e.NodeEnvironment ] [LGHRC-d] heap size [989.8mb], compressed ordinary object pointers [true] [2018-12-27T10:33:51,494][INFO ][o.e.n.Node ] node name [LGHRC-d] derived from node ID [LGHRC-dHRqCXmCwcXI6XSg]; set [node.name] to override [2018-12-27T10:33:51,494][INFO ][o.e.n.Node ] version[6.2.4], pid[19099], build[ccec39f/2018-04-12T20:37:28.497551Z], OS[Linux/3.10.0-693.2.2.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_181/25.181-b13] [2018-12-27T10:33:51,494][INFO ][o.e.n.Node ] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.O9wXX03d, -XX:+HeapDumpOnOutOfMemoryError, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/home/elk/elasticsearch, -Des.path.conf=/home/elk/elasticsearch/config] [2018-12-27T10:33:52,038][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [aggs-matrix-stats] [2018-12-27T10:33:52,038][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [analysis-common] [2018-12-27T10:33:52,038][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [ingest-common] [2018-12-27T10:33:52,038][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [lang-expression] [2018-12-27T10:33:52,038][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [lang-mustache] [2018-12-27T10:33:52,038][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [lang-painless] [2018-12-27T10:33:52,039][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [mapper-extras] [2018-12-27T10:33:52,039][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [parent-join] [2018-12-27T10:33:52,039][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [percolator] [2018-12-27T10:33:52,039][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [rank-eval] [2018-12-27T10:33:52,039][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [reindex] [2018-12-27T10:33:52,039][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [repository-url] [2018-12-27T10:33:52,039][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [transport-netty4] [2018-12-27T10:33:52,039][INFO ][o.e.p.PluginsService ] [LGHRC-d] loaded module [tribe] [2018-12-27T10:33:52,040][INFO ][o.e.p.PluginsService ] [LGHRC-d] no plugins loaded [2018-12-27T10:33:54,229][INFO ][o.e.d.DiscoveryModule ] [LGHRC-d] using discovery type [zen] [2018-12-27T10:33:54,701][INFO ][o.e.n.Node ] initialized [2018-12-27T10:33:54,702][INFO ][o.e.n.Node ] [LGHRC-d] starting ... [2018-12-27T10:33:54,827][INFO ][o.e.t.TransportService ] [LGHRC-d] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300} [2018-12-27T10:33:57,878][INFO ][o.e.c.s.MasterService ] [LGHRC-d] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {LGHRC-d}{LGHRC-dHRqCXmCwcXI6XSg}{0rZYMLjHT9Wib9xZBG18Fg}{127.0.0.1}{127.0.0.1:9300} [2018-12-27T10:33:57,883][INFO ][o.e.c.s.ClusterApplierService] [LGHRC-d] new_master {LGHRC-d}{LGHRC-dHRqCXmCwcXI6XSg}{0rZYMLjHT9Wib9xZBG18Fg}{127.0.0.1}{127.0.0.1:9300}, reason: apply cluster state (from master [master {LGHRC-d}{LGHRC-dHRqCXmCwcXI6XSg}{0rZYMLjHT9Wib9xZBG18Fg}{127.0.0.1}{127.0.0.1:9300} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]]) [2018-12-27T10:33:57,895][INFO ][o.e.h.n.Netty4HttpServerTransport] [LGHRC-d] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200} [2018-12-27T10:33:57,895][INFO ][o.e.n.Node ] [LGHRC-d] started [2018-12-27T10:33:57,904][INFO ][o.e.g.GatewayService ] [LGHRC-d] recovered [0] indices into cluster_state
验证
[root@insure elasticsearch]# curl 127.0.0.1:9200
{
"name" : "LGHRC-d",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "NuqE84ouT463APhJLg21uA",
"version" : {
"number" : "6.2.4",
"build_hash" : "ccec39f",
"build_date" : "2018-04-12T20:37:28.497551Z",
"build_snapshot" : false,
"lucene_version" : "7.2.1",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
如果要在外网的浏览器访问
修改config/elasticsearch.yml
#network.host: 192.168.0.1 改为network.host: 47.96.14.108 这个是物理的IP
如果是阿里云服务器，一般外网访问都会做IP映射，那么这个IP就要改成可以全部访问的地址
可以改为 network.host: 0.0.0.0
[esuser@insure bin]$ jps 2560 Jps 1595 Elasticsearch [esuser@insure bin]$ kill -9 1595
[esuser@insure bin]$ ./elasticsearch &
浏览器访问
到此elasticsearch 搜索引擎搭建成功
可能会遇到的问题
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解决办法
在/etc/sysctl.conf文件最后添加一行
vm.max_map_count=262144
执行命令 sysctl -p
下一节我们搭建kibana
