ELK部署安装+springboot

elasticsearchStack相关软件下载

https://elasticsearch.cn/download/

1. elasticsearch 安装

   参考 https://www.cnblogs.com/zsls-lang/p/10171367.html

2. Kibana 安装

   参考 https://www.cnblogs.com/zsls-lang/p/12522204.html

3. logstash 安装

https://www.elastic.co/cn/downloads/past-releases/logstash-6-8-13

下载安装：

[root@10-23-171-197 src]# curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-6.8.13.tar.gz
# 或者  wget  https://artifacts.elastic.co/downloads/logstash/logstash-6.8.13.tar.gz 下载
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  170M  100  170M    0     0  5574k      0  0:00:31  0:00:31 --:--:-- 5900k
[root@10-23-171-197 src]# tar -zxvf logstash-6.8.13.tar.gz  -C /usr/local
[root@10-23-171-197 local]# cd logstash-6.8.13/
[root@10-23-171-197 logstash-6.8.13]# ls
bin config CONTRIBUTORS data Gemfile Gemfile.lock lib LICENSE.txt logstash-core logstash-core-plugin-api modules NOTICE.TXT tools vendor x-pack
[root@10-23-171-197 logstash-6.8.13]# cd config/
[root@10-23-171-197 config]# ls
jvm.options log4j2.properties logstash-sample.conf logstash.yml pipelines.yml startup.options
[root@10-23-171-197 config]# cp logstash-sample.conf logstash.conf
[root@10-23-171-197 config]# vim logstash.conf
修改：

http.host: "0.0.0.0"
http.port: 9600

开放防火墙：

[root@localhost config]# firewall-cmd --zone=public --add-port=9600/tcp --permanent
success
[root@localhost config]# firewall-cmd --reload
success

测试：

logstash-test.conf 内容 ：

input { stdin { } }
output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }
}

### rubydebug 详细的信息

[root@localhost logstash-6.8.13]# bin/logstash -f config/logstash-test.conf

访问：http://192.168.124.102:9600/

 

{"host":"localhost.localdomain","version":"6.8.13","http_address":"0.0.0.0:9600","id":"3a6d7765-c743-415f-ab79-3f6bb8dead35","name":"localhost.localdomain","build_date":"2020-10-16T10:43:55Z","build_sha":"28af0b362e5b7e6d90d9cfadfb46fed35eb643f0","build_snapshot":false}

　

 4. 在测试的时候虚机器内存小

根据自己情况进行设置jvm，情况好的不用设置

 ES 配置jvm大小位置：

[root@localhost config]# vim jvm.options
[root@localhost config]# pwd
/usr/local/elasticsearch-6.8.13/config
[root@localhost config]#

-Xms512m
-Xmx512m

logstash 配置jvm大小位置:

[root@localhost config]# vim jvm.options
[root@localhost config]# pwd
/usr/local/logstash-6.8.13/config
[root@localhost config]#

-Xms512m
-Xmx512m

5. springboot 日志推送

项目链接：https://github.com/zsls-lang/zsls/tree/master/chapter39

                  https://gitee.com/zsls-lang/zsls/tree/master/chapter39

 

logback.xml ：

<appender name="LOGSTASH"
              class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        <!-- logstash 服务器 ip:port -->
        <destination>192.168.124.102:4560</destination>
        <!-- encoder必须配置,有多种可选 -->
        <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder" >
            <!-- "appname":"logstash-test" 的作用是指定创建索引的名字时用，并且在生成的文档中会多了这个字段  -->
            <customFields>{"appname":"${spring.application.name}"}</customFields>
        </encoder>
        <!-- 这个配置是 向logstash输出日志如果有多个logstash IP或端口可以轮询负载各端口 -->
        <connectionStrategy>
            <roundRobin>
                <connectionTTL>5 minutes</connectionTTL>
            </roundRobin>
        </connectionStrategy>
    </appender>

    <root level="info">
        <appender-ref ref="CONSOLE" />
        <appender-ref ref="LOGSTASH" />
    </root>

 

maven 引用jar：

        <!-- https://mvnrepository.com/artifact/net.logstash.logback/logstash-logback-encoder -->
        <dependency>
            <groupId>net.logstash.logback</groupId>
            <artifactId>logstash-logback-encoder</artifactId>
            <version>7.0.1</version>
        </dependency>

 

logstash服务器的logstash.conf配置：

input {
   stdin { }
     tcp {
         # host:port就是上面appender中的 destination，
          # 这里其实把logstash作为服务，开启4560端口接收logback发出的消息
              host => "192.168.124.102"
              port => 4560
              mode => "server"
              tags => ["tags"]
              codec => json_lines
         }
}
output {
# 输出到es
   elasticsearch {
      hosts => ["192.168.124.101:9200"]
      index => "%{[appname]}"
   }
   stdout { codec => rubydebug }
   # 输出到文件
   file {
     path => "/data/test/logstash/out"
     codec => line
   }
}

 

6. 查找日志

索引：

在 logback.xml中  customFields 中配置的 用作索引 如： logstash-test*

 

 

 

 

 查找：