运维之日志系统 nginx错误日志（三）

第一章 filebeat配置

1.1filebeat配置access和error日志进行分割

root@kibana01 nginx]# cat /etc/filebeat/filebeat.yml

filebeat.inputs:

- type: log

  enabled: true

  paths:

    - /var/log/nginx/access.log

  json.keys_under_root: true

  json.overwrite_keys: true

  tags: ["access"]

- type: log

  enabled: true

  paths:

    - /var/log/nginx/error.log

  tags: ["error"]

setup.kibana:

  host: "localhost:5601"

output.elasticsearch:

  hosts: ["139.9.186.7:9200"]

  indices:

    - index: "nginx-access-%{[beat.version]}-%{+yyyy.MM}"

      when.contains:

        tags: "access"

    - index: "nginx-error-%{[beat.version]}-%{+yyyy.MM}"

      when.contains:

        tags: "error"

 

setup.template.name: "nginx"

setup.template.pattern: "nginx-*"

setup.template.enabled: false

setup.template.overwrite: true

 

1.2.对nginx 进行正常和error日志数据进行压测

[root@kibana01 nginx]# ab -n 10000 -c 10000 http://121.36.10.238/

This is ApacheBench, Version 2.3 <$Revision: 1430300 $>

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 121.36.10.238 (be patient)

Completed 1000 requests

Completed 2000 requests

Completed 3000 requests

Completed 4000 requests

Completed 5000 requests

Completed 6000 requests

Completed 7000 requests

Completed 8000 requests

Completed 9000 requests

Completed 10000 requests

Finished 10000 requests 

Server Software:        nginx/1.16.1

Server Hostname:        121.36.10.238

Server Port:            80

Document Path:          /

Document Length:        4833 bytes

Concurrency Level:      10000

Time taken for tests:   39.111 seconds

Complete requests:      10000

Failed requests:        2

   (Connect: 0, Receive: 0, Length: 2, Exceptions: 0)

Write errors:           0

Total transferred:      50674208 bytes

HTML transferred:       48324208 bytes

Requests per second:    255.68 [#/sec] (mean)

Time per request:       39111.125 [ms] (mean)

Time per request:       3.911 [ms] (mean, across all concurrent requests)

Transfer rate:          1265.28 [Kbytes/sec] received

 

Connection Times (ms)

              min  mean[+/-sd] median   max

Connect:        0 1126 2147.1   1001   15056

Processing:     1 1055 4462.4      1   29759

Waiting:        1 1045 4444.5      1   29759

Total:          1 2181 5082.5   1002   36780

 

Percentage of the requests served within a certain time (ms)

  50%   1002

  66%   1004

  75%   1204

  80%   2840

  90%   4085

  95%   9159

  98%  27052

  99%  27055

100%  36780 (longest request)

 

[root@kibana01 nginx]# wc -l access.log

10000 access.log

 

[root@kibana01 nginx]# ab -n 10000 -c 10000 http://121.36.10.238/test.html

[root@kibana01 nginx]# wc -l error.log

7736 error.log

 

1.3.filebeat 进行日志数据验证