//雪花飘落特效 //右上角github跳转   

FileBeat多行消息Multiline

官方地址:
https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html

服务日志输出样例:

根据样例可以看出,每一行开始是有相同规律:

[时间]2021-10-22   详细学习可以查阅正则表达式
'^(.{4})([0-9]{4}-[0-9]{2}-[0-9]{2})' 
^(.{4}):表示以任意四个字符开始/  [时间]
[0-9]{4}-:表示4个0到9数字/  2021-
[0-9]{2}-: 表示2个0到9数字/  10-
[0-9]{2}: 表示2个0到9数字/   22

filebeta的filebeat.yml配置文件:

└─# grep -v "^#" filebeat.yml | grep -v "^$" | grep -v "#"
filebeat.inputs:
filebeat.config.inputs:
  path: config/*.yml
  reload.enabled: true

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
setup.kibana:
  host: "http://172.17.12.174:5601"
output.elasticsearch:
  username: filebeta_log
  hosts: ["http://es-cn-v0h1dthan00259sf2.elasticsearch.aliyuncs.com:9200"]
  indices:
    - index: "%{[fields.servicename]}-%{+yyyy.ww}"
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
  - drop_fields:
      fields: ["agent.ephemeral_id","agent.hostname","agent.id","agent.name","agent.type","agent.version","host.id","host.hostname","host.architecture","host.os.platform","host.ip","log.file.path","host.mac","host.os.build","host.os.family","host.os.kernel","ecs.version","log.offset","log.flags"]
      ignore_missing: false

filebeta日志收集配置项input-win.yml配置:

- type: log
  enabled: true
  paths:
    - D:\BetaWM\BetaWM.Server\Beta.VipManagerH5API\Logs\*.log
  encoding: GB2312
  multiline.pattern: '^(.{4})([0-9]{4}-[0-9]{2}-[0-9]{2})'
  multiline.negate: true
  multiline.match: after
  fields:
    servicename: "beta-vipmanagerh5api"
- type: log
  enabled: true
  paths:
    - D:\BetaWM\BetaWM.Server\Beta.WxWorkApi\Log\*
  encoding: GB2312
  multiline.pattern: '^([0-9]{4}-[0-9]{2}-[0-9]{2})'
  multiline.negate: true
  multiline.match: after
  fields:
    servicename: "beta-wxworkapi"

kibana日志收集展示根据配置message展示如下,和我们在服务的日志输出文件中message一致:

posted @ 2021-10-22 13:05  农夫运维  阅读(553)  评论(0编辑  收藏  举报