filebeat系列-快速调试demo

如何做热部署

热部署至少需要一个reload的yml文件,以及一个或多个具体处理逻辑的yml文件

reload.yml文件内容示例:输出到kafka

复制代码
filebeat.config.inputs:
  enabled: true
  path: /opt/filebeat-7.6.2-linux-x86_64/prod-yml-test/*.yml
  reload.enabled: true
  reload.period: 3s
processors:
- rename:
   fields:
     - {from: "log.file.path",to: "log_file_path"}
     - {from: "agent.hostname",to: "log_host_name"}
- drop_fields:
   fields: ["log.file"]
output.kafka:
  hosts: ["10.192.78.27:9092"]
  topic: "sbdp_test"
http.enabled: true
http.host: 10.192.78.27
http.port: 5518
复制代码

这里只要是   /opt/filebeat-7.6.2-linux-x86_64/prod-yml-test/*.yml   里面的yml文件有变化,会自动reload,频率是3s检查一次变化

接着配置一个kafka输入,放在上面定义的文件夹下面:

复制代码
filebeat.inputs:
- type: kafka hosts: ["10.10.10.10:9092"] topics: ["aaa"] group_id: "group001" tags: ["bbb"] processors: - rename: fields: - {from: "message",to: "testmessage"}
复制代码

采集.log文件

复制代码
name: hahaha
filebeat.inputs:
- type: log
  enabled: true
  paths: /opt/filebeat-7.6.2-linux-x86_64/test-log.log
  tags: ["bb","aa"]
  fields: 
    ip: 10.192.78.24
output.console:
  pretty: true
复制代码

测试各个位置的name,fields,tags

复制代码
name: aaa
tags: ["tag01", "tag02"]
fields: {field01: "001", field02: "002"}
fields_under_root: true
filebeat.inputs:
- type: log
  enabled: true
  paths: /opt/filebeat/filebeat-7.6.2/tmpLogFile/bbb.log
  tags: ["tag03","tag04"]
  fields_under_root: true
  fields: 
    field03: field1
    field04: ${HOSTNAME}
processors:
  - rename:
      fields:
        - {from: "log.file.path", to: "logPath"}
  - drop_fields:
      fields: ["log.file"]
  - add_fields:
      target: "field05"
      fields:
        field005: 005
        fiel006: 006
  - add_tags:
      tags: [tag05, tag06]
      target: "p_tag"
  - add_id: ~
  - fingerprint: 
      fields: ["field01"]
output.console:
  pretty: true
  enable: true
复制代码

采集包含多行的java程序的.log日志输出到logstash

复制代码
filebeat.inputs:
- type: log
  enabled: true
  paths: /opt/filebeat/filebeat-7.6.2/tmpLogFile/bbb.log
  multiline.pattern: '^[[:space:]]+(at|\.{3})[[:space:]]+\b|^Caused by:'
  multiline.negate: false
  multiline.match: after
  fields_under_root: true
  tags: ["aa","bb"]
  fields: 
     ip: 10.192.77.150
output.logstash:
   hosts: [10.192.77.152:5045]
复制代码

 

posted @   鼠标的博客  阅读(478)  评论(0编辑  收藏  举报
相关博文:
阅读排行:
· 一个费力不讨好的项目,让我损失了近一半的绩效!
· 清华大学推出第四讲使用 DeepSeek + DeepResearch 让科研像聊天一样简单!
· 实操Deepseek接入个人知识库
· CSnakes vs Python.NET:高效嵌入与灵活互通的跨语言方案对比
· Plotly.NET 一个为 .NET 打造的强大开源交互式图表库
点击右上角即可分享
微信分享提示