大数据基础之Logstash(3)应用之http(in和out)
一个logstash很容易通过http打断成两个logstash实现跨服务器或者跨平台间数据同步,比如原来的流程是
logstash: nginx log -> kafka
打断成两个是
logstash1: nginx log -> http out
logstash2: http in ->kafka
具体如下
http out
filter {
ruby {
code => "event.cancel if not event.get('message').include?'something'"
}
}
output { http { url => "http://test.server:10000" codec => "plain" format => "json" content_type => "application/json" http_method => "post" } }
可以通过filter跳过不需要的记录
http in
input { http { host => "0.0.0.0" port => 10000 additional_codecs => {"application/json"=>"json"} codec => "plain" threads => 4 ssl => false } }
http in
参考:https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http.html
http out
参考:https://www.elastic.co/guide/en/logstash/current/plugins-outputs-http.html
---------------------------------------------------------------- 结束啦,我是大魔王先生的分割线 :) ----------------------------------------------------------------
- 由于大魔王先生能力有限,文中可能存在错误,欢迎指正、补充!
- 感谢您的阅读,如果文章对您有用,那么请为大魔王先生轻轻点个赞,ありがとう