flume 需求一:从指定网络端口采集数据输出到控制台
需求一:从指定网络端口采集数据输出到控制台
flume官网:https://flume.apache.org/
例子:Flume 1.9.0 User Guide — Apache Flume
使用flume的关键就是写配置文件:
A)配置Source
B)配置Channe
C)配置Sink
D)把以上三个组件串起来
a1:agent的名称
r1:source的名称
k1:sinks的名称
c1:channels的名称
注意:
一个source可以输出到多个channels
配置过程:
(1)
1、cd /kkb/install/apache-flume-1.7.0-bin/conf
2、在/conf
下创建exmaple.conf
example.conf:
# example.conf: A single-node Flume configuration # Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source
#绑定的source a1.sources.r1.type = netcat a1.sources.r1.bind =localhost a1.sources.r1.port = 44444 # Describe the sink a1.sinks.k1.type = logger # Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1
(2)启动agent:
./flume-ng agent -n a1 -c $FLUME_HOME/conf -f /kkb/install/apache-flume-1.7.0-bin/conf/exmaple.conf -Dflume.root.logger=INFO,console
(3)使用telnet
进行测试
另起个终端,输入:telnet localhost 44444
输出成功!!!
其他:
Event: { headers:{} body: 68 65 6C 6C 30 0D hell0. }
Event是flume数据传输的基本单元
Event = 可选的headers +bety array
参考:https://blog.csdn.net/fanfan4569/article/details/82957998