flume 需求二:监控一个文件实时采集新增的数据输出到控制台
需求二:监控一个文件实时采集新增的数据输出到控制台
agent选型:exec source + memory channel +logger sink
1)配置文件
#新建data文件
cd /kkb/
mkdir data
#新建日志文件
/kkb/data
touch data.log
在/conf 下 新建配置 exec-memory-logger.conf
配置内容
a1.sources.r1.command = tail -F /kkb/data/data.log 指 监听的文件
# example.conf: A single-node Flume configuration # Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = exec a1.sources.r1.command = tail -F /kkb/data/data.log a1.sources.r1.shell = /bin/sh -c # Describe the sink a1.sinks.k1.type = logger # Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1
(2)运行:
flume-ng agent -n a1 -c $FLUME_HOME/conf -f /kkb/install/apache-flume-1.7.0-bin/conf/exec-menory.logger.conf -Dflume.root.logger=INFO,console
(3)测试:
cd /kkb/data/ echo hello >> data.log echo nihao >> data.log
成功!!