Flume Netcat Source

1.cd /usr/local2/flume/conf

sudo vim netcat.conf

  • # Name the components on this agent
  • a1.sources = r1
  • a1.sinks = k1
  • a1.channels = c1
  •  
  • # Describe/configure the source
  • a1.sources.r1.type = netcat
  • a1.sources.r1.bind = localhost
  • a1.sources.r1.port = 8888
  • # Describe the sink
  • a1.sinks.k1.type = logger
  •  
  • # Use a channel which buffers events in memory
  • a1.channels.c1.type = memory
  • a1.channels.c1.capacity = 1000
  • a1.channels.c1.transactionCapacity = 100
  •  
  • # Bind the source and sink to the channel
  • a1.sources.r1.channels = c1
  • a1.sinks.k1.channel = c1   
2.启动flume agent:
cd /usr/local2/flume/bin
soyo@soyo-VPCCB3S1C:/usr/local2/flume/bin$ ./flume-ng agent --conf conf -f /usr/local2/flume/conf/netcat.conf  --name a1
3.打开另一个shell：
nc localhost 8888
输入内容 即可以被flume捕获 

注意：flume 无论是netcat还是avro 都无法显示中文 需要到 **.conf中做配置，但Kafka默认是可以显示中文的
备注：这个问题在IDEA，spark Streaming 以flume为数据源的程序中 已经解决


posted @ 2017-10-18 15:19  soyosuyang  阅读(337)  评论(0)  编辑  收藏  举报