Windows apache-flume-1.6.0+Kafka+Es
apache-flume-1.6.0
kafka_2.11-1.1.0
elasticsearch1.5.1
flume配置
a1.sources = kafkaSource a1.channels = memoryChannel a1.sinks = elasticsearch a1.sources.kafkaSource.channels = memoryChannel a1.sources.kafkaSource.type = org.apache.flume.source.kafka.KafkaSource a1.sources.kafkaSource.zookeeperConnect = 127.0.0.1:2181 a1.sources.kafkaSource.topic = test1 a1.sources.kafkaSource.groupId = flume a1.sinks.elasticsearch.channel = memoryChannel a1.sinks.elasticsearch.type=org.apache.flume.sink.elasticsearch.ElasticSearchSink a1.sinks.elasticsearch.hostNames=127.0.0.1:9300 a1.sinks.k1.indexType = bar_type a1.sinks.elasticsearch.indexName=logstash a1.sinks.elasticsearch.clusterName=jachs a1.sinks.elasticsearch.serializer=org.apache.flume.sink.elasticsearch.ElasticSearchLogStashEventSerializer a1.channels.memoryChannel.type=memory a1.channels.memoryChannel.capacity=10000 a1.channels.memoryChannel.transactionCapacity=1000
启动
flume-ng.cmd agent -conf ../conf -conf-file ../conf/a.conf -name a1 -property flume.root.logger=INFO,console