常用source

Kafka Sources

复制代码
a3.sources.r1.type = org.apache.flume.source.kafka.KafkaSource
a3.sources.r1.batchSize = 5000
a3.sources.r1.batchDurationMillis = 2000
a3.sources.r1.kafka.bootstrap.servers = hadoop102:9092,hadoop103:9092,hadoop104:9092
a3.sources.r1.kafka.topics = business_db
a3.sources.r1.kafka.consumer.group.id = business-flume
a3.sources.r1.setTopicHeader = false
a3.sources.r1.interceptors = i1
a3.sources.r1.interceptors.i1.type = com.atguigu.interceptor.business_db.ETLTimestampInterceptor$Builder
a3.sources.r1.interceptors = i2
a3.sources.r1.interceptors.i2.type = com.atguigu.interceptor.user_log.TableNameInterceptor$Builder
复制代码

TailDir Source

a1.sources.r1.type = TAILDIR
a1.sources.r1.filegroups = f1
a1.sources.r1.filegroups.f1 = /opt/module/data_mocker/log/app.*
a1.sources.r1.positionFile = /opt/module/flume-1.9.0/position/taildir_position.json
a1.sources.r1.interceptors =  i1
a1.sources.r1.interceptors.i1.type = com.atguigu.interceptor.business_db.ETLTimestampInterceptor$Builder

 

posted @   Avery_rainys  阅读(17)  评论(0编辑  收藏  举报
相关博文:
阅读排行:
· 震惊!C++程序真的从main开始吗?99%的程序员都答错了
· winform 绘制太阳,地球,月球 运作规律
· 【硬核科普】Trae如何「偷看」你的代码?零基础破解AI编程运行原理
· 上周热点回顾(3.3-3.9)
· 超详细:普通电脑也行Windows部署deepseek R1训练数据并当服务器共享给他人
点击右上角即可分享
微信分享提示