Using KafkaBolt to write to a kafka topic
https://community.hortonworks.com/questions/27187/using-kafkabolt-to-write-to-a-kafka-topic.html
--------------------------------------------------------------------------------------------------------------
I'm building a kafka and storm based streaming application based on the usecase:
1. Application produces JSON message to a kafka topic
2. Kafka storm spout ingests the message and does processing
3. Produce output to another kafka topic using a storm bolt
Here are components and versions I'm using:
Storm: 0.10.0
Kafka_2.10: 0.9.0.1
HDP: 2.3.4
storm-kafka: 0.10.0
I accomplished steps 1 and 2 using "storm.kafka.KafkaSpout" that comes with storm-kafka
I'm trying to use "storm.kafka.KafkaBolt" to write processed data to a topic and I couldn't figure out exactly how to do it. Here is the code snippet:
1 2 3 4 5 6 7 8 9 10 11 | Properties props = new Properties(); props.put( "metadata.broker.list" , "192.168.56.102:9092" ); props.put( "request.required.acks" , "1" ); props.put( "serializer.class" , "kafka.serializer.StringEncoder" ); Config conf = new Config(); conf.put(KafkaBolt.KAFKA_BROKER_PROPERTIES, props); KafkaBolt kafkaBolt = new KafkaBolt().withTopicSelector( new DefaultTopicSelector( "OUTBOUND_TOPIC" ))<br> .withTupleToKafkaMapper( new FieldNameBasedTupleToKafkaMapper()); builder.setBolt( "kafka_outbound_bolt" , kafkaBolt, 3 ).shuffleGrouping( "process_bolt" ); |
"process_bolt" sends out a tuple that is a serializable domain model object (SubModel.java). I would like to send SubModel.toString() data to the outbound topic. I'm running storm in local cluster mode connected to kafka in a local VM.
- Am I doing anything wrong ?
- How do I use "FieldNameBasedTupleToKafkaMapper"?
【推荐】国内首个AI IDE,深度理解中文开发场景,立即下载体验Trae
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· AI与.NET技术实操系列:基于图像分类模型对图像进行分类
· go语言实现终端里的倒计时
· 如何编写易于单元测试的代码
· 10年+ .NET Coder 心语,封装的思维:从隐藏、稳定开始理解其本质意义
· .NET Core 中如何实现缓存的预热?
· 分享一个免费、快速、无限量使用的满血 DeepSeek R1 模型,支持深度思考和联网搜索!
· 基于 Docker 搭建 FRP 内网穿透开源项目(很简单哒)
· 25岁的心里话
· ollama系列01:轻松3步本地部署deepseek,普通电脑可用
· 按钮权限的设计及实现
2016-08-22 elasticSearch indices VS type
2016-08-22 一个不错的简历制作网站
2016-08-22 php active-record 设计模式