kafka ReadFile2Kafka
public class ReadFile2Kafka {
public static void main(String[] args) throws Exception {
if (args.length != 4) {
throw new RuntimeException("参数列表: bootstrap.servers、topic、sleep、input");
}
Properties props = new Properties();
props.put("bootstrap.servers", args[0]);
props.put("acks", "all");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
Producer<String, String> producer = new KafkaProducer<>(props);
BufferedReader bufferedReader = new BufferedReader(new FileReader(new File(args[3])));
String line = null;
while ((line = bufferedReader.readLine()) != null) {
Thread.sleep(Integer.parseInt(args[2]));
producer.send(new ProducerRecord<String, String>(args[1], line));
}
producer.flush();
producer.close();
Thread.sleep(3000);
}
}
shell 脚本
java -jar readFile2Kafka.jar node01:9092 $1 1000 /root/maoxiangyi/streaming/data/$1
个人网站:shuoyizui.com
公众号:写个框架玩
近期在公众号会发布一系列文章,主要是想完成一个简化的MapReduce框架的编写。实现Mapreduce编程模型、任务远程提交、任务分配、任务执行等功能。设计到了动态代理、反射、网络通信、序列化、消息队列、netty、自定义类加载器、多线程、shell等技术点。