zookeeper删除kafka元数据

问题:卸载kafka前未删除kafka topic,重新安装kafka后,生成跟之前topic名字相同的topic时报错,显示topic已存在

[root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create  --topic hive-test-table --partitions 5 --replication-factor 2
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Error while executing topic command : Topic "hive-test-table" already exists.
[2017-01-16 10:48:30,766] ERROR kafka.common.TopicExistsException: Topic "hive-test-table" already exists.
	at kafka.admin.AdminUtils$.createOrUpdateTopicPartitionAssignmentPathInZK(AdminUtils.scala:261)
	at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:245)
	at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:107)
	at kafka.admin.TopicCommand$.main(TopicCommand.scala:60)
	at kafka.admin.TopicCommand.main(TopicCommand.scala)
 (kafka.admin.TopicCommand$)

原因:kafka topic的元数据存储在zookeeper里,卸载kafka前未删除topic,之前topic的元数据仍存在zk里,需要手动删除zk里的元数据

解决:

zookeeper shell命令

[zk: d65:2181/kafka_hive(CONNECTED) 7] ll
ZooKeeper -server host:port cmd args
	connect host:port
	get path [watch]
	ls path [watch]
	set path data [version]
	rmr path
	delquota [-n|-b] path
	quit 
	printwatches on|off
	create [-s] [-e] path data acl
	stat path [watch]
	close 
	ls2 path [watch]
	history 
	listquota path
	setAcl path acl
	getAcl path
	sync path
	redo cmdno
	addauth scheme auth
	delete path [version]
	setquota -n|-b val path

进入zookeeper目录,删除kafka topic 元数据

[root@d100 bin]# sh zkCli.sh -server  d65:2181/kafka_hive

[zk: d65:2181/kafka_hive(CONNECTED) 8] ls /
[consumers, config, controller, isr_change_notification, brokers, admin, controller_epoch]

[zk: d65:2181/kafka_hive(CONNECTED) 9] ls /brokers
[seqid, topics, ids]

[zk: d65:2181/kafka_hive(CONNECTED) 10] ls /brokers/topics
[略...]

[zk: d65:2181/kafka_hive(CONNECTED) 11] rmr /brokers/topics/hive-test-table

[zk: d65:2181/kafka_hive(CONNECTED) 13] ls /brokers/topics
[略...]

重新创建topic成功

[root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create  --topic hive-test-table --partitions 5 --replication-factor 2
Created topic "hive-test-table".

[root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --desc  --topic hive-test-table
Topic:hive-test-table	PartitionCount:5	ReplicationFactor:2	Configs:
	Topic: hive-test-table	Partition: 0	Leader: 1285	Replicas: 1285,1281	Isr: 1285,1281
	Topic: hive-test-table	Partition: 1	Leader: 1281	Replicas: 1281,1282	Isr: 1281,1282
	Topic: hive-test-table	Partition: 2	Leader: 1282	Replicas: 1282,1283	Isr: 1282,1283
	Topic: hive-test-table	Partition: 3	Leader: 1283	Replicas: 1283,1284	Isr: 1283,1284
	Topic: hive-test-table	Partition: 4	Leader: 1284	Replicas: 1284,1285	Isr: 1284,1285

posted on   XIAO的博客  阅读(7564)  评论(0编辑  收藏  举报

编辑推荐:
· 如何编写易于单元测试的代码
· 10年+ .NET Coder 心语,封装的思维:从隐藏、稳定开始理解其本质意义
· .NET Core 中如何实现缓存的预热?
· 从 HTTP 原因短语缺失研究 HTTP/2 和 HTTP/3 的设计差异
· AI与.NET技术实操系列:向量存储与相似性搜索在 .NET 中的实现
阅读排行:
· 周边上新:园子的第一款马克杯温暖上架
· Open-Sora 2.0 重磅开源!
· 分享 3 个 .NET 开源的文件压缩处理库,助力快速实现文件压缩解压功能!
· Ollama——大语言模型本地部署的极速利器
· DeepSeek如何颠覆传统软件测试?测试工程师会被淘汰吗?

导航

统计

点击右上角即可分享
微信分享提示