问题:卸载kafka前未删除kafka topic,重新安装kafka后,生成跟之前topic名字相同的topic时报错,显示topic已存在
[root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create --topic hive-test-table --partitions 5 --replication-factor 2
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Error while executing topic command : Topic "hive-test-table" already exists.
[2017-01-16 10:48:30,766] ERROR kafka.common.TopicExistsException: Topic "hive-test-table" already exists.
at kafka.admin.AdminUtils$.createOrUpdateTopicPartitionAssignmentPathInZK(AdminUtils.scala:261)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:245)
at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:107)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:60)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
(kafka.admin.TopicCommand$)
原因:kafka topic的元数据存储在zookeeper里,卸载kafka前未删除topic,之前topic的元数据仍存在zk里,需要手动删除zk里的元数据
解决:
zookeeper shell命令
[zk: d65:2181/kafka_hive(CONNECTED) 7] ll
ZooKeeper -server host:port cmd args
connect host:port
get path [watch]
ls path [watch]
set path data [version]
rmr path
delquota [-n|-b] path
quit
printwatches on|off
create [-s] [-e] path data acl
stat path [watch]
close
ls2 path [watch]
history
listquota path
setAcl path acl
getAcl path
sync path
redo cmdno
addauth scheme auth
delete path [version]
setquota -n|-b val path
进入zookeeper目录,删除kafka topic 元数据
[root@d100 bin]# sh zkCli.sh -server d65:2181/kafka_hive
[zk: d65:2181/kafka_hive(CONNECTED) 8] ls /
[consumers, config, controller, isr_change_notification, brokers, admin, controller_epoch]
[zk: d65:2181/kafka_hive(CONNECTED) 9] ls /brokers
[seqid, topics, ids]
[zk: d65:2181/kafka_hive(CONNECTED) 10] ls /brokers/topics
[略...]
[zk: d65:2181/kafka_hive(CONNECTED) 11] rmr /brokers/topics/hive-test-table
[zk: d65:2181/kafka_hive(CONNECTED) 13] ls /brokers/topics
[略...]
重新创建topic成功
[root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create --topic hive-test-table --partitions 5 --replication-factor 2
Created topic "hive-test-table".
[root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --desc --topic hive-test-table
Topic:hive-test-table PartitionCount:5 ReplicationFactor:2 Configs:
Topic: hive-test-table Partition: 0 Leader: 1285 Replicas: 1285,1281 Isr: 1285,1281
Topic: hive-test-table Partition: 1 Leader: 1281 Replicas: 1281,1282 Isr: 1281,1282
Topic: hive-test-table Partition: 2 Leader: 1282 Replicas: 1282,1283 Isr: 1282,1283
Topic: hive-test-table Partition: 3 Leader: 1283 Replicas: 1283,1284 Isr: 1283,1284
Topic: hive-test-table Partition: 4 Leader: 1284 Replicas: 1284,1285 Isr: 1284,1285