Kafka命令行操作

1. 查看当前服务器中的所有topic

[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper hadoop1:2181/kafka --list

2. 创建topic

[hadoop@hadoop1 ~]$ kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 2 --partitions 3 --topic test
Created topic test.

参数说明：

• --create：创建一个topic
• --zookeeper: 指定zookeeper集群的主机列表，多个server可使用逗号分隔，这里因为kafka和zk是在同一个server，所以直接连接了本机的2181端口
• --replication-factor：指定创建这个topic的副本数
• --partitions：指定该topic的分区数
• --topic：指定topic的名称

3. 删除 topic

#执行删除操作
[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper localhost:2181 --delete --topic test1
Topic test1 is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.
#查看topic，可以看到myfirsttopic已被删除
[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper localhost:2181 --list
__consumer_offsets
hadoop
test
topic-dll

4. 列出现有的topic

[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper localhost:2181 --list
__consumer_offsets
hadoop
test
test1
topic-dll

5. 查看topic的详细信息

[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper localhost:2181 --describe --topic test
Topic: test     PartitionCount: 3       ReplicationFactor: 2    Configs: 
        Topic: test     Partition: 0    Leader: 2       Replicas: 2,3   Isr: 2,3
        Topic: test     Partition: 1    Leader: 3       Replicas: 3,1   Isr: 3,1
        Topic: test     Partition: 2    Leader: 1       Replicas: 1,2   Isr: 1,2

参数说明：

• --describe：查看topic的详细信息

输出说明：

• Topic：主题名
• PartitionCount：分区数
• ReplicationFactor：副本数
• Leader: 当前负责读写的leader broker
• Replicas：当前分区所有的副本对应的broker列表
• Isr：处于活动状态的broker

6. 更加topic的partition数量

#topic: test 增加到6个分区
[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper localhost:2181 --alter --partitions 6 --topic test
WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
Adding partitions succeeded!
#查看topic：test 详细信息
[hadoop@hadoop1 ~]$ kafka-topics.sh --zookeeper localhost:2181 --describe  --topic test
Topic: test     PartitionCount: 6       ReplicationFactor: 2    Configs: 
        Topic: test     Partition: 0    Leader: 2       Replicas: 2,3   Isr: 2,3
        Topic: test     Partition: 1    Leader: 3       Replicas: 3,1   Isr: 3,1
        Topic: test     Partition: 2    Leader: 1       Replicas: 1,2   Isr: 1,2
        Topic: test     Partition: 3    Leader: 2       Replicas: 2,3   Isr: 2,3
        Topic: test     Partition: 4    Leader: 3       Replicas: 3,1   Isr: 3,1
        Topic: test     Partition: 5    Leader: 1       Replicas: 1,2   Isr: 1,2

7. 通过producer 生产消息

[hadoop@hadoop1 ~]$ kafka-console-producer.sh --broker-list hadoop1:9092 --topic test
>hello kafka
>just a test
>hello world
>
>hahahahah

8. 通过consumer消费消息

[hadoop@hadoop2 ~]$ kafka-console-consumer.sh --bootstrap-server hadoop1:9092 --topic test --from-beginning
hello kafka
just a test
hello world

hahahahah