Kafka 入门

Kafka是由LinkedIn开发的一个分布式的消息系统，使用Scala编写，它以可水平扩展和高吞吐率而被广泛使用。目前越来越多的开源分布式处理系统如Cloudera、Apache Storm、Spark都支持与Kafka集成。

Kafka拓扑结构

安装和启动

Download

Download the 0.10.2.1 release and un-tar it.

tar -xzf kafka_2.11-0.10.2.1.tgz
cd kafka_2.11-0.10.2.1

Start the server

Kafka uses ZooKeeper so you need to first start a ZooKeeper server if you don't already have one. You can use the convenience script packaged with kafka to get a quick-and-dirty single-node ZooKeeper instance.

bin/zookeeper-server-start.sh config/zookeeper.properties

Now start the Kafka server:

bin/kafka-server-start.sh config/server.properties

Create a topic

bin/kafka-topics.sh --create --zookeeper localhost:8181 --replication-factor 1 --partitions 1 --topic test

查看topic

bin/kafka-topics.sh --list --zookeeper localhost:8181
test

Send some messages

bin/kafka-console-producer.sh --broker-list localhost:8092 --topic test

Start a consumer

bin/kafka-console-consumer.sh --bootstrap-server localhost:8092 --topic test --from-beginning

common commands

1. list topic command

bin/kafka-topics.sh --list --zookeeper localhost:8181

2. 查询topic

bin/kafka-topics.sh --describe --zookeeper localhost:8181 --topic my-replicated-topic
Topic:my-replicated-topic	PartitionCount:1	ReplicationFactor:3	Configs:
	Topic: my-replicated-topic	Partition: 0	Leader: 0	Replicas: 0,2,1	Isr: 0

参考

1. kafka学习笔记：知识点整理
2. Kafka入门经典教程