Kafka 验证集群部署和吞吐量测试

测试Topic的创建和删除

bin/kafka-topics.sh --zookeeper localhost:2181,localhost:2182,localhost:2813 
--create --topic test-topic --partitions 3 --replication-factor 3

bin/kafka-topics.sh --zookeeper localhost:2181,localhost:2182,localhost:2183 
--delete --topic test-topic

 

测试消息的发送和消费

bin/kafka-console-producer.sh --broker-list localhost:9092,localhost:9093,localhost:9094 --topic test-topic

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --topic test-topic --from-beginning

生产者吞吐量测试

bin/kafka-producer-perf-test.sh --topic test-topic --num-records 500000 --record-size 200 
--throughput -1 --producer-props bootstrap.servers=localhost:9092,localhost:9093,localhost:9094 acks=1

输出：
500000 records sent, 196386.488610 records/sec (37.46 MB/sec), 464.37 ms avg latency, 699.00 ms max latency, 563 ms 50th, 688 ms 95th, 693 ms 99th, 698 ms 99.9th.

平均吞吐量是37.56MB/s ，平均每秒发送消息196386条，平均延迟0.464秒，最大延迟0.699秒，平均有50%的消息发送需要0.563秒，平均95%的消息需要0.688秒，平均99%的消息需要0.693秒，平均99.9%的消息需要0.698秒

 

消费者吞吐量测试

bin/kafka-consumer-perf-test.sh --broker-list localhost:9092,localhost:9093,localhost:9094 --fetch-size 200 
--messages 500000 --topic test-topic

输出：
start.time, end.time, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec, rebalance.time.ms, fetch.time.ms, fetch.MB.sec, fetch.nMsg.sec
2019-05-25 17:29:49:653, 2019-05-25 17:29:52:605, 95.3676, 32.3061, 500009, 169379.7425, 12, 2940, 32.4379, 170071.0884

测试消费50w条消息的吞吐量，1秒多消耗95MB，吞吐量大约92MB/s 也就是736Mb/s