centos安装kafka

一、首先安装zookeeper

1、下载

[root@orderer src]# cd /usr/local/src/
[root@orderer src]# wget http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
--2020-06-02 16:49:53--  http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
?????? mirrors.hust.edu.cn (mirrors.hust.edu.cn)... 202.114.18.160
???? mirrors.hust.edu.cn (mirrors.hust.edu.cn)|202.114.18.160|:80... ????
??? HTTP ??,??????... 200 OK
??:37676320 (36M) [application/octet-stream]
?????: “zookeeper-3.4.14.tar.gz.1”

100%[=================================================================================================================>] 37,676,320  1.91MB/s ?? 16s      

2020-06-02 16:50:09 (2.22 MB/s) - ??? “zookeeper-3.4.14.tar.gz.1” [37676320/37676320])

[root@orderer src]# 

2、解压

[root@orderer src]# tar -zxvf zookeeper-3.4.14.tar.gz

3、移动到目标目录下

[root@orderer src]# mv zookeeper-3.4.14/ ../

4、复制conf下的zoo_sample.cfg为zoo.cfg

[root@orderer conf]# cd /usr/local/zookeeper-3.4.14/conf/
[root@orderer conf]# cp zoo_sample.cfg zoo.cfg
[root@orderer conf]# 

5、在/usr/local/zookeeper-3.4.14下新建一个data目录，作为数据目录

root@orderer zookeeper-3.4.14]# mkdir /usr/local/zookeeper-3.4.14/data
[root@orderer zookeeper-3.4.14]# 

6、编辑zoo.cfg文件

dataDir=/usr/local/zookeeper-3.4.14/data

设置dataDir为刚才建立的目录路径

7、启动zookeeper

[root@orderer zookeeper-3.4.14]# /usr/local/zookeeper-3.4.14/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@orderer zookeeper-3.4.14]#

8、查看状态,默认监听端口为2181

[root@orderer zookeeper-3.4.14]# /usr/local/zookeeper-3.4.14/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: standalone
[root@orderer zookeeper-3.4.14]# 

9、停止

[root@orderer zookeeper-3.4.14]# /usr/local/zookeeper-3.4.14/bin/zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.14/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@orderer zookeeper-3.4.14]# 

 

二、安装kafka

1、下载

[root@orderer src]# wget http://mirrors.hust.edu.cn/apache/kafka/2.2.2/kafka_2.11-2.2.2.tgz

2、解压

[root@orderer src]# tar -zxvf kafka_2.11-2.2.2.tgz 

 3、移动到目标目录下

[root@orderer src]# mv kafka_2.11-2.2.2/ ../

 4、编辑config/server.properties配置文件

[root@orderer kafka_2.11-2.2.2]# vim config/server.properties 

log.dirs=/usr/local/kafka_2.11-2.2.2/kafka-logs

5、首先启动zookpeer

[root@orderer zookeeper-3.4.14]# bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@orderer zookeeper-3.4.14]#

6、再启动kafka

[root@orderer kafka_2.11-2.2.2]# bin/kafka-server-start.sh config/server.properties

最后显示

[2020-06-02 17:08:50,824] INFO [ProducerId Manager 0]: Acquired new producerId block (brokerId:0,blockStartProducerId:0,blockEndProducerId:999) by writing to Zk with path version 1 (kafka.coordinator.transaction.ProducerIdManager)
[2020-06-02 17:08:50,867] INFO [TransactionCoordinator id=0] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)
[2020-06-02 17:08:50,870] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
[2020-06-02 17:08:50,871] INFO [Transaction Marker Channel Manager 0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2020-06-02 17:08:50,946] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
[2020-06-02 17:08:50,966] INFO [SocketServer brokerId=0] Started data-plane processors for 1 acceptors (kafka.network.SocketServer)
[2020-06-02 17:08:50,973] INFO Kafka version: 2.2.2 (org.apache.kafka.common.utils.AppInfoParser)
[2020-06-02 17:08:50,973] INFO Kafka commitId: 1d348535a0a747d1 (org.apache.kafka.common.utils.AppInfoParser)
[2020-06-02 17:08:50,976] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)

启动成功。

7、查看

[root@orderer kafka_2.11-2.2.2]# jps
27937 QuorumPeerMain
28052 Kafka
28602 Jps
[root@orderer kafka_2.11-2.2.2]# 

默认服务端口为9092

[root@orderer kafka_2.11-2.2.2]# netstat -tunlp|grep 9092
tcp6       0      0 :::9092                 :::*                    LISTEN      28052/java          
[root@orderer kafka_2.11-2.2.2]# 

8、停止

[root@orderer kafka_2.11-2.2.2]# bin/kafka-server-stop.sh config/server.properties 

9,、创建topic

[root@orderer kafka_2.11-2.2.2]# bin/kafka-topics.sh --zookeeper localhost:2181 --create --replication-factor 1 --partitions 1 --topic demo
Created topic demo.
[root@orderer kafka_2.11-2.2.2]# 

10、查看topic列表

[root@orderer kafka_2.11-2.2.2]# bin/kafka-topics.sh --zookeeper localhost:2181 --list
demo
[root@orderer kafka_2.11-2.2.2]# 

11、查看指定topic信息

[root@orderer kafka_2.11-2.2.2]# bin/kafka-topics.sh  --zookeeper localhost:2181 --describe --topic demo
Topic:demo      PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: demo     Partition: 0    Leader: 0       Replicas: 0     Isr: 0
[root@orderer kafka_2.11-2.2.2]# 

12、启动生产者

[root@orderer kafka_2.11-2.2.2]# bin/kafka-console-producer.sh --broker-list localhost:9092 --topic demo
>

13、启动消费者

新开一个会话窗口

[root@orderer src]# cd /usr/local/kafka_2.11-2.2.2/
[root@orderer kafka_2.11-2.2.2]# bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic demo

14、切换到生产者窗口输入数据

[root@orderer kafka_2.11-2.2.2]# bin/kafka-console-producer.sh --broker-list localhost:9092 --topic demo
>hello!
>

切换到消费者窗口查看

[root@orderer kafka_2.11-2.2.2]# bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic demo
hello!

消费者已经消费到数据了

 15、删除topic

[root@orderer kafka_2.11-2.2.2]# bin/kaftopics.sh --zookeeper localhost:2181 --delete --topic demo
Topic demo is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.