Ubuntu18搭建kafka单机集群

1. 配置zookeeper

https://www.cnblogs.com/convict/p/16438634.html

2. 下载kafka

http://archive.apache.org/dist/kafka/2.8.0/kafka_2.12-2.8.0.tgz

3. 解压

tar -zxvf apache-zookeeper-3.6.3-bin.tar.gz

放到opt目录下
mv zookeeper-3.6.3/ /opt

4. 配置kafka数据目录

mkdir -p /data/kafka/k1/log
mkdir -p /data/kafka/k2/log
mkdir -p /data/kafka/k3/log

5. 配置文件

cd /opt/kafka_2.12-2.8.0/config

config目录下有个server.properties，这个是模板配置文件，以此为模板复制一个server1.properties
cp server.properties server1.properties

修改server1.properties内容

broker.id=1

listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://192.168.123.201:9092  # 暴露给外部的地址

log.dirs=/data/kafka/k1/log
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183

以server1.properties，复制server2.properties
cp server.properties server2.properties

修改server2.properties

broker.id=2

listeners=PLAINTEXT://:9093
advertised.listeners=PLAINTEXT://192.168.123.201:9093

log.dirs=/data/kafka/k2/log
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183

以server1.properties，复制server3.properties
cp server.properties server3.properties

修改server3.properties

broker.id=3

listeners=PLAINTEXT://:9094
advertised.listeners=PLAINTEXT://192.168.123.201:9094

log.dirs=/data/kafka/k3/log
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183

6. 启动kafka集群

分开多个shell启动

/opt/kafka_2.12-2.8.0/bin/kafka-server-start.sh /opt/kafka_2.12-2.8.0/config/server1.properties
/opt/kafka_2.12-2.8.0/bin/kafka-server-start.sh /opt/kafka_2.12-2.8.0/config/server2.properties
/opt/kafka_2.12-2.8.0/bin/kafka-server-start.sh /opt/kafka_2.12-2.8.0/config/server3.properties

7. 查看zookeeper上的brokers

/opt/zookeeper-3.6.3/bin/zkCli.sh
ls /
ls /brokers/ids

确认kafka注册到zookeeper上了

8. 测试

cd /opt/kafka_2.12-2.8.0/bin

8.1 测试创建topic

./kafka-topics.sh --bootstrap-server 127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094 --replication-factor 1 --partitions 1 --create --topic MY_TEST_TOPIC

8.2 查看创建的topic

./kafka-topics.sh --bootstrap-server 127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094 --list

8.3 测试生产者

./kafka-console-producer.sh --bootstrap-server 127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094 --topic MY_TEST_TOPIC

8.4 测试消费者

./kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094 --topic MY_TEST_TOPIC --from-beginning