Kakfa Kraft模式部署

下载包

官网https://www.apache.org/dyn/closer.cgi?path=/kafka/3.2.0/kafka_2.12-3.2.0.tgz

解压

tar -xzf kafka_2.13-3.2.0.tgz
cd kafka_2.13-3.2.0

可以查看config/kraft/README.md了解一下

生成集群ID

bin/kafka-storage.sh random-uuid > uuid
cat uuid

单机版部署

修改配置文件

vim config/kraft/server.properties

使用前面生成的uuid格式化Kafka存储目录

bin/kafka-storage.sh format -t `cat uuid` -c config/kraft/server.properties

启动

bin/kafka-server-start.sh -daemon config/kraft/server.properties
jps

测试

# 创建主机
bin/kafka-topics.sh --create --topic test --partitions 1 --replication-factor 1 --bootstrap-server localhost:9092
# 查看主题
bin/kafka-topics.sh --bootstrap-server localhost:9092 --list
# 查看主题详情
bin/kafka-topics.sh --describe --topic test --bootstrap-server localhost:9092
# 开启消费者：
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test
# 开启生产者：
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
# 查看消费者组
bin/kafka-consumer-groups.sh --list --bootstrap-server localhost:9092
# 删除topic：
bin/kafka-topics.sh --delete --topic test --bootstrap-server localhost:9092

集群版本部署

部署三台

ip映射域名分别为：EndeavourOS01 EndeavourOS02 EndeavourOS03

配置

第一台

# The role of this server. Setting this puts us in KRaft mode
# controller相当于主机、broker节点相当于从机
process.roles=broker, controller

# The node id associated with this instance's roles
# 节点 ID
node.id=1

# The connect string for the controller quorum
# 全Controller列表
controller.quorum.voters=1@EndeavourOS01:9093,2@EndeavourOS02:9093,3@EndeavourOS03:9093

# Listener name, hostname and port the broker will advertise to clients.
# If not set, it uses the value for "listeners".
# 对外暴露的地址
advertised.Listeners=PLAINTEXT://EndeavourOS01:9092

# A comma separated list of directories under which to store log files
# kafka 数据存储目录
log.dirs=/opt/kafka_2.12-3.2.0/kraft-combined-logs

第二台

# The role of this server. Setting this puts us in KRaft mode
# controller相当于主机、broker节点相当于从机
process.roles=broker, controller

# The node id associated with this instance's roles
# 节点 ID
node.id=2

# The connect string for the controller quorum
# 全Controller列表
controller.quorum.voters=1@EndeavourOS01:9093,2@EndeavourOS02:9093,3@EndeavourOS03:9093

# Listener name, hostname and port the broker will advertise to clients.
# If not set, it uses the value for "listeners".
# 对外暴露的地址
advertised.Listeners=PLAINTEXT://EndeavourOS02:9092

# A comma separated list of directories under which to store log files
# kafka 数据存储目录
log.dirs=/opt/kafka_2.12-3.2.0/kraft-combined-logs

第三台

# The role of this server. Setting this puts us in KRaft mode
# controller相当于主机、broker节点相当于从机
process.roles=broker, controller

# The node id associated with this instance's roles
# 节点 ID
node.id=3

# The connect string for the controller quorum
# 全Controller列表
controller.quorum.voters=1@EndeavourOS01:9093,2@EndeavourOS02:9093,3@EndeavourOS03:9093

# Listener name, hostname and port the broker will advertise to clients.
# If not set, it uses the value for "listeners".
# 对外暴露的地址
advertised.Listeners=PLAINTEXT://EndeavourOS03:9092

# A comma separated list of directories under which to store log files
# kafka 数据存储目录
log.dirs=/opt/kafka_2.12-3.2.0/kraft-combined-logs

集群ID

一样要先生成集群ID，根据上面操作，只在第一台生成uuid就好。后面两台使用和第一台一样的ID。操作省略

格式化 kafka 存储目录（三台节点）

三台都要执行，uuid使用一样的

bin/kafka-storage.sh format -t `cat uuid` -c config/kraft/server.properties

启动

三台都执行

bin/kafka-server-start.sh -daemon config/kraft/server.properties