docker-compose部署kafka+kafka-console

环境:
OS:Centos 7

1.创建外挂目录并修改权限
[root@node2 kafka]#mkdir -p /opt/kafka/kafka_data
[root@node2 kafka]#mkdir -p /opt/kafka/zookeeper_data
[root@node2 kafka]#cd /opt
[root@node2 kafka]#chmod -R 777 /opt/kafka

 

2.进入/opt/kafka 创建docker-compose.yml文件

[root@node2 kafka]# more docker-compose.yml 
version: '3.9'
services:
  kafka:
    image: registry.cn-shenzhen.aliyuncs.com/hxlk8s/kafka:3.4
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_BROKER_ID=0
      - KAFKA_CFG_LISTENERS=PLAINTEXT://192.168.1.106:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://192.168.1.106:9092
      - KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092
    ports:
      - "9092:9092"
    volumes:
      - "/opt/kafka/kafka_data:/bitnami"
    depends_on:
      - zookeeper
 
  zookeeper:
    image: registry.cn-shenzhen.aliyuncs.com/hxlk8s/zookeeper:3.8
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
    volumes:
      - "/opt/kafka/zookeeper_data:/bitnami"
  kafka-console:
    restart: always
    image: registry.cn-shenzhen.aliyuncs.com/hxlk8s/kafka-console:latest
    container_name: kafka-console
    ports:
      - "8080:8080"
    depends_on:
      - kafka
    environment:
     - KAFKA_BROKERS=192.168.1.106:9092

 

3.登录kafka-consle

 

4.下载kafka客户端

下载地址

wget https://downloads.apache.org/kafka/3.8.1/kafka_2.12-3.8.1.tgz

 

5.解压

tar -xvf kafka_2.12-3.8.1.tgz

 

6.查看topic

 

[root@node2 bin]# ./kafka-topics.sh --zookeeper 192.168.1.106:2181 --list
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
zookeeper is not a recognized option
joptsimple.UnrecognizedOptionException: zookeeper is not a recognized option
        at joptsimple.OptionException.unrecognizedOption(OptionException.java:108)
        at joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:510)
        at joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:56)
        at joptsimple.OptionParser.parse(OptionParser.java:396)
        at org.apache.kafka.tools.TopicCommand$TopicCommandOptions.<init>(TopicCommand.java:830)
        at org.apache.kafka.tools.TopicCommand.execute(TopicCommand.java:100)
        at org.apache.kafka.tools.TopicCommand.mainNoExit(TopicCommand.java:90)
        at org.apache.kafka.tools.TopicCommand.main(TopicCommand.java:85)

[root@node2 bin]# 
[root@node2 bin]# ./kafka-topics.sh --bootstrap-server localhost:9092 --list
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N

[root@node2 bin]# ./kafka-topics.sh --bootstrap-server localhost:9092 --list
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
my_topic_test

 

发现是因为在kafka较新版本（2.2 及更高版本）不再需要ZooKeeper连接字符串，即：–zookeeper localhost:2181
需要使用Kafka Broker的 --bootstrap-server localhost:9092来替代–zookeeper localhost:2181