docker部署zookeeper与kafka
---
version: '3'
services:
zookeeper:
image: zookeeper:3.6.3
restart: always
networks:
- kz
hostname: zookeeper
ports:
- 2181:2181
kafka:
image: 'bitnami/kafka:3.2.1'
ports:
- '9092:9092'
- '9093:9093'
restart: always
networks:
- kz
environment:
- KAFKA_BROKER_ID=1
- KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
- KAFKA_CFG_LISTENERS=CLIENT://:9092,EXTERNAL://0.0.0.0:9093
- KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://kafka:9092,EXTERNAL://10.0.0.6:9093
- KAFKA_CFG_INTER_BROKER_LISTENER_NAME=CLIENT
- KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true
- KAFKA_CFG_NUM_PARTITIONS=6
- KAFKA_CFG_DELETE_TOPIC_ENABLE=true
- KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
- ALLOW_PLAINTEXT_LISTENER=yes
depends_on:
- zookeeper
networks:
kz:
driver: bridge
注意事项:
需要将10.0.0.6修改成宿主机,方便外部访问kafka
因为需要让外部访问,所以将原配置PLAINTEXT
拆分成了内部的访问端口CLIENT和外部的访问端口EXTERNAL
拆分的环境变量是
KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
所以外部的通过9093访问,内部的通过9092访问
同时还需要把EXTERNAL修改为0.0.0.0,例如KAFKA_CFG_LISTENERS=CLIENT://:9092,EXTERNAL://0.0.0.0:9093
在映射端口,也要多映射一个9093,外部的filebeat和logstash取数据都需要从9093读取。
测试有两种
-
通过容器内部访问端口9092访问
docker-compose exec kafka kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic os --from-beginning
这里的kafka指的是容器的名称
-
通过外部端口9093访问
docker-compose exec kafka kafka-console-consumer.sh --bootstrap-server 10.0.0.6:9093 --topic os --from-beginning
9093就是通过宿主机ip访问
生产数据的方式也是类似的
本文来自博客园,作者:厚礼蝎,转载请注明原文链接:https://www.cnblogs.com/guangdelw/p/17282279.html