Kafka0.10.0安装配置

1 解压文件

　　tar -zvxf kafka_2.11-0.10.0.0.tgz

2 修改配置server.properties

　　vim server.properties

　　broker.id=1

　　zookeeper.connect=hdp1:2181,hdp2:2181,hdp3:2181

3 修改配置 zookeeper.properties

　　 vim zookeeper.properties　　

　　

# the directory where the snapshot is stored.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181

 

4 复制到hdp2，hdp3节点修改broker.id

　　修改配置server.properties  分别修改为2 ，3 

　　vim server.properties

　　broker.id=2

　　zookeeper.connect=hdp1:2181,hdp2:2181,hdp3:2181

 

5 编辑集群启动脚本

　　

#!/bin/bash
for host in hdp1 hdp2 hdp3
do
ssh $host "source /etc/profile;/mnt/software/kafka_2.11-0.10.0.0/bin/kafka-server-start.sh -daemon /mnt/software/kafka_2.11-0.10.0.0/config/server.properties"
done
sleep 3
for host in hdp1 hdp2 hdp3
do
echo "---------"
ssh $host "source /etc/profile;jps"
done

 

6 启动测试

[root@hdp1 /mnt/software/shell]#start-all-kafka.sh

7 创建topic

  kafka-topics.sh --create --zookeeper hdp1:2181,hdp2:2181,hdp3:2181 --replication-factor 1 --partitions 1 --topic test

 

8 查看topic

[root@hdp1 /mnt/software/shell]#kafka-topics.sh --list --zookeeper hdp1:2181,hdp2:2181,hdp3:2181
__consumer_offsets
cmcc
hellokafka
hellokafka1
test
topicA
topicB

 

9 之前需要自行安装好zookeeper集群，hdp1,hdp2,hdp3