kafka安装、相关命令以及PHP使用
1、安装JAVA
#下载安装包 https://www.oracle.com/technetwork/java/javase/downloads/index.html tar -xzvf jdk-8u211-linux-x64.tar.gz vim /etc/profile export JAVA_HOME=/www/softs/jdk1.8 export JRE_HOME=${JAVA_HOME}/jre export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib export PATH=${JAVA_HOME}/bin:$PATH source /etc/profile java -version
2、安装zookeeper
#下载zookeeper https://zookeeper.apache.org/ #解压 tar -xzvf apache-zookeeper-3.5.5-bin.tar.gz mv apache-zookeeper-3.5.5-bin zookeeper cd /www/softs/zookeeper mkdir /www/softs/zookeeper/data #修改配置文件 cp conf/zoo_sample.cfg conf/zoo.cfg vim conf/zoo.cfg dataDir=/www/softs/zookeeper/data server.1=172.19.60.175:2888:3888 server.2=172.19.60.174:2888:3888 【服务器-1】 echo "1" > /www/softs/zookeeper/data/myid 【服务器-2】 echo "2" > /www/softs/zookeeper/data/myid #启动服务 /www/softs/zookeeper/bin/zkServer.sh start /www/softs/zookeeper/bin/zkServer.sh status /www/softs/zookeeper/bin/zkServer.sh stop /www/softs/zookeeper/bin/zkServer.sh restart 【从服务器显示状态】 Mode: follower 【主服务器显示状态】 Mode: leader
3、安装kafka
#下载 https://www.apache.org/dyn/closer.cgi?path=/kafka/1.0.0/kafka_2.12-1.0.0.tgz #解压 tar -xzvf kafka_2.12-2.2.1.tgz mv kafka_2.12-2.2.1 kafka #编辑配置文件 vim /www/softs/kafka/config/server.properties 【服务器-1】 broker.id=0 port=9092 log.dirs=/www/softs/kafka/kafka-logs listeners=PLAINTEXT://192.168.10.41:9092
4、命令
# 启动zookeeper /www/softs/kafka/bin/zookeeper-server-start.sh -daemon /www/softs/kafka/config/zookeeper.properties # 启动broker nohup /www/softs/kafka/bin/kafka-server-start.sh /www/softs/kafka/config/server.properties 1>/dev/null 2>&1 & # 停止zookeeper /www/softs/kafka/bin/zookeeper-server-stop.sh # 停止broker /www/softs/kafka/bin/kafka-server-stop.sh # 创建topic /www/softs/kafka/bin/kafka-topics.sh --create --bootstrap-server 172.19.60.175:9092,172.19.60.174:9092 --replication-factor 1 --partitions 6 --topic test # 查看所有topic /www/softs/kafka/bin/kafka-topics.sh --list --bootstrap-server 172.19.60.175:9092,172.19.60.174:9092 # 查看单个topic信息 /www/softs/kafka/bin/kafka-topics.sh --describe --bootstrap-server 172.19.60.175:9092,172.19.60.174:9092 --topic test # 给topic增加partitions /www/softs/kafka/bin/kafka-topics.sh --bootstrap-server 172.19.60.175:9092,172.19.60.174:9092 --alter --topic test --partitions 6 # 生产消息 /www/softs/kafka/bin/kafka-console-producer.sh --broker-list 172.19.60.175:9092,172.19.60.174:9092 --topic test # 消费消息 /www/softs/kafka/bin/kafka-console-consumer.sh --bootstrap-server 172.19.60.175:9092,172.19.60.174:9092 --topic test --from-beginning # 删除topic /www/softs/kafka/bin/kafka-topics.sh --bootstrap-server 172.19.60.175:9092,172.19.60.174:9092 --delete --topic test #查看消费组个数 /www/softs/kafka/bin/kafka-consumer-groups.sh --bootstrap-server 172.19.60.175:9092,172.19.60.174:9092 --list #查看消费者数量 /www/softs/kafka/bin/kafka-consumer-groups.sh --bootstrap-server 172.19.60.175:9092,172.19.60.174:9092 --describe --group test --members # 检测消费者位置 /www/softs/kafka/bin/kafka-consumer-groups.sh --bootstrap-server 172.19.60.175:9092,172.19.60.174:9092 --describe --group test # 查看消费者状态 /www/softs/kafka/bin/kafka-consumer-groups.sh --bootstrap-server 172.19.60.175:9092,172.19.60.174:9092 --describe --group test # 杀死进程 ps -ef | grep java | grep -v grep | awk '{print $2}' | xargs kill -9 # 清除日志 rm -rf /tmp/kafka-logs/ && rm -rf /tmp/zookeeper/ && rm -rf /home/kafka/logs/*
5、PHP扩展
# kafka php库 # 下载源码librdkafka https://github.com/edenhill/librdkafka.git cd librdkafka-master ./configure make && make install # 安装php kafka扩展 php-rdkafka extension # 下载源码 https://github.com/arnaud-lb/php-rdkafka.git # 编译安装 unzip php-rdkafka-master.zip cd php-rdkafka-master /www/server/php/72/bin/phpize ./configure --with-php-config=/www/server/php/72/bin/php-config make && make install # 查看.so名称 ls /www/server/php/72/lib/php/extensions/no-debug-non-zts-20170718/ # 修改php.ini extension = /www/server/php/72/lib/php/extensions/no-debug-non-zts-20170718/rdkafka.so # 重启php,查看phpinfo,搜索kafka,找到即成功 # 使用示例: https://arnaud-lb.github.io/php-rdkafka/phpdoc/rdkafka.examples.html
6、PHP使用
参考:https://github.com/arnaud-lb/php-rdkafka