AWS EC2 CentOS release 6.5 部署zookeeper、kafka、dubbo
AWS EC2 CentOS release 6.5 部署zookeeper、kafka、dubbo
参考：
http://blog.csdn.net/yizezhong/article/details/47777625
一．安装zookeeper
1) 下载zookeeper安装包
可去官网下载 ，也可通过wget命令：
[root@ip-172-31-46-4 ~]# wget http://apache.fayea.com/zookeeper/zookeeper-3.4.9/zookeeper-3.4.9.tar.gz
2) 解压
执行命令：
[root@ip-172-31-46-4 ~]# tar -zxvf zookeeper-3.4.9.tar.gz
3) 安装
执行命令：
[root@ip-172-31-46-4 local]# cd /usr/local/
[root@ip-172-31-46-4 local]# mv ~/zookeeper-3.4.9 ./
修改配置项
[root@ip-172-31-46-4 local]# cd zookeeper-3.4.9/conf
[root@ip-172-31-46-4 conf]# cp zoo_sample.cfg zoo.cfg
[root@ip-172-31-46-4 conf]# vim zoo.cfg
dataDir=/tmp/zookeeper
dataLogDir=/tmp/zookeeper/log
创建文件夹
[root@ip-172-31-46-4 conf]# mkdir -p /tmp/zookeeper
[root@ip-172-31-46-4 conf]# mkdir -p /tmp/zookeeper/log
导入环境变量
[root@ip-172-31-46-4 conf]# export ZOOKEEPER_INSTALL=/usr/local/zookeeper-3.4.9/
[root@ip-172-31-46-4 conf]# export PATH=$PATH:$ZOOKEEPER_INSTALL/bin
启动Zookeeper
[root@ip-172-31-46-4 conf]# cd /usr/local/zookeeper-3.4.9/bin/
[root@ip-172-31-46-4 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.9/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@ip-172-31-46-4 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.9/bin/../conf/zoo.cfg
Mode: standalone
启动客户端
[root@ip-172-31-46-4 bin]# ./zkCli.sh
Connecting to localhost:2181
2017-01-12 02:02:35,075 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.9-1757313, built on 08/23/2016 06:50 GMT
2017-01-12 02:02:35,080 [myid:] - INFO [main:Environment@100] - Client environment:host.name=ip-172-31-46-4.us-west-2.compute.internal
2017-01-12 02:02:35,080 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_111
2017-01-12 02:02:35,082 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2017-01-12 02:02:35,082 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-0.b15.el6_8.x86_64/jre
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper-3.4.9/bin/../build/classes:/usr/local/zookeeper-3.4.9/bin/../build/lib/*.jar:/usr/local/zookeeper-3.4.9/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper-3.4.9/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper-3.4.9/bin/../lib/netty-3.10.5.Final.jar:/usr/local/zookeeper-3.4.9/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper-3.4.9/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper-3.4.9/bin/../zookeeper-3.4.9.jar:/usr/local/zookeeper-3.4.9/bin/../src/java/lib/*.jar:/usr/local/zookeeper-3.4.9/bin/../conf:
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:os.version=2.6.32-573.26.1.el6.x86_64
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/usr/local/zookeeper-3.4.9/bin
2017-01-12 02:02:35,085 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@69d0a921
Welcome to ZooKeeper!
2017-01-12 02:02:35,123 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2017-01-12 02:02:35,275 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2017-01-12 02:02:35,317 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x159906873c60000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0]
二．安装kafka
下载
[root@ip-172-31-46-4 tools]# wget http://www-us.apache.org/dist/kafka/0.10.1.1/kafka_2.10-0.10.1.1.tgz
解压
[root@ip-172-31-46-4 tools]# tar -zxvf kafka_2.10-0.10.1.1.tgz
移动
[root@ip-172-31-46-4 tools]# mv kafka_2.10-0.10.1.1 /usr/local/
配置文件
[root@ip-172-31-46-4 tools]# cd /usr/local/kafka_2.10-0.10.1.1/config/
[root@ip-172-31-46-4 config]# vi server.properties
# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.
# FORMAT:
# listeners = security_protocol://host_name:port
# EXAMPLE:
# listeners = PLAINTEXT://your.host.name:9092
#listeners=PLAINTEXT://:9092
port=9092
# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=35.160.181.*:2181
启动停止
[root@ip-172-31-46-4 tools]# cd /usr/local/kafka_2.10-0.10.1.1/bin/
#启动
[root@ip-172-31-46-4 bin]# sh bin/kafka-server-start.sh config/server.properties &
#停止
[root@ip-172-31-46-4 bin]# sh bin/kafka-server-stop.sh
生产/消费消息
#创建topic
bin/kafka-topics.sh --create --zookeeper 35.160.181.*:2181 --replication-factor 1 --partitions 1 --topic test1
#生产消息
bin/kafka-console-producer.sh --broker-list 35.160.181.*:9092 --topic test1
#消费消息
bin/kafka-console-consumer.sh --zookeeper 35.160.181.*:2181 --topic test1 --from-beginning
错误提示：
Java Hotspot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# hs_err_pid5535.log
解决办法：
将 kafka-server-start.sh的
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
修改为
export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"
因为我的内存是1G
详细配置说明，请参考：http://www.cnblogs.com/rilley/p/5391268.html
(当然也可以使用kafka自带的zookeeper，本文只是为初学者提供教程)
三．dubbo
参考：http://www.tuicool.com/articles/fayqAvJ
使用上步安装好的zookeeper即可
dubbo-admin-2.5.4.war
http://download.csdn.net/detail/liweifengwf/7784901
参考：
http://blog.csdn.net/yizezhong/article/details/47777625
一．安装zookeeper
1) 下载zookeeper安装包
可去官网下载 ，也可通过wget命令：
[root@ip-172-31-46-4 ~]# wget http://apache.fayea.com/zookeeper/zookeeper-3.4.9/zookeeper-3.4.9.tar.gz
2) 解压
执行命令：
[root@ip-172-31-46-4 ~]# tar -zxvf zookeeper-3.4.9.tar.gz
3) 安装
执行命令：
[root@ip-172-31-46-4 local]# cd /usr/local/
[root@ip-172-31-46-4 local]# mv ~/zookeeper-3.4.9 ./
修改配置项
[root@ip-172-31-46-4 local]# cd zookeeper-3.4.9/conf
[root@ip-172-31-46-4 conf]# cp zoo_sample.cfg zoo.cfg
[root@ip-172-31-46-4 conf]# vim zoo.cfg
dataDir=/tmp/zookeeper
dataLogDir=/tmp/zookeeper/log
创建文件夹
[root@ip-172-31-46-4 conf]# mkdir -p /tmp/zookeeper
[root@ip-172-31-46-4 conf]# mkdir -p /tmp/zookeeper/log
导入环境变量
[root@ip-172-31-46-4 conf]# export ZOOKEEPER_INSTALL=/usr/local/zookeeper-3.4.9/
[root@ip-172-31-46-4 conf]# export PATH=$PATH:$ZOOKEEPER_INSTALL/bin
启动Zookeeper
[root@ip-172-31-46-4 conf]# cd /usr/local/zookeeper-3.4.9/bin/
[root@ip-172-31-46-4 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.9/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@ip-172-31-46-4 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.9/bin/../conf/zoo.cfg
Mode: standalone
启动客户端
[root@ip-172-31-46-4 bin]# ./zkCli.sh
Connecting to localhost:2181
2017-01-12 02:02:35,075 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.9-1757313, built on 08/23/2016 06:50 GMT
2017-01-12 02:02:35,080 [myid:] - INFO [main:Environment@100] - Client environment:host.name=ip-172-31-46-4.us-west-2.compute.internal
2017-01-12 02:02:35,080 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_111
2017-01-12 02:02:35,082 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2017-01-12 02:02:35,082 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-0.b15.el6_8.x86_64/jre
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper-3.4.9/bin/../build/classes:/usr/local/zookeeper-3.4.9/bin/../build/lib/*.jar:/usr/local/zookeeper-3.4.9/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper-3.4.9/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper-3.4.9/bin/../lib/netty-3.10.5.Final.jar:/usr/local/zookeeper-3.4.9/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper-3.4.9/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper-3.4.9/bin/../zookeeper-3.4.9.jar:/usr/local/zookeeper-3.4.9/bin/../src/java/lib/*.jar:/usr/local/zookeeper-3.4.9/bin/../conf:
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:os.version=2.6.32-573.26.1.el6.x86_64
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2017-01-12 02:02:35,083 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/usr/local/zookeeper-3.4.9/bin
2017-01-12 02:02:35,085 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@69d0a921
Welcome to ZooKeeper!
2017-01-12 02:02:35,123 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2017-01-12 02:02:35,275 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2017-01-12 02:02:35,317 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x159906873c60000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0]
二．安装kafka
下载
[root@ip-172-31-46-4 tools]# wget http://www-us.apache.org/dist/kafka/0.10.1.1/kafka_2.10-0.10.1.1.tgz
解压
[root@ip-172-31-46-4 tools]# tar -zxvf kafka_2.10-0.10.1.1.tgz
移动
[root@ip-172-31-46-4 tools]# mv kafka_2.10-0.10.1.1 /usr/local/
配置文件
[root@ip-172-31-46-4 tools]# cd /usr/local/kafka_2.10-0.10.1.1/config/
[root@ip-172-31-46-4 config]# vi server.properties
# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.
# FORMAT:
# listeners = security_protocol://host_name:port
# EXAMPLE:
# listeners = PLAINTEXT://your.host.name:9092
#listeners=PLAINTEXT://:9092
port=9092
# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=35.160.181.*:2181
启动停止
[root@ip-172-31-46-4 tools]# cd /usr/local/kafka_2.10-0.10.1.1/bin/
#启动
[root@ip-172-31-46-4 bin]# sh bin/kafka-server-start.sh config/server.properties &
#停止
[root@ip-172-31-46-4 bin]# sh bin/kafka-server-stop.sh
生产/消费消息
#创建topic
bin/kafka-topics.sh --create --zookeeper 35.160.181.*:2181 --replication-factor 1 --partitions 1 --topic test1
#生产消息
bin/kafka-console-producer.sh --broker-list 35.160.181.*:9092 --topic test1
#消费消息
bin/kafka-console-consumer.sh --zookeeper 35.160.181.*:2181 --topic test1 --from-beginning
错误提示：
Java Hotspot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# hs_err_pid5535.log
解决办法：
将 kafka-server-start.sh的
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
修改为
export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M"
因为我的内存是1G
详细配置说明，请参考：http://www.cnblogs.com/rilley/p/5391268.html
(当然也可以使用kafka自带的zookeeper，本文只是为初学者提供教程)
三．dubbo
参考：http://www.tuicool.com/articles/fayqAvJ
使用上步安装好的zookeeper即可
dubbo-admin-2.5.4.war
http://download.csdn.net/detail/liweifengwf/7784901
