storm环境搭建

线上kafka版本 0.8.2.0
线上zookeeper版本：zookeeper-3.4.5

1.配置三台服务的host
127.0.0.1 storm01
127.0.0.2 storm02
127.0.0.3 storm03

开通防火墙
#zookeeper,storm
-A INPUT -s 127.0.0.1/32 -p tcp -m multiport --dports 22:65535 -j ACCEPT
-A INPUT -s 127.0.0.2/32 -p tcp -m multiport --dports 22:65535 -j ACCEPT
-A INPUT -s 127.0.0.3/32 -p tcp -m multiport --dports 22:65535 -j ACCEPT

2.配置jdk
wget rpm包url
配置环境变量
export JAVA_HOME=/home/storm/jdk1.8.0_111
export ZOOKEEPER_HOME=/home/storm/zookeeper-3.4.10
export STORM_HOME=/home/storm/apache-storm-1.1.0
PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$STORM_HOME/bin:$PATH:$HOME/bin

3.配置zookeeper

tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/home/storm/zookeeper-3.4.10/data
dataLogDir=/home/storm/zookeeper-3.4.10/datalog
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

zookeeper.log.maxbackupindex=7

server.0=storm01:2888:3888
server.1=storm02:2888:3888
server.2=storm03:2888:3888

在dataDir下新建一个myid文件，写上对应的id

启动zookeeper
zkServer.sh start /home/storm/zookeeper-3.4.10/conf/zoo.cfg
查看zookeeper状态
zkServer.sh status

 

4.配置storm

storm.zookeeper.servers:
     - "storm01"
     - "storm02"
     - "storm03"
# 
nimbus.seeds: ["storm01"]
storm.local.dir: "/home/storm/apache-storm-1.1.0/data"
#slot数量一般以每个服务器的CPU线程数来计算。
supervisor.slots.ports:
     - 6700
     - 6701
     - 6702
     - 6703
     - 6704
     - 6705

 

同步配置

scp storm.yaml storm@storm02:/home/storm/apache-storm-1.1.0/conf/

scp storm.yaml storm@storm03:/home/storm/apache-storm-1.1.0/conf/

scp zoo.cfg storm@storm02:/home/storm/zookeeper-3.4.10/conf
scp zoo.cfg storm@storm03:/home/storm/zookeeper-3.4.10/conf

 

启动storm
storm nimbus >/dev/null 2>&1 &
storm ui >/dev/null 2>&1 &
storm supervisor >/dev/null 2>&1 &

 

错误1:
[storm@storm01 gavin]$storm nimbus
Need python version > 2.6
storm启动需要python2.6以上版本，storm中启动脚本python路径指的不正确，如果/usr/bin/python2.6不存在，$PYTHON也不存在，则使用PYTHON=/usr/bin/python这个

在第二个if后面手动指定PYTHON=/usr/local/Python-2.7.10/bin/python