Storm 安装部署

环境要求
JDK 1.6+
java -version
Python 2.6.6+
python -V

ZooKeeper3.4.5+
storm 0.9.4+

单机模式
上传解压

$ tar xf apache-storm-0.9.4.tar.gz 
$ cd apache-storm-0.9.4

$ mkdir logs
$ bin/storm --help

 

下面分别启动ZooKeeper、Nimbus、UI、supervisor、logviewer

$ ./bin/storm dev-zookeeper >> ./logs/zk.out 2>&1 &
$ ./bin/storm nimbus >> ./logs/nimbus.out 2>&1 &
$ ./bin/storm ui >> ./logs/ui.out 2>&1 &
$ ./bin/storm supervisor >> ./logs/supervisor.out 2>&1 &
$ ./bin/storm logviewer &

 

需要多等一会儿，通过jps观察

$ jps
6966 Jps
6684 logviewer
6680 dev_zookeeper
6681 nimbus
6682 core
6683 supervisor

 

登录ui

http://node1:8080/

 

运行测试程序

$ bin/storm jar examples/storm-starter/storm-starter-topologies-0.9.4.jar storm.starter.WordCountTopology wordcount
$ bin/storm jar examples/storm-starter/storm-starter-topologies-0.9.4.jar storm.starter.WordCountTopology test

 

完全分布式

3个节点部署ZooKeeper集群 node1为nimbus

3台机器分别安装解压storm安装包，修改配置文件

$ vim conf/storm.yaml

storm.zookeeper.servers:
    - "node1"
    - "node2"
    - "node3"

storm.local.dir: "/tmp/storm"

nimbus.host: "node1"

supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703

 

在storm目录中创建logs目录

$ mkdir logs

启动ZooKeeper集群

 

node1上启动Nimbus

$ ./bin/storm nimbus >> ./logs/nimbus.out 2>&1 &
$ tail -f logs/nimbus.log
$ ./bin/storm ui >> ./logs/ui.out 2>&1 &
$ tail -f logs/ui.log

节点node2和node3启动supervisor，按照配置，每启动一个supervisor就有了4个slots

$ ./bin/storm supervisor >> ./logs/supervisor.out 2>&1 &
$ tail -f logs/supervisor.log

 

登录ui

http://node1:8080/

运行测试程序

$ bin/storm jar examples/storm-starter/storm-starter-topologies-0.9.4.jar storm.starter.WordCountTopology test

 

环境变量配置

export STORM_HOME=/usr/storm
export PATH=$PATH:$STORM_HOME/bin

 

观察关闭一个supervisor后，nimbus的重新调度
再次启动一个新的supervisor后，观察，并rebalance