Storm 单机版环境搭建
1 需要安装的软件
要使用storm首先要安装以下工具:python、zookeeper、zeromq、jzmq、storm
1.1 安装zeromq
wget http://download.zeromq.org/zeromq-2.2.0.tar.gz ##http://zeromq.org tar zxf zeromq-2.2.0.tar.gz cd zeromq-2.2.0 ./configure make make install
1.2 安装jzmq
wget https://github.com/zeromq/jzmq.git ./autogen.sh
./configure
make
make install
1.3 安装Zookeeper
http://apache.fayea.com/zookeeper/stable/
1.上传zk安装包
2.解压
3.配置
3.1添加一个zoo.cfg配置文件
$ZOOKEEPER/conf
mv zoo_sample.cfg zoo.cfg
3.2修改配置文件(zoo.cfg)
dataDir=/itcast/zookeeper-3.4.5/data
server.1=主机名:2888:3888
在(dataDir=/itcast/zookeeper-3.4.5/data)创建一个myid文件,里面内容是server.N中的N(server.2里面内容为2)
echo "5" > myid
4.启动zk
./zkServer.sh start
1.4 安装Storm
1 http://www.apache.org/dyn/closer.lua/storm/apache-storm-0.9.6/apache-storm-0.9.6.tar.gz
2 修改conf/storm.yaml 文件, 在下面加入下面的内容:
storm.zookeeper.servers:
- "192.168.67.133"
storm.zookeeper.port: 2181
ui.port: 18080
nimbus.host: "192.168.67.133"
storm.local.dir: "/home/sms/apache-storm-0.9.6/tmp"
supervisor.slots.ports:
- 6700
- 6701
3 创建tmp文件夹,对应storm.local.dir
4 最后,配置用户的profile,加入 ZOOKEEPER_HOME 和 STORM_HOME 方便使用相关shell 命令
说明一下:
storm.local.dir表示storm需要用到的本地目录。
nimbus.host表示那一台机器是master机器,即 nimbus。
storm.zookeeper.servers表示哪几台机器是zookeeper服务器。
storm.zookeeper.port表示zookeeper的端口号,这里一定要与zookeeper配置的端口号一致,否则会出现通信错误,切记切记。
当然你也可以配 superevisor.slot.port,supervisor.slots.ports表示supervisor节点的槽数,就是最多能跑几个 worker进程(每个sprout或bolt默认只启动一个worker,但是可以通过conf修改成多个)。
2 启动和关闭Storm
1 启动ZOOPKEEPER zkServer.sh start 2 启动NIMBUS storm nimbus & 3 启动SUPERVISOR storm supervisor & 4 启动UI storm ui & 5 部署TOPOLOGY storm jar storm-starter-topologies-0.9.6.jar storm.starter.ExclamationTopology test 6 删除TOPOLOGY storm kill {toponame} 7 激活TOPOLOGY storm active {toponame} 8 不激活TOPOLOGY storm deactive {toponame} 9 列出所有TOPOLOGY storm list
core 对应的进程是Storm UI Jps 对应的进程是Java jps nimbus 对应的进程是Storm nimbus supervisor 对应的进程是Storm supervisor worker 对应的线程是Storm main函数里面设置的 conf.setNumWorkers(2) ,所以本例中有2个worker。 QuorumPeerMain 对应的进程是zkServer.sh 其中在nimbus上nimbus、 core和 QuorumPeerMain 这三个进程必须启动,如果启动失败,需要研究一下日志中失败的原因,按照上述的顺序再启动一次。否则 会导致下一步的topology提交失败,切记。
3 UI查看
浏览器输入 本机Ip:18080, 即可
4 注意事项
zeromq和jzmq的安装过程中,需要依赖很多文件,请保持联网状态。若编译失败,请根据说明自行安装相关的依赖包。
5 问题汇集

0 down vote favorite I'm trying to setup storm 0.9.2 in CentOS 5.8 before I install dependency jzmq, zeromq 4.0.4 this is my error log File "./storm", line 51 normclasspath = cygpath if sys.platform == 'cygwin' else identity ^ SyntaxError: invalid syntax linux centos storm shareimprove this question edited Mar 23 at 22:53 skaffman 248k56563620 asked Jul 31 '14 at 2:15 user3462081 11 i try storm 0.9.1 is work! – user3462081 Jul 31 '14 at 2:37 1 It's not necessary to install zeromq since 0.9.0.1. – halfelf Jul 31 '14 at 13:08 ---------------------------------------------- 2 Answers active oldest votes up vote 1 down vote This worked for my Redhat 6 install on Python 3.4. I changed the hashbang in the /bin/storm file from: #!/usr/bin/python to: #!/usr/bin/env python shareimprove this answer edited Mar 23 at 22:40 answered Feb 9 at 19:40 billz 112 add a comment up vote 0 down vote This is a python version issue. The storm file is a python script file. Seems storm updated the python version in 0.9.2. Python 2.6 or higher could resolve this problem. shareimprove this answer answered Aug 7 '14 at 7:02 DeepNightTwo 1,18531736 i install python 2.7 and reruning 0.9.2 but same error – user3462081 Sep 4 '14 at 5:51 make sure you are using python2.7. Type python -version to check the current python version. If it's not 2.7, you can sudo move python27 python. – DeepNightTwo Sep 5 '14 at 8:15
6 参考资料
0 storm 入门 http://storm.apache.org/releases/0.9.6/Setting-up-a-Storm-cluster.html
1 storm 官方安装文档 http://storm.apache.org/documentation/Documentation.html
2 storm 启动 http://www.blogjava.net/paulwong/archive/2013/09/11/403942.html
3 storm 配置 http://www.xuebuyuan.com/1932716.html
4 storm 原理 http://my.oschina.net/zhangglin/blog/352032?fromerr=DlnSzaed
5 storm 相关软件网盘下载地址 http://pan.baidu.com/s/1ntXpqdz
7 zookper配置 http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html
8 zookper配置 http://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#ch_zkSessions
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】凌霞软件回馈社区,博客园 & 1Panel & Halo 联合会员上线
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】博客园社区专享云产品让利特惠,阿里云新客6.5折上折
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步