storm集群配置
环境：centos6.4
软件：
jzmq-master-----java与c++通讯的桥梁，有了它，就可以使用zeromp了
storm-0.8.2
zeromq-2.1.7-----号称史上最牛逼的消息队列（用c++写的）
zookeeper-3.4.5
主机配置：
1.修改主机名：
vim /etc/sysconfig/network
NETWORKING=yes HOSTNAME=MASTER
2.修改IP：
vim /etc/sysconfig/network-scripts/ifcfg-eth0
BOOTPROTO="static" HWADDR="00:0C:29:FC:62:B6" IPV6INIT="yes" NM_CONTROLLED="yes" ONBOOT="yes" TYPE="Ethernet" UUID="d0731a46-36df-4ab1-9bac-42bb151acc41" IPADDR="192.168.1.250" NETMASK="255.255.255.0" GATEWAY="192.168.1.1" DNS1="8.8.8.8" DNS2="8.8.4.4"
3.修改主机和IP的映射关系
vim /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.1.250 MASTER 192.168.1.251 SLAVE-1 192.168.1.252 SLAVE-2
4.关闭防火墙
vim /etc/sysconfig/selinux
# This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted
继续执行命令：
service iptables stop
chkconfig iptables off
reboot
5.上传jdk安装包
6.解压安装
tar -xzvf jdk-8u151-linux-i586.tar.gz
7.配置环境变量
vim /etc/profile
在文件最后面添加：
export JAVA_HOME=/usr/java/jdk1.6.0_45 export CLASSPATH=$JAVA_HOME/lib export PATH=$PATH:$JAVA_HOME/bin
更新配置：
source /etc/profile
上传zeromq-2.1.7.tar.gz、jzmq-master.zip、storm-0.8.2.zip、zookeeper-3.4.5.tar.gz等文件
1.编译安装ZMQ：
tar -xzf zeromq-2.1.7.tar.gz cd zeromq-2.1.7 ./configure
#编译可能会出错：configure: error: Unable to find a working C++ compiler
#安装一下依赖的rpm包：libstdc++-devel gcc-c++
可以上网的情况下：
yum install gcc-c++
虚拟机不能上网情况：首先到http://mirrors.163.com/centos/6.4/os/x86_64/Packages/ 下载rpm
rpm -i libstdc++-devel-4.4.7-3.el6.x86_64.rpm rpm -i gcc-c++-4.4.7-3.el6.x86_64.rpm rpm -i libuuid-devel-2.17.2-12.9.el6.x86_64.rpm yum install libuuid-devel ./configure make make install
2.编译安装JZMQ：
unzip jzmq-master.zip
cd jzmq
./autogen.sh
#报错:autogen.sh: error: could not find libtool. libtool is required to run autogen.sh. 缺少libtool
yum install libtool
或者手动安装
rpm -i autoconf-2.63-5.1.el6.noarch.rpm rpm -i automake-1.11.1-4.el6.noarch.rpm rpm -i libtool-2.2.6-15.5.el6.x86_64.rpm ./configure make make install
3.编译安装zookeeper：
将zookeeper-3.4.6/conf目录下的zoo_sample.cfg文件拷贝一份，命名为为“zoo.cfg”
tar –zxvf zookeeper-3.4.5.tar.gz
cp zoo_sample.cfg zoo.cfg
./zkServer.sh start
4.下载并解压Storm发布版本
下一步，需要在Nimbus和Supervisor机器上安装Storm发行版本。
1. 下载Storm发行版本
wget https://dl.dropbox.com/u/133901206/storm-0.8.2.zip
2. 解压到安装目录下：
unzip storm-0.8.1.zip
修改配置文件 conf/storm.yaml
storm.zookeeper.servers:
- "192.168.1.222"
- "192.168.1.223"
- "192.168.1.224"
- "192.168.1.225"
nimbus.host: "192.168.2.222"
ui.port: 9090
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
# - org.mycompany.MyType
# - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
# - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
# - "server1"
# - "server2"
#
storm.local.dir: "/home/storm/storm_workdir"
java.library.path: "/usr/local/lib:/usr/lib"
./storm nimbus ./storm ui > /dev/null 2>&1 &
从机配置
storm.zookeeper.servers:
- "主机ip地址或者hostname"
nimbus.host: "主机ip地址或者hostname"
ui.port: 9090
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
# - org.mycompany.MyType
# - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
# - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
# - "server1"
# - "server2"
#
storm.local.dir: "/home/storm/storm_workdir"
java.library.path: "/usr/local/lib:/usr/lib"
storm supervisor &
执行jps 命令
在没有运行任务时，我们必须应该要看到4个进程：
QuorumPeerMain、nimbus、core、supervisor
使用浏览器:
http://192.168.1.222:9090/ 可以看到Storm UI 界面
