(一) storm的集群安装与配置
机器:
192.168.180.101
192.168.187.16
需要准备的软件有:
zookeeper(zookeeper-3.4.4.tar.gz),storm(storm-0.8.1.zip) ,jdk
1、配置zookeeper
解压zookeeper,将conf目录下的zoo_sample.cfg 重命名为:zoo.cfg
修改后内容为:
# The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/data/zookeeper/data dataLogDir=/data/zookeeper/log # the port at which the clients will connect clientPort=2181 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1 server.1=192.168.187.16:2888:3888 server.2=192.168.180.101:2888:3888
具体配置可以参看:
http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_configuration
注意,最后两行的配置:
格式为:server.id=host:port:port
id只能为数字 1-255,同时需要在 dataDir目录下面新建一个文件名为myid的文件,里面的内容只有一行:"id"
Every machine that is part of the ZooKeeper ensemble should know about every other machine in the ensemble. You accomplish this with the series of lines of the form server.id=host:port:port. The parameters host and port are straightforward. You attribute the server id to each machine by creating a file named myid, one for each server, which resides in that server's data directory, as specified by the configuration file parameter dataDir.
接着还需要添加环境变量:
export ZOOKEEPER_HOME=/home/zhxia/apps/db/zookeeper
两台机器上环境配置相同,但是myid文件内的id指不一样
2、配置storm
解压storm
进入conf目录,编辑storm.yaml文件
########## These MUST be filled in for a storm configuration storm.zookeeper.servers: - "192.168.187.16" - "192.168.180.101" nimbus.host: "192.168.187.16" storm.local.dir: "/data/storm/data" ##### These may optionally be filled in: # List of custom serializations # topology.kryo.register: # - org.mycompany.MyType # - org.mycompany.MyType2: org.mycompany.MyType2Serializer # ## List of custom kryo decorators # topology.kryo.decorators: # - org.mycompany.MyDecorator # Locations of the drpc servers # drpc.servers: # - "127.0.0.1" #- "server2" ## to nimbus #nimbus.childopts: "-Xmx1024m" # ## to supervisor #supervisor.childopts: "-Xmx1024m" # ## to worker #worker.childopts: "-Xmx768m"
配置完成之后,开始启动zookeeper和storm
启动zookeeper
bin/zkServer.sh start
启动storm
bin/storm nimbus
bin/storm supervisor
bin/storm ui
浏览器打开: http://localhost:8080 查看集群的运行状态