1.安装hbase
参考:http://hbase.apache.org/book.html#quickstart
一.下载hbase去apache下载hbase,然后解压到/usr/local/hbase-1.1.3,设置HBASE_HOME=/usr/local/hbase-1.1.3
二.安装单机版本
修改conf/hbase-site.xml
加入:
<configuration>
<property>
<name>hbase.rootdir</name>
<value>file:///home/testuser/hbase</value> #改成自己的目录
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name> #改成自己的目录
<value>/home/testuser/zookeeper</value>
</property>
</configuration>
修改conf/hbase-env.sh,添加export JAVA_HOME=/usr/local/jdk1.7.0_79
然后就可以测试了:
#bin/start-base.sh
进入shell
bin/hbase shell
三.安装伪分布版
1.关闭hbase
#bin/stop-hbase.sh
2.修改conf/hbase-site.xml 加入:
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://localhost:8020/hbase</value> #改成自己的端口,我是的node1:9000
</property>
四.安装分布式版本
1.机器环境
192.168.75.10 | namenode resourcesManager regionserver zookeeper | centos6.4 x86_64 |
192.168.75.11 | datanode nodemanager zookeeper regionserver | centos6.4 x86_64 |
192.168.75.12 | datanode nodemanager zookeeper regionserver | centos6.4 x86_64 |
1.设置conf/hbase-env.sh
添加:export HBASE_MANAGES_ZK=true
2.配置regionserver
[root@node1 hbase-1.1.3]# vi conf/regionservers
node2
node3
3.配置zookeeper
修改conf/hbase-site.xml如下:
<configuration>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/local/hbase-1.1.3/zookeeper</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://node1:9000/hbase</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>node1,node2,node3</value>
</property>
</configuration>
4.将配置好的整个hbase目录复制到node2 \node3上
[root@node2~] #scp -r root@node1:/usr/local/hbase-1.1.3 /usr/local
[root@node3~] #scp -r root@node1:/usr/local/hbase-1.1.3 /usr/local
5.测试
在node1上启动:
[root@node1 hbase-1.1.3]# bin/start-hbase.sh
node3: starting zookeeper, logging to /usr/local/hbase-1.1.3/bin/../logs/hbase-root-zookeeper-node3.out
node1: starting zookeeper, logging to /usr/local/hbase-1.1.3/bin/../logs/hbase-root-zookeeper-node1.out
node2: starting zookeeper, logging to /usr/local/hbase-1.1.3/bin/../logs/hbase-root-zookeeper-node2.out
starting master, logging to /usr/local/hbase-1.1.3/logs/hbase-root-master-node1.out
node2: starting regionserver, logging to /usr/local/hbase-1.1.3/bin/../logs/hbase-root-regionserver-node2.out
node3: starting regionserver, logging to /usr/local/hbase-1.1.3/bin/../logs/hbase-root-regionserver-node3.out
zookeeper先在三个结点启动,然后是master和resionserver
查看进程:
[root@node1 hbase-1.1.3]# jps
29474 HMaster
18857 NameNode
29412 HQuorumPeer
19020 SecondaryNameNode
19154 ResourceManager
29590 Jps
[root@node2 hbase-1.1.3]# jps
5267 HQuorumPeer
4569 NodeManager
5461 Jps
4474 DataNode
[root@node3 hbase-1.1.3]# jps
4507 NodeManager
5447 HQuorumPeer
4412 DataNode
5649 Jps
node2 node3上HRegionServer没用启动,查看日志:
2015-12-28 23:19:02,325 FATAL [regionserver/node2/192.168.75.11:16020] regionserver.HRegionServer: Master rejected startup because clock is out of sync
org.apache.hadoop.hbase.ClockOutOfSyncException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server node2,16020,1451315935495 has been rejected; Reported time is too far out of sync with master. Time difference of 282196927ms > max allowed of 30000ms
at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:388)
发现节点之间时间不同步。
同步时间:
date -s "20160225 18:44:00"
再查看进程:
[root@node1 hbase-1.1.3]# jps
31058 HMaster
18857 NameNode
31286 Jps
30994 HQuorumPeer
19020 SecondaryNameNode
19154 ResourceManager
[root@node2 logs]# jps
5815 HQuorumPeer
5897 HRegionServer
4569 NodeManager
4474 DataNode
6041 Jps
[root@node3 hbase-1.1.3]# jps
4507 NodeManager
5994 Jps
5820 HRegionServer
4412 DataNode
5732 HQuorumPeer
发现HRegionServer进程已经启动
hbase web页面:
五、集群配置知识点
1.配置文件
(1)hbase的配置文件放在conf下。hbase-site.xml是核心配置文件,该文件覆盖hbase-deault.xml,可以参照hbase-default-tempate.xml来配置。
(2)hbase-policy.xml用于hbase的安全配置。
(3)regionservers,配置哪些机器启动regionserver
注意:所有节点上的配置文件要一致,可以从一个节点复制conf到其它节点
2.hbase集群时间需要一致,相差过大会导致regionserver失败。可以在集群内部弄一个ntp服务器定期同步时间。
3.linux系统的限制
hbase需要同时打开大量的文件,很多linux系统的同一个用户同时打开的文件数默认是1024,可能不够用而出错。
2010-04-06 03:04:37,542 INFO org.apache.hadoop.hdfs.DFSClient: Exception increateBlockOutputStream java.io.EOFException
2010-04-06 03:04:37,542 INFO org.apache.hadoop.hdfs.DFSClient: Abandoning block blk_-6935524980745310745_1391901
用ulimit -n查看。建议修改为10240.
4.