Hbase单节点Standalone & Pseudo Distribute
官方网站:
环境:
CentOS 7.1
hadoop-2.7.2
hbase-1.2.1
jdk-1.8
说明:仅个人测试学习记录,大神们可以一笑而过
一.jdk
二.安装hbase
1.创建用户
useradd -r -m hbadmin
echo hbadmin:hb.123|chpasswd
说明:服务启停都由hbadmin用户来操作
2.调整资源限制
cat
>/etc/security/limits.d/99-hbase-nproc.conf <<HERE
hbadmin soft fsize
unlimited
hbadmin soft cpu
unlimited
hbadmin soft
as unlimited
hbadmin soft
nofile 64000
hbadmin soft
nproc 64000
hbadmin hard
fsize unlimited
hbadmin hard
cpu unlimited
hbadmin hard
as unlimited
hbadmin hard
nofile 64000
hbadmin hard
nproc 64000
HERE
3.安装
tar -xvf hbase-1.2.1-bin.tar.gz -C /opt
mv /opt/hbase-1.2.1/ /opt/hbase
chown -R hbadmin:
/opt/hbase
三.环境变量
cat >/etc/profile.d/hbase.sh
<<HERE
export JAVA_HOME=/opt/jdk
export HBASE_HOME=/opt/hbase
export HBASE_CONF_DIR=/opt/hbase/conf
export
HBASE_PID_DIR=/opt/hbase/run
HERE
提示:还可以直接修改/opt/hbase/conf/hbase-env.sh
四.配置
Standalone模式
1.修改hbase主配置文件
conf/hbase-site.xml
![Hbase单节点Standalone <wbr>& <wbr>Pseudo <wbr>Distribute Hbase单节点Standalone <wbr>& <wbr>Pseudo <wbr>Distribute](http://simg.sinajs.cn/blog7style/images/common/sg_trans.gif)
指定的目录hbase会自动创建,无需手动另建,这里指定到
/home/hbadmin
2.启动hbase
/opt/hbase/bin/start-hbase.sh
[hbadmin@node-a ~]$
/opt/hbase/bin/start-hbase.sh
starting
master, logging to
/opt/hbase/bin/../logs/hbase-hbadmin-master-node-a.example.com.out
Java
HotSpot(TM) 64-Bit Server VM warning: ignoring option
PermSize=128m; support was removed in 8.0
Java
HotSpot(TM) 64-Bit Server VM warning: ignoring option
MaxPermSize=128m; support was removed in 8.0
[hbadmin@node-a ~]$
/opt/jdk/bin/jps
5692
HMaster
6062
Jps
[hbadmin@node-a ~]$ ls
/home/hbadmin/
hbase
zookeeper
3.hbase
shell
[hbadmin@node-a
~]$/opt/hbase/bin/hbase
shell
2016-05-30 11:26:23,230 WARN
[main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java
classes where applicable
SLF4J: Class path contains
multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See
http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of
type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell; enter 'help' for
list of supported commands.
Type "exit" to leave the HBase
Shell
Version 1.2.1,
r8d8a7107dc4ccbf36a92f64675dc60392f85c015, Wed Mar 30 11:19:21 CDT
2016
hbase(main):002:0>
create 'test','cf'
0 row(s) in 1.5690
seconds
=> Hbase::Table -
test
hbase(main):003:0>
list 'test'
TABLE
test
1 row(s) in 0.0290
seconds
=> ["test"]
hbase(main):004:0>
put
'test','row1','cf:a','value1'
0 row(s) in 0.3210
seconds
hbase(main):005:0> put
'test','row2','cf:b','value2'
0 row(s) in 0.0090
seconds
hbase(main):006:0> put
'test','row3','cf:c','value3'
0 row(s) in 0.0040
seconds
hbase(main):007:0>
scan 'test'
ROW
COLUMN+CELL
row1
column=cf:a, timestamp=1464578876525,
value=value1
row2
column=cf:b, timestamp=1464578889386,
value=value2
row3
column=cf:c, timestamp=1464578898035,
value=value3
3 row(s) in 0.0390
seconds
hbase(main):008:0>
get 'test', 'row1'
COLUMN
CELL
cf:a
timestamp=1464578876525, value=value1
1 row(s) in 0.0500
seconds
hbase(main):013:0> drop
'test'
ERROR: Table test is enabled.
Disable it first.
Here is some help for this
command:
Drop the named table. Table
must first be disabled:
hbase>
drop 't1'
hbase>
drop 'ns1:t1'
hbase(main):014:0>
disable 'test'
0 row(s) in 2.2380
seconds
hbase(main):015:0>
drop 'test'
0 row(s) in 1.2610
seconds
hbase(main):016:0>
quit
4.停止hbase
[hbadmin@node-a
~]$/opt/hbase/bin/stop-hbase.sh
stopping
hbase.......................
Pseudo
Distribute模式
1.启动hdfs
2.配置hbase
![Hbase单节点Standalone <wbr>& <wbr>Pseudo <wbr>Distribute Hbase单节点Standalone <wbr>& <wbr>Pseudo <wbr>Distribute](http://simg.sinajs.cn/blog7style/images/common/sg_trans.gif)
You do not need to create the directory in HDFS. HBase will do this
for you. If you create the directory, HBase will attempt to do a
migration, which is not what you want.
3.hdfs写入权限
注意:
hbase的拥有人要有写入hdfs的权限,否则会在启动hbase时HMaster无法启动
2016-05-31
11:04:34,165 FATAL [node-a:16000.activeMasterManager]
master.HMaster: Failed to become
active
master
org.apache.hadoop.security.AccessControlException:
Permission denied: user=hbadmin,
access=WRITE, inode="
/hbase":hadoop:supergroup:drwxr-xr-x
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
相对于本实验环境即
hbadmin用户要有hadoop用户的权限,当然,如果hbase用户直接设为hadoop用户就可以省略此步骤
su -
root
gpasswd -a
hbadmin hadoop
http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-common/FileSystemShell.html#chmod
su -
hadoop
/opt/hadoop/bin/hdfs dfs -chown -R
hadoop:hadoop /
/opt/hadoop/bin/hdfs dfs -chmod
-R 775 /
提示:也可以通过acl控制
[hadoop@node-a ~]$ /opt/hadoop/bin/hdfs dfs -ls -d /
16/05/31 11:24:09 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java
classes where applicable
drwxrwxr-x - hadoop hadoop
0 2016-05-31 11:16 /
4.ssh密钥认证
su -
hbadmin
ssh-keygen -t dsa -P '' -f
~/.ssh/id_dsa
cat ~/.ssh/id_dsa.pub >>
~/.ssh/authorized_keys
chmod 0600
~/.ssh/authorized_keys
说明:/opt/hbase/bin/start-hbase.sh脚本会通过ssh连接调用,所以要密钥认证
5.启动(或重启hbase)
/opt/hbase/bin/stop-hbase.sh
/opt/hbase/bin/start-hbase.sh
[hbadmin@node-a
~]$/opt/hadoop/bin/hadoop
fs -ls /hbase
16/05/31
11:28:16 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where
applicable
Found 7
items
drwxr-xr-x
- hbadmin hadoop
0 2016-05-31 11:16 /hbase/.tmp
drwxr-xr-x
- hbadmin hadoop
0 2016-05-31 11:17
/hbase/MasterProcWALs
drwxr-xr-x
- hbadmin hadoop
0 2016-05-31 11:16 /hbase/WALs
drwxr-xr-x
- hbadmin hadoop
0 2016-05-31 11:16 /hbase/data
-rw-r--r--
1 hbadmin hadoop
42
2016-05-31 11:16 /hbase/hbase.id
-rw-r--r--
1 hbadmin hadoop
7 2016-05-31 11:16
/hbase/hbase.version
drwxr-xr-x
- hbadmin hadoop
0 2016-05-31 11:16 /hbase/oldWALs
4.启动多个backup
HMaster(可选)
目前最大可以1 master HMaster, 9 backup HMaster
[hbadmin@node-a
~]$/opt/hbase/bin/local-master-backup.sh start 2 3 5
starting
master, logging to
/opt/hbase/bin/../logs/hbase-hbadmin-2-master-node-a.example.com.out
starting
master, logging to
/opt/hbase/bin/../logs/hbase-hbadmin-3-master-node-a.example.com.out
starting
master, logging to
/opt/hbase/bin/../logs/hbase-hbadmin-5-master-node-a.example.com.out
说明:测试之用,可以在同一台主机上的多个端口启动backup
HMaster,脚本是通过偏移量来确定端口的
每个HMaster都会占用3个端口(默认16010, 16020, and 16030)。backup
HMaster占用端口就是在master
HMaster3个端口的基础之上分别加上偏移量,如下:16012/16022/16032, 16013/16023/16033,
and 16015/16025/16035
hbadmin
54321 54307
11 13:50 pts/2 00:00:22
/opt/jdk/bin/java -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p
-XX:+UseConcMarkSweepGC -Dhbase.log.dir=/opt/hbase/bin/../logs
-Dhbase.log.file=hbase-root-2-master-config-p.example.com.log
-Dhbase.home.dir=/opt/hbase/bin/.. -Dhbase.id.str=root-2
-Dhbase.root.logger=INFO,RFA
-Djava.library.path=/opt/hadoop/lib/native
-Dhbase.security.logger=INFO,RFAS
org.apache.hadoop.hbase.master.HMaster -D
hbase.master.port=16002 -D hbase.master.info.port=16012 -D hbase.regionserver.port=16022 -D hbase.regionserver.info.port=16032 --backup start
hbadmin
54424 54410
11 13:50 pts/2 00:00:22
/opt/jdk/bin/java -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p
-XX:+UseConcMarkSweepGC -Dhbase.log.dir=/opt/hbase/bin/../logs
-Dhbase.log.file=hbase-root-3-master-config-p.example.com.log
-Dhbase.home.dir=/opt/hbase/bin/.. -Dhbase.id.str=root-3
-Dhbase.root.logger=INFO,RFA
-Djava.library.path=/opt/hadoop/lib/native
-Dhbase.security.logger=INFO,RFAS
org.apache.hadoop.hbase.master.HMaster -D
hbase.master.port=16003 -D hbase.master.info.port=16013 -D hbase.regionserver.port=16023 -D hbase.regionserver.info.port=16033 --backup start
hbadmin
54494 54480
11 13:50 pts/2 00:00:21
/opt/jdk/bin/java -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p
-XX:+UseConcMarkSweepGC -Dhbase.log.dir=/opt/hbase/bin/../logs
-Dhbase.log.file=hbase-root-5-master-config-p.example.com.log
-Dhbase.home.dir=/opt/hbase/bin/.. -Dhbase.id.str=root-5
-Dhbase.root.logger=INFO,RFA
-Djava.library.path=/opt/hadoop/lib/native
-Dhbase.security.logger=INFO,RFAS
org.apache.hadoop.hbase.master.HMaster -D
hbase.master.port=16005 -D hbase.master.info.port=16015 -D hbase.regionserver.port=16025 -D hbase.regionserver.info.port=16035 --backup start
停止backup HMaster
[hbadmin@node-a
~]$/opt/hbase/bin/local-master-backup.sh stop 2 3 5
stopping master.
stopping master.
stopping master.
5.启动additional
RegionServers(可选)
RegionServer占用两个端口(默认16200,16300)和local-master-backup.sh类似,local-regionservers.sh也可以通过偏移量来启动多个RegionServers,最多可以达到99个additional
RegionServers
[hbadmin@node-a
~]$/opt/hbase/bin/local-regionservers.sh start 2 3 4 5
starting
regionserver, logging to
/opt/hbase/bin/../logs/hbase-hbadmin-2-regionserver-node-a.example.com.out
starting
regionserver, logging to
/opt/hbase/bin/../logs/hbase-hbadmin-3-regionserver-node-a.example.com.out
starting
regionserver, logging to
/opt/hbase/bin/../logs/hbase-hbadmin-4-regionserver-node-a.example.com.out
starting
regionserver, logging to
/opt/hbase/bin/../logs/hbase-hbadmin-5-regionserver-node-a.example.com.out
停止additional
RegionServers
[hbadmin@node-a
~]$/opt/hbase/bin/local-regionservers.sh stop 2 3 4 5
stopping
regionserver.........
stopping
regionserver.......
stopping
regionserver...
stopping
regionserver.
6.web UI
每个HMaster都有个web UI,如上就有
192.168.192.10:16010
192.168.192.10:16012
192.168.192.10:16013
192.168.192.10:16015![Hbase单节点Standalone <wbr>& <wbr>Pseudo <wbr>Distribute Hbase单节点Standalone <wbr>& <wbr>Pseudo <wbr>Distribute](http://simg.sinajs.cn/blog7style/images/common/sg_trans.gif)