Hbase单节点Standalone & Pseudo Distribute

Hbase单节点Standalone & Pseudo Distribute

官方网站:


环境:
CentOS 7.1
hadoop-2.7.2
hbase-1.2.1
jdk-1.8


说明:仅个人测试学习记录,大神们可以一笑而过

一.jdk
jdk版本兼容性测试,请看http://hbase.apache.org/book.html#java


二.安装hbase
1.创建用户
useradd -r -m hbadmin
echo hbadmin:hb.123|chpasswd
说明:服务启停都由hbadmin用户来操作
2.调整资源限制
cat >/etc/security/limits.d/99-hbase-nproc.conf <<HERE
hbadmin soft fsize unlimited 
hbadmin soft cpu unlimited 
hbadmin soft as unlimited 
hbadmin soft nofile 64000 
hbadmin soft nproc 64000 
hbadmin hard fsize unlimited 
hbadmin hard cpu unlimited 
hbadmin hard as unlimited 
hbadmin hard nofile 64000 
hbadmin hard nproc 64000 
HERE
3.安装
tar -xvf hbase-1.2.1-bin.tar.gz -C /opt
mv /opt/hbase-1.2.1/ /opt/hbase
chown -R hbadmin: /opt/hbase


三.环境变量
cat >/etc/profile.d/hbase.sh <<HERE
export JAVA_HOME=/opt/jdk
export HBASE_HOME=/opt/hbase
export HBASE_CONF_DIR=/opt/hbase/conf
export HBASE_PID_DIR=/opt/hbase/run
HERE
提示:还可以直接修改/opt/hbase/conf/hbase-env.sh


四.配置
Standalone模式
1.修改hbase主配置文件
conf/hbase-site.xml
Hbase单节点Standalone <wbr>& <wbr>Pseudo <wbr>Distribute
指定的目录hbase会自动创建,无需手动另建,这里指定到/home/hbadmin

2.启动hbase
/opt/hbase/bin/start-hbase.sh

[hbadmin@node-a ~]$ /opt/hbase/bin/start-hbase.sh 
starting master, logging to /opt/hbase/bin/../logs/hbase-hbadmin-master-node-a.example.com.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
[hbadmin@node-a ~]$ /opt/jdk/bin/jps
5692 HMaster
6062 Jps
[hbadmin@node-a ~]$ ls /home/hbadmin/
hbase  zookeeper

3.hbase shell
[hbadmin@node-a ~]$/opt/hbase/bin/hbase shell
2016-05-30 11:26:23,230 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell; enter 'help' for list of supported commands.
Type "exit" to leave the HBase Shell
Version 1.2.1, r8d8a7107dc4ccbf36a92f64675dc60392f85c015, Wed Mar 30 11:19:21 CDT 2016

hbase(main):002:0> create 'test','cf'
0 row(s) in 1.5690 seconds

=> Hbase::Table - test
hbase(main):003:0> list 'test'
TABLE                                                                                                                
test                                                                                                                 
1 row(s) in 0.0290 seconds

=> ["test"]
hbase(main):004:0> put 'test','row1','cf:a','value1'
0 row(s) in 0.3210 seconds

hbase(main):005:0> put 'test','row2','cf:b','value2'
0 row(s) in 0.0090 seconds

hbase(main):006:0> put 'test','row3','cf:c','value3'
0 row(s) in 0.0040 seconds

hbase(main):007:0> scan 'test'
ROW                            COLUMN+CELL                                                                           
 row1                          column=cf:a, timestamp=1464578876525, value=value1                                    
 row2                          column=cf:b, timestamp=1464578889386, value=value2                                    
 row3                          column=cf:c, timestamp=1464578898035, value=value3                                    
3 row(s) in 0.0390 seconds

hbase(main):008:0> get 'test', 'row1'
COLUMN                         CELL                                                                                  
 cf:a                          timestamp=1464578876525, value=value1                                                 
1 row(s) in 0.0500 seconds

hbase(main):013:0> drop 'test'

ERROR: Table test is enabled. Disable it first.

Here is some help for this command:
Drop the named table. Table must first be disabled:
  hbase> drop 't1'
  hbase> drop 'ns1:t1'


hbase(main):014:0> disable 'test'
0 row(s) in 2.2380 seconds

hbase(main):015:0> drop 'test'
0 row(s) in 1.2610 seconds

hbase(main):016:0> quit

4.停止hbase
[hbadmin@node-a ~]$/opt/hbase/bin/stop-hbase.sh 
stopping hbase.......................


Pseudo Distribute模式
1.启动hdfs

2.配置hbase
Hbase单节点Standalone <wbr>& <wbr>Pseudo <wbr>Distribute

You do not need to create the directory in HDFS. HBase will do this for you. If you create the directory, HBase will attempt to do a migration, which is not what you want.


3.hdfs写入权限

注意: hbase的拥有人要有写入hdfs的权限,否则会在启动hbase时HMaster无法启动

2016-05-31 11:04:34,165 FATAL [node-a:16000.activeMasterManager] master.HMaster: Failed to become active 

master

org.apache.hadoop.security.AccessControlException: Permission denied: user=hbadmin, access=WRITE, inode="

/hbase":hadoop:supergroup:drwxr-xr-x

        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

        at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)



相对于本实验环境即

hbadmin用户要有hadoop用户的权限,当然,如果hbase用户直接设为hadoop用户就可以省略此步骤

su - root

gpasswd -a hbadmin hadoop

http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-common/FileSystemShell.html#chmod

su - hadoop

/opt/hadoop/bin/hdfs dfs -chown -R hadoop:hadoop /

/opt/hadoop/bin/hdfs dfs -chmod -R 775 /
提示:也可以通过acl控制
[hadoop@node-a ~]$ /opt/hadoop/bin/hdfs dfs -ls -d /
16/05/31 11:24:09 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
drwxrwxr-x   - hadoop hadoop          0 2016-05-31 11:16 /


4.ssh密钥认证

su - hbadmin

ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

chmod 0600 ~/.ssh/authorized_keys

说明:/opt/hbase/bin/start-hbase.sh脚本会通过ssh连接调用,所以要密钥认证



5.启动(或重启hbase)

/opt/hbase/bin/stop-hbase.sh 

/opt/hbase/bin/start-hbase.sh 


[hbadmin@node-a ~]$/opt/hadoop/bin/hadoop fs -ls /hbase

16/05/31 11:28:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 7 items
drwxr-xr-x   - hbadmin hadoop          0 2016-05-31 11:16 /hbase/.tmp
drwxr-xr-x   - hbadmin hadoop          0 2016-05-31 11:17 /hbase/MasterProcWALs
drwxr-xr-x   - hbadmin hadoop          0 2016-05-31 11:16 /hbase/WALs
drwxr-xr-x   - hbadmin hadoop          0 2016-05-31 11:16 /hbase/data
-rw-r--r--   1 hbadmin hadoop         42 2016-05-31 11:16 /hbase/hbase.id
-rw-r--r--   1 hbadmin hadoop          7 2016-05-31 11:16 /hbase/hbase.version
drwxr-xr-x   - hbadmin hadoop          0 2016-05-31 11:16 /hbase/oldWALs


4.启动多个backup HMaster(可选)
目前最大可以1 master HMaster, 9 backup HMaster
[hbadmin@node-a ~]$/opt/hbase/bin/local-master-backup.sh start 2 3 5
starting master, logging to /opt/hbase/bin/../logs/hbase-hbadmin-2-master-node-a.example.com.out
starting master, logging to /opt/hbase/bin/../logs/hbase-hbadmin-3-master-node-a.example.com.out
starting master, logging to /opt/hbase/bin/../logs/hbase-hbadmin-5-master-node-a.example.com.out
说明:测试之用,可以在同一台主机上的多个端口启动backup HMaster,脚本是通过偏移量来确定端口的
每个HMaster都会占用3个端口(默认16010, 16020, and 16030)。backup HMaster占用端口就是在master HMaster3个端口的基础之上分别加上偏移量,如下:16012/16022/16032, 16013/16023/16033, and 16015/16025/16035
hbadmin      54321  54307 11 13:50 pts/2    00:00:22 /opt/jdk/bin/java -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p -XX:+UseConcMarkSweepGC -Dhbase.log.dir=/opt/hbase/bin/../logs -Dhbase.log.file=hbase-root-2-master-config-p.example.com.log -Dhbase.home.dir=/opt/hbase/bin/.. -Dhbase.id.str=root-2 -Dhbase.root.logger=INFO,RFA -Djava.library.path=/opt/hadoop/lib/native -Dhbase.security.logger=INFO,RFAS org.apache.hadoop.hbase.master.HMaster -D hbase.master.port=16002 -D hbase.master.info.port=16012 -D hbase.regionserver.port=16022 -D hbase.regionserver.info.port=16032 --backup start
hbadmin      54424  54410 11 13:50 pts/2    00:00:22 /opt/jdk/bin/java -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p -XX:+UseConcMarkSweepGC -Dhbase.log.dir=/opt/hbase/bin/../logs -Dhbase.log.file=hbase-root-3-master-config-p.example.com.log -Dhbase.home.dir=/opt/hbase/bin/.. -Dhbase.id.str=root-3 -Dhbase.root.logger=INFO,RFA -Djava.library.path=/opt/hadoop/lib/native -Dhbase.security.logger=INFO,RFAS org.apache.hadoop.hbase.master.HMaster -D hbase.master.port=16003 -D hbase.master.info.port=16013 -D hbase.regionserver.port=16023 -D hbase.regionserver.info.port=16033 --backup start
hbadmin      54494  54480 11 13:50 pts/2    00:00:21 /opt/jdk/bin/java -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p -XX:+UseConcMarkSweepGC -Dhbase.log.dir=/opt/hbase/bin/../logs -Dhbase.log.file=hbase-root-5-master-config-p.example.com.log -Dhbase.home.dir=/opt/hbase/bin/.. -Dhbase.id.str=root-5 -Dhbase.root.logger=INFO,RFA -Djava.library.path=/opt/hadoop/lib/native -Dhbase.security.logger=INFO,RFAS org.apache.hadoop.hbase.master.HMaster -D hbase.master.port=16005 -D hbase.master.info.port=16015 -D hbase.regionserver.port=16025 -D hbase.regionserver.info.port=16035 --backup start

停止backup HMaster
[hbadmin@node-a ~]$/opt/hbase/bin/local-master-backup.sh stop 2 3 5
stopping master.
stopping master.
stopping master.

5.启动additional RegionServers(可选)
RegionServer占用两个端口(默认16200,16300)和local-master-backup.sh类似,local-regionservers.sh也可以通过偏移量来启动多个RegionServers,最多可以达到99个additional RegionServers
[hbadmin@node-a ~]$/opt/hbase/bin/local-regionservers.sh start 2 3 4 5
starting regionserver, logging to /opt/hbase/bin/../logs/hbase-hbadmin-2-regionserver-node-a.example.com.out
starting regionserver, logging to /opt/hbase/bin/../logs/hbase-hbadmin-3-regionserver-node-a.example.com.out
starting regionserver, logging to /opt/hbase/bin/../logs/hbase-hbadmin-4-regionserver-node-a.example.com.out
starting regionserver, logging to /opt/hbase/bin/../logs/hbase-hbadmin-5-regionserver-node-a.example.com.out

停止additional RegionServers
[hbadmin@node-a ~]$/opt/hbase/bin/local-regionservers.sh stop 2 3 4 5
stopping regionserver.........
stopping regionserver.......
stopping regionserver...
stopping regionserver.

6.web UI
每个HMaster都有个web UI,如上就有
192.168.192.10:16010
192.168.192.10:16012
192.168.192.10:16013
192.168.192.10:16015Hbase单节点Standalone <wbr>& <wbr>Pseudo <wbr>Distribute


Hbase单节点Standalone <wbr>& <wbr>Pseudo <wbr>Distribute

posted @ 2016-05-30 13:36  李庆喜  阅读(283)  评论(0编辑  收藏  举报