HBase安装与应用

 

1.  参考说明

参考文档:

 

https://hbase.apache.org/book.html

 

2.  安装环境说明

2.1.  环境说明

 

CentOS7.4+ Hadoop2.7.5的伪分布式环境

 

主机名

NameNode

SecondaryNameNode

DataNodes

centoshadoop.smartmap.com

192.168.1.80

192.168.1.80

192.168.1.80

 

 

 

 

 

Hadoop的安装目录为:/opt/hadoop/hadoop-2.7.5

 

3.  安装

3.1.  Zookeeper

 

3.1.1.  Zookeeper下载

http://zookeeper.apache.org/releases.html#download

 

3.1.2.  Zookeeper解压

将下载的zookeeper-3.4.12.tar.gz解压到/opt/hadoop/zookeeper-3.4.12目录下

 

3.1.3.  Zookeeper配置

3.1.3.1. 创建目录

 

[root@centoshadoop conf]# mkdir -p /opt/hadoop/zookeeper-3.4.12/data/db

[root@centoshadoop conf]# mkdir -p /opt/hadoop/zookeeper-3.4.12/data/log

 

3.1.3.2. 修改zoo.cfg

[root@centoshadoop zookeeper-3.4.12]# cd /opt/hadoop/zookeeper-3.4.12/conf/

[root@centoshadoop conf]# cp zoo_sample.cfg zoo.cfg

[root@centoshadoop conf]# vi zoo.cfg

 

tickTime=2000

initLimit=10

syncLimit=5

clientPort=2181

dataDir=/opt/hadoop/zookeeper-3.4.12/data/db

dataLogDir=/opt/hadoop/zookeeper-3.4.12/data/log

#自动清除日志文件

autopurge.snapRetainCount=20

autopurge.purgeInterval=48

 

# Zookeeper Cluster

# server.1=192.168.1.81:2888:3888

# server.2=192.168.1.82:2888:3888

# server.3=192.168.1.83:2888:3888

 

3.2.  HBase

3.2.1.  HBase下载

https://hbase.apache.org/downloads.html

 

3.2.2.  HBase解压

将下载的apache-hive-2.3.3-bin.tar.gz解压到/opt/hadoop/hbase-2.0.0目录下

 

4.  配置

4.1.  修改profile文件

vi /etc/profile

 

# export JAVA_HOME=/opt/java/jdk1.7.0_45

export JAVA_HOME=/opt/java/jdk1.8.0_171

# export CLASSPATH=.:$JAVA_HOME/lib

export HADOOP_HOME=/opt/hadoop/hadoop-2.7.5

export HADOOP_INSTALL=$HADOOP_HOME

export HADOOP_MAPRED_HOME=$HADOOP_HOME

export HADOOP_COMMON_HOME=$HADOOP_HOME

export HADOOP_HDFS_HOME=$HADOOP_HOME

export YARN_HOME=$HADOOP_HOME

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

 

export PIG_HOME=/opt/hadoop/pig-0.17.0

export HIVE_HOME=/opt/hadoop/hive-2.3.3

 

export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

export PATH=$PATH:$PIG_HOME/bin

export PATH=$PATH:$HIVE_HOME/bin

 

export SQOOP_HOME=/opt/hadoop/sqoop-1.99.7

export PATH=$PATH:$SQOOP_HOME/bin

export CLASSPATH=$CLASSPATH:$SQOOP_HOME/server/lib:$SQOOP_HOME/shell/lib:$SQOOP_HOME/tools/lib

export SQOOP_SERVER_EXTRA_LIB=/opt/hadoop/sqoop-1.99.7/jdbcDriver/

 

export HBASE_HOME=/opt/hadoop/hbase-2.0.0

export PATH=$PATH:$HBASE_HOME/bin

 

export CLASSPATH=.:$JAVA_HOME/lib:$HIVE_HOME/lib

export LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/server:/usr/local/lib:$HADOOP_HOME/lib/native

export JAVA_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH

export HADOOP_USER_NAME=hadoop

 

4.2.  JDK升级为1.8版本

 

JDK切换成1.8的版本,并修改所有与JAVA_HOME相关的变量

 

4.3.  修改HBase的配置文件

4.3.1.  创建目录

 

[root@centoshadoop hbase-2.0.0]# mkdir -p /opt/hadoop/hbase-2.0.0/data/tmp

[root@centoshadoop hbase-2.0.0]# mkdir -p /opt/hadoop/hbase-2.0.0/data/tmp/zookeeper

 

4.3.2.  修改hbase-env.sh

[root@centoshadoop conf]# vi /opt/hadoop/hbase-2.0.0/conf/hbase-env.sh

 

export JAVA_HOME=/opt/java/jdk1.8.0_171

export HBASE_MANAGES_ZK=false

 

4.3.3.  修改hbase-site.xml

[root@centoshadoop conf]# vi /opt/hadoop/hbase-2.0.0/conf/hbase-site.xml

 

       <property>   

        <name>hbase.rootdir</name>    

        <value>hdfs://192.168.1.80:9000/hbase</value>   

    </property>   

    <property>   

        <name>hbase.cluster.distributed</name>   

        <value>true</value>   

    </property>   

    <property>   

        <name>hbase.zookeeper.quorum</name>   

        <value>192.168.1.80</value>   

    </property>   

    <property>   

        <name>hbase.temp.dir</name>   

        <value>/opt/hadoop/hbase-2.0.0/data/tmp</value>   

    </property>   

    <property>   

        <name>hbase.zookeeper.property.dataDir</name>   

        <value>/opt/hadoop/hbase-2.0.0/data/tmp/zookeeper</value>   

    </property>

         <property>

           <name>hbase.zookeeper.property.clientPort</name>

           <value>2181</value>

         </property>       

    <property>   

        <name>hbase.master.info.port</name>   

        <value>60010</value>   

    </property>

 

4.3.4.  修改regionservers

 

[root@centoshadoop conf]# vi /opt/hadoop/hbase-2.0.0/conf/regionservers

 

192.168.1.80

 

5.  启动Hadoop

 

5.1.  启动YARNHDFS

cd /opt/hadoop/hadoop-2.7.5/sbin

 

start-all.sh

 

5.2.  启动historyserver

 

cd /opt/hadoop/hadoop-2.7.5/sbin

 

mr-jobhistory-daemon.sh start historyserver

 

6.  启动Zookeeper

 

[root@centoshadoop bin]# cd /opt/hadoop/zookeeper-3.4.12/bin/

[root@centoshadoop bin]# ./zkServer.sh start

ZooKeeper JMX enabled by default

Using config: /opt/hadoop/zookeeper-3.4.12/bin/../conf/zoo.cfg

Starting zookeeper ... STARTED

[root@centoshadoop bin]#

 

7.  启动Hbase

 

[root@centoshadoop bin]# cd /opt/hadoop/hbase-2.0.0/bin/

[root@centoshadoop bin]# ./start-hbase.sh

 

[root@centoshadoop bin]# jps

41984 DataNode

42192 SecondaryNameNode

45920 QuorumPeerMain

46643 Jps

42484 NodeManager

46294 HRegionServer

41850 NameNode

46170 HMaster

29245 JobHistoryServer

42365 ResourceManager

[root@centoshadoop bin]#

 

 

8.  应用HBase工具

 

8.1.  测试

8.1.1.  用浏览器访问Hbase状态信息

http://192.168.1.80:16030/

 

clip_image002

 

8.2.  HBase的交互式Shell

 

名称

命令表达式

创建表

create '表名称', '列名称1','列名称2','列名称N'

添加记录     

put '表名称', '行名称', '列名称:', ''

查看记录

get '表名称', '行名称'

查看表中的记录总数

count  '表名称'

删除记录

delete  '表名' ,'行名称' , '列名称'

删除一张表

先要屏蔽该表,才能对该表进行删除,

第一步disable '表名称'

第二步  drop '表名称'

查看所有记录

scan "表名称

查看某个表某个列中所有数据

scan "表名称" , ['列名称:']

更新记录 

就是重写一遍进行覆盖

 

 

8.2.1.  启动运行HBase的交互式Shell环境

 

[root@centoshadoop bin]# cd /opt/hadoop/hbase-2.0.0/bin/

[root@centoshadoop bin]# ./hbase shell

 

8.2.2.  创建表

 

hbase(main):004:0> create 'test', 'data'

Created table test

Took 1.4466 seconds

=> Hbase::Table - test

hbase(main):005:0>

 

test: 表名

data: 列族column family

 

8.2.3.  显示表

 

hbase(main):005:0> list

TABLE

test

1 row(s)

Took 0.0250 seconds

=> ["test"]

 

 

8.2.4.  加入数据

 

hbase(main):006:0> put 'test', 'row1', 'data:1', 'value1'

Took 0.1997 seconds

hbase(main):007:0> put 'test', 'row2', 'data:2', 'value2'

Took 0.0140 seconds

hbase(main):008:0> put 'test', 'row3', 'data:3', 'value3'

Took 0.0061 seconds

hbase(main):009:0>

 

test:表名

row1:行键(row key

data:1:列(column

 

8.2.5.  查询数据

 

hbase(main):009:0> scan 'test'

ROW                           COLUMN+CELL

 row1                         column=data:1, timestamp=1527136702544, value=value1

 row2                         column=data:2, timestamp=1527136720577, value=value2

 row3                         column=data:3, timestamp=1527136732695, value=value3

3 row(s)

Took 0.0360 seconds

 

8.2.6.  删除表

 

 

posted @ 2018-05-24 17:51  ParamousGIS  阅读(624)  评论(0编辑  收藏  举报