hbase1.3.1安装
下载:
hadoop@master:~$ wget https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/1.3.1/hbase-1.3.1-bin.tar.gz
解压到/opt/Hadoop,
hadoop@master:~$ sudo tar -zxvf hbase-1.3.1-bin.tar.gz -C /opt/Hadoop
编辑/etc/profile文件,配置环境变量
sudo vim /etc/profile
增加以下内容:这一步在另外两台机器上也要做
#set hbase
export HBASE_INSTALL=/opt/Hadoop/hbase-1.3.1
export PATH=.:$HBASE_INSTALL/bin
source /etc/profile
配置hbase-site.xml
到/opt/Hadoop/hbase-1.3.1/conf下,修改hbase-site.xml
cd /opt/Hadoop/hbase-1.3.1/conf
sudo vim hbase-site.xml
编辑/opt/Hadoop/hbase-1.3.1/conf下的hadoop-env.sh 文件,找到 JAVA_HOME 改为 JDK 的安装目录
vi hbase-env.sh
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
export HBASE_MANAGES_ZK=false
其实HBase里面自带了一个ZooKeeper,而这个属性的值就是是否使用这个自带的ZooKeeper,很显然我这里要使用自己的ZooKeeper,所以修改为false。
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://master:9000/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>master,slaver1,slaver2</value>
</property>
<property>
<name>hbase.tmp.dir</name>
<value>/opt/Hadoop/hbase-1.3.1/tmp/hbase</value>
</property>
<property>
<name>hbase.master</name>
<value>hdfs://master:60000</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/opt/Hadoop/zookeeper-3.4.10/tmp/zookeeper</value>
</property>
</configuration>
建立hbase数据文件夹
因为在上一步中设置了hbase.tmp.dir的目录,这里需要手动建立目录:在上面提到的地址/opt/Hadoop/hbase-1.3.1中建立一个tmp文件夹,在tmp下建一个hbase文件夹。
cd /opt/Hadoop/hbase-1.3.1
sudo mkdir tmp
cd tmp
sudo mkdir hbase
使用scp将文件(这里指hbase的安装及配置文件)/opt/Hadoop/hbase-1.3.1复制到另外两个节点slaver1和slaver2命令如下:
注意文件的所有者权限
在master上执行以下两条命令:
scp -r /opt/Hadoop/hbase-1.3.1 hadoop@slaver1:~
scp -r /opt/Hadoop/hbase-1.3.1 hadoop@slaver2:~
在slaver1上:hadoop@slaver1:~$ sudo mv ./hbase-1.3.1/ /opt/Hadoop/
执行完上述命令之后,你会发现hadoop_s1和hadoop_s2的/opt/Hadoop下面了hbase-1.3.1文件夹。
到每个从节点的/opt/Hadoop目录下修改所有都权限:
cd /op/Hadoop
sudo chown -R hadoop:hadoop hbase-1.3.1/
在主节点上执行下面的操作开启服务,在根目录下(需要保证hdfs和zookeeper服务已开启):
start-hbase.sh
在master上查看:
hadoop@master:/opt/Hadoop/hbase-1.3.1/bin$ ./start-hbase.sh
starting master, logging to /opt/Hadoop/hbase-1.3.1/bin/../logs/hbase-hadoop-master-master.out
OpenJDK 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
slaver1: starting regionserver, logging to /opt/Hadoop/hbase-1.3.1/bin/../logs/hbase-hadoop-regionserver-slaver1.out
slaver2: starting regionserver, logging to /opt/Hadoop/hbase-1.3.1/bin/../logs/hbase-hadoop-regionserver-slaver2.out
master: starting regionserver, logging to /opt/Hadoop/hbase-1.3.1/bin/../logs/hbase-hadoop-regionserver-master.out
slaver1: OpenJDK 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
slaver1: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
slaver2: OpenJDK 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
slaver2: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
master: OpenJDK 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
master: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
hadoop@master:/opt/Hadoop/hbase-1.3.1/bin$ jps
29152 NameNode
33266 Jps
29638 ResourceManager
29416 SecondaryNameNode
32761 HMaster
32924 HRegionServer
在slaver1上查看:
hadoop@slaver1:/opt/Hadoop/zookeeper-3.4.10/bin$ sudo jps
[sudo] password for hadoop:
22531 Jps
20934 DataNode
22137 HRegionServer
10123 QuorumPeerMain
21085 NodeManager
在slaver2上查看:
hadoop@slaver2:/opt/Hadoop/zookeeper-3.4.10/bin$ sudo jps
[sudo] password for hadoop:
9875 QuorumPeerMain
20452 NodeManager
21960 Jps
21481 HRegionServer
20301 DataNode
通过可通过访问http://192.168.93.140:16010/master-status在web端访问可以看到一个界面
若想要停止此进程,在根目录下,执行stop-hbase.sh命令即可。
在/opt/Hadoop//opt/Hadoop/hbase-1.3.1/bin 下可以用hbase shell
hadoop@master:/opt/Hadoop/hbase-1.3.1/bin$ ./hbase shell
2017-09-21 22:57:35,381 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.3.1, r930b9a55528fe45d8edce7af42fef2d35e77677a, Thu Apr 6 19:36:54 PDT 2017
查询服务器状态
hbase(main):001:0> status
1 active master, 0 backup masters, 3 servers, 0 dead, 0.6667 average load
查询hive版本
hbase(main):002:0> version
1.3.1, r930b9a55528fe45d8edce7af42fef2d35e77677a, Thu Apr 6 19:36:54 PDT 2017
创建一个表
hbase(main):003:0> create 'member','member_id','address','info'
0 row(s) in 1.6030 seconds
=> Hbase::Table - member
列出所有的表
hbase(main):004:0> list
TABLE
member
1 row(s) in 0.0420 seconds
=> ["member"]
表的描述
hbase(main):005:0> describe 'member'
Table member is ENABLED
member
COLUMN FAMILIES DESCRIPTION
{NAME => 'address', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false',
KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER',
COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE =>
'65536', REPLICATION_SCOPE => '0'}
{NAME => 'info', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KE
EP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', CO
MPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65
536', REPLICATION_SCOPE => '0'}
{NAME => 'member_id', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false
', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER
', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE =
> '65536', REPLICATION_SCOPE => '0'}
3 row(s) in 0.1900 seconds
查询表是否存在
hbase(main):006:0> exists 'member'
Table member does exist
插入几条记录
put'member','scutshuxue','info:age','24'
put'member','scutshuxue','info:birthday','1987-06-17'
put'member','scutshuxue','info:company','alibaba'
put'member','scutshuxue','address:contry','china'
put'member','scutshuxue','address:province','zhejiang'
put'member','scutshuxue','address:city','hangzhou'
put'member','xiaofeng','info:birthday','1987-4-17'
put'member','xiaofeng','info:favorite','movie'
put'member','xiaofeng','info:company','alibaba'
put'member','xiaofeng','address:contry','china'
put'member','xiaofeng','address:province','guangdong'
put'member','xiaofeng','address:city','jieyang'
put'member','xiaofeng','address:town','xianqiao'
获取一个id的所有数据
hbase(main):014:0> get 'member','scutshuxue'
COLUMN CELL
address:city timestamp=1506060761570, value=hangzhou
address:contry timestamp=1506060733774, value=china
address:province timestamp=1506060750024, value=zhejiang
info:age timestamp=1506060673856, value=24
info:birthday timestamp=1506060706524, value=1987-06-17
info:company timestamp=1506060721466, value=alibaba
获取一个id,一个列族的所有数据
hbase(main):015:0> get 'member','scutshuxue','info'
COLUMN CELL
info:age timestamp=1506060673856, value=24
info:birthday timestamp=1506060706524, value=1987-06-17
info:company timestamp=1506060721466, value=alibaba
1 row(s) in 0.0260 seconds
获取一个id,一个列族中一个列的所有数据
hbase(main):016:0> get 'member','scutshuxue','info:age'
COLUMN CELL
info:age timestamp=1506060673856, value=24
1 row(s) in 0.0190 seconds
更新一条记录
将scutshuxue的年龄改成99
hbase(main):017:0> put 'member','scutshuxue','info:age' ,'99'
0 row(s) in 0.0220 seconds
hbase(main):018:0> get 'member','scutshuxue','info:age'
COLUMN CELL
info:age timestamp=1506061228012, value=99
1 row(s) in 0.0160 seconds
通过timestamp来获取两个版本的数据
hbase(main):020:0> get 'member','scutshuxue',{COLUMN=>'info:age',TIMESTAMP=>1506061228012}
COLUMN CELL
info:age timestamp=1506061228012, value=99
1 row(s) in 0.0230 seconds
hbase(main):022:0> get 'member','scutshuxue',{COLUMN=>'info:age',TIMESTAMP=>1506060673856}
COLUMN CELL
info:age timestamp=1506060673856, value=24
1 row(s) in 0.0170 seconds
全表扫描:
hbase(main):023:0> scan 'member'
ROW COLUMN+CELL
scutshuxue column=address:city, timestamp=1506060761570, value=hangzh
ou
scutshuxue column=address:contry, timestamp=1506060733774, value=chin
a
scutshuxue column=address:province, timestamp=1506060750024, value=zh
ejiang
scutshuxue column=info:age, timestamp=1506061228012, value=99
scutshuxue column=info:birthday, timestamp=1506060706524, value=1987-
06-17
scutshuxue column=info:company, timestamp=1506060721466, value=alibab
a
xiaofeng column=address:city, timestamp=1506060783577, value=jieyan
g
xiaofeng column=address:contry, timestamp=1506060783512, value=chin
a
xiaofeng column=address:province, timestamp=1506060783544, value=gu
angdong
xiaofeng column=address:town, timestamp=1506060786301, value=xianqi
ao
xiaofeng column=info:birthday, timestamp=1506060783397, value=1987-
4-17
xiaofeng column=info:company, timestamp=1506060783477, value=alibab
a
xiaofeng column=info:favorite, timestamp=1506060783444, value=movie
2 row(s) in 0.1030 seconds
插入一行数据:
put 'member', 'temp', 'info:age', '77'
删除id为temp的值的‘info:age’字段
hbase(main):026:0> delete 'member','temp','info:age'
0 row(s) in 0.0590 seconds
hbase(main):027:0> get 'member' ,'temp'
COLUMN CELL
0 row(s) in 0.0070 seconds
删除整行
hbase(main):030:0> deleteall 'member','xiaofeng'
0 row(s) in 0.0290 seconds
查询表中有多少行:
hbase(main):031:0> count 'member'
1 row(s) in 0.0400 seconds
=> 1
插入数据:
put'member','xiaofeng','info:birthday','1987-4-17'
put'member','xiaofeng','info:favorite','movie'
put'member','xiaofeng','info:company','alibaba'
put'member','xiaofeng','address:contry','china'
put'member','xiaofeng','address:province','guangdong'
put'member','xiaofeng','address:city','jieyang'
put'member','xiaofeng','address:town','xianqiao'
给‘xiaofeng’这个id增加'info:age'字段,并使用counter实现递增
hbase(main):039:0> incr 'member','xiaofeng','info:age'
COUNTER VALUE = 1
0 row(s) in 0.0330 seconds
hbase(main):040:0> get 'member' ,'xiaofeng'
COLUMN CELL
address:city timestamp=1506063247883, value=jieyang
address:contry timestamp=1506063247830, value=china
address:province timestamp=1506063247855, value=guangdong
address:town timestamp=1506063249582, value=xianqiao
info:age timestamp=1506063280597, value=\x00\x00\x00\x00\x00\x00\x0
0\x01
info:birthday timestamp=1506063247738, value=1987-4-17
info:company timestamp=1506063247802, value=alibaba
info:favorite timestamp=1506063247776, value=movie
1 row(s) in 0.0200 seconds
hbase(main):041:0> incr 'member','xiaofeng','info:age'
COUNTER VALUE = 2
0 row(s) in 0.0350 seconds
hbase(main):042:0> get 'member','xiaofeng','info:age'
COLUMN CELL
info:age timestamp=1506063350162, value=\x00\x00\x00\x00\x00\x00\x0
0\x02
1 row(s) in 0.0110 seconds
获取当前count的值
hbase(main):043:0> get_counter 'member','xiaofeng','info:age'
COUNTER VALUE = 2
将整张表清空:
hbase(main):044:0> truncate 'member'
Truncating 'member' table (it may take a while):
- Disabling table...
- Truncating table...
0 row(s) in 3.8830 seconds
hbase(main):045:0> scan 'member'
ROW COLUMN+CELL
0 row(s) in 0.2060 seconds
hbase(main):049:0> exists 'member'
Table member does exist
0 row(s) in 0.0170 seconds
判断表是否enable
hbase(main):050:0> is_enabled 'member'
true
0 row(s) in 0.0180 seconds
判断表是否disable
hbase(main):051:0> is_disabled 'member'
false
0 row(s) in 0.0180 seconds
创建一个表
hbase(main):053:0> create 'temp_table', 'id', 'info'
0 row(s) in 2.2540 seconds
=> Hbase::Table - temp_table
drop一个表:先disable后drop
hbase(main):056:0> disable 'temp_table'
0 row(s) in 2.2670 seconds
hbase(main):057:0> drop 'temp_table'
0 row(s) in 1.2730 seconds
hbase(main):059:0> exists 'temp_table'
Table temp_table does not exist
0 row(s) in 0.0090 seconds
.删除一个列族,alter,disable,enable
我们之前建了3个列族,但是发现member_id这个列族是多余的,因为他就是主键,所以我们要将其删除。
hbase(main):060:0> alter 'member',{NAME=>'member_id',METHOD=>'delete'}
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 2.2260 seconds
hbase(main):061:0> scan 'member'
ROW COLUMN+CELL
0 row(s) in 0.1910 seconds
hbase(main):062:0> count 'member'
0 row(s) in 0.0170 seconds
=> 0