Zookeeper与HBase的安装
一、Zookeeper的安装
1.http://www-us.apache.org/dist/zookeeper/stable/下载Zookeeper安装包，并将zookeeper-3.4.12.tar.gz复制到/home/jun下并解压
[jun@master ~]$ cd zookeeper-3.4.12/ [jun@master zookeeper-3.4.12]$ ls -l total 1604 drwxr-xr-x. 2 jun jun 149 Mar 27 12:32 bin -rw-rw-r--. 1 jun jun 87945 Mar 27 12:32 build.xml drwxr-xr-x. 2 jun jun 77 Mar 27 12:32 conf drwxr-xr-x. 10 jun jun 130 Mar 27 12:32 contrib drwxr-xr-x. 2 jun jun 4096 Mar 27 12:37 dist-maven drwxr-xr-x. 6 jun jun 4096 Mar 27 12:32 docs -rw-rw-r--. 1 jun jun 1709 Mar 27 12:32 ivysettings.xml -rw-rw-r--. 1 jun jun 8197 Mar 27 12:32 ivy.xml drwxr-xr-x. 4 jun jun 4096 Mar 27 12:32 lib -rw-rw-r--. 1 jun jun 11938 Mar 27 12:32 LICENSE.txt -rw-rw-r--. 1 jun jun 3132 Mar 27 12:32 NOTICE.txt -rw-rw-r--. 1 jun jun 1585 Mar 27 12:32 README.md -rw-rw-r--. 1 jun jun 1770 Mar 27 12:32 README_packaging.txt drwxr-xr-x. 5 jun jun 47 Mar 27 12:32 recipes drwxr-xr-x. 8 jun jun 211 Mar 27 12:32 src -rw-rw-r--. 1 jun jun 1483366 Mar 27 12:32 zookeeper-3.4.12.jar -rw-rw-r--. 1 jun jun 819 Mar 27 12:36 zookeeper-3.4.12.jar.asc -rw-rw-r--. 1 jun jun 33 Mar 27 12:32 zookeeper-3.4.12.jar.md5 -rw-rw-r--. 1 jun jun 41 Mar 27 12:32 zookeeper-3.4.12.jar.sha1
2.使用gedit编辑zoo.cfg配置文件
[jun@master zookeeper-3.4.12]$ cp /home/jun/zookeeper-3.4.12/conf/zoo_sample.cfg /home/jun/zookeeper-3.4.12/conf/zoo.cfg [jun@master zookeeper-3.4.12]$ gedit /home/jun/zookeeper-3.4.12/conf/zoo.cfg
将下面的配置增加到最后面，配置的意思是“服务器编号、服务器地址、LF通信端口（Leader与Follower）、选举端口”
server.1=master:2888:3888 server.2=slave0:2888:3888 server.3=slave1:2888:3888
3.创建节点标志文件
由于在zoo.cfg当中配置了三台服务器，每一台服务器本地也需要一个myid文件，里面包含服务器的ID号，需要在三台机器上都创建myid文件。以master为例：
首先递归创建目录（-p）
[jun@master zookeeper-3.4.12]$ mkdir -p /tmp/zookeeper [jun@master zookeeper-3.4.12]$ gedit /tmp/zookeeper/myid
然后新建myid文件并输入对应的id号，只输入一个1即可。
[jun@master zookeeper-3.4.12]$ cat /tmp/zookeeper/myid 1
另外两台机器同理。
4.将Zookeeper安装文件从master复制到slave上（-r的意思是拷贝所有的文件和文件夹）
[jun@master zookeeper-3.4.12]$ scp -r /home/jun/zookeeper-3.4.12/ slave0:~ [jun@master zookeeper-3.4.12]$ scp -r /home/jun/zookeeper-3.4.12/ slave1:~
5.Zookeeper集群的启动
（1）需要在三个节点上分别执行启动操作
[jun@master zookeeper-3.4.12]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh start ZooKeeper JMX enabled by default Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg Starting zookeeper ... STARTED
[jun@slave0 ~]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[jun@slave1 ~]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
（2）至少启动两台服务器后，集群才会开始选举leader节点，可以在三台服务器上分别查看集群的状态，可以看到slave0节点是leader节点。
[jun@master ~]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg Mode: follower [jun@slave0 ~]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg Mode: leader [jun@slave1 ~]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg Mode: follower
（3）Zookeeper客户端交互式操作
[jun@master ~]$ /home/jun/zookeeper-3.4.12/bin/zkCli.sh -server master:2181 Connecting to master:2181 2018-07-23 14:06:51,047 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.12-e5259e437540f349646870ea94dc2658c4e44b3b, built on 03/27/2018 03:55 GMT 2018-07-23 14:06:51,049 [myid:] - INFO [main:Environment@100] - Client environment:host.name=master 2018-07-23 14:06:51,050 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_171 2018-07-23 14:06:51,051 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2018-07-23 14:06:51,051 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/java/jdk1.8.0_171/jre 2018-07-23 14:06:51,051 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/home/jun/zookeeper-3.4.12/bin/../build/classes:/home/jun/zookeeper-3.4.12/bin/../build/lib/*.jar:/home/jun/zookeeper-3.4.12/bin/../lib/slf4j-log4j12-1.7.25.jar:/home/jun/zookeeper-3.4.12/bin/../lib/slf4j-api-1.7.25.jar:/home/jun/zookeeper-3.4.12/bin/../lib/netty-3.10.6.Final.jar:/home/jun/zookeeper-3.4.12/bin/../lib/log4j-1.2.17.jar:/home/jun/zookeeper-3.4.12/bin/../lib/jline-0.9.94.jar:/home/jun/zookeeper-3.4.12/bin/../lib/audience-annotations-0.5.0.jar:/home/jun/zookeeper-3.4.12/bin/../zookeeper-3.4.12.jar:/home/jun/zookeeper-3.4.12/bin/../src/java/lib/*.jar:/home/jun/zookeeper-3.4.12/bin/../conf: 2018-07-23 14:06:51,051 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2018-07-23 14:06:51,051 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2018-07-23 14:06:51,051 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2018-07-23 14:06:51,052 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2018-07-23 14:06:51,052 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2018-07-23 14:06:51,052 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-862.el7.x86_64 2018-07-23 14:06:51,052 [myid:] - INFO [main:Environment@100] - Client environment:user.name=jun 2018-07-23 14:06:51,052 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/home/jun 2018-07-23 14:06:51,052 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/home/jun 2018-07-23 14:06:51,053 [myid:] - INFO [main:ZooKeeper@441] - Initiating client connection, connectString=master:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@799f7e29 Welcome to ZooKeeper! 2018-07-23 14:06:51,111 [myid:] - INFO [main-SendThread(master:2181):ClientCnxn$SendThread@1028] - Opening socket connection to server master/192.168.1.100:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2018-07-23 14:06:51,191 [myid:] - INFO [main-SendThread(master:2181):ClientCnxn$SendThread@878] - Socket connection established to master/192.168.1.100:2181, initiating session [zk: master:2181(CONNECTING) 0] 2018-07-23 14:06:51,269 [myid:] - INFO [main-SendThread(master:2181):ClientCnxn$SendThread@1302] - Session establishment complete on server master/192.168.1.100:2181, sessionid = 0x1000005d8900000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: master:2181(CONNECTED) 0] help ZooKeeper -server host:port cmd args stat path [watch] set path data [version] ls path [watch] delquota [-n|-b] path ls2 path [watch] setAcl path acl setquota -n|-b val path history redo cmdno printwatches on|off delete path [version] sync path listquota path rmr path get path [watch] create [-s] [-e] path data acl addauth scheme auth quit getAcl path close connect host:port [zk: master:2181(CONNECTED) 1] quit Quitting... 2018-07-23 14:07:36,575 [myid:] - INFO [main:ZooKeeper@687] - Session: 0x1000005d8900000 closed 2018-07-23 14:07:36,577 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@521] - EventThread shut down for session: 0x1000005d8900000
二、HBase的安装
1.从http://apache.fayea.com/hbase/stable/下载hbase-1.2.6.1-bin.tar.gz并赋值到/home/jun下并解压。、
[jun@master ~]$ cd hbase-1.2.6.1/ [jun@master hbase-1.2.6.1]$ ls -l total 340 drwxr-xr-x. 4 jun jun 4096 Jun 4 09:43 bin -rw-r--r--. 1 jun jun 129753 Jun 4 09:43 CHANGES.txt drwxr-xr-x. 2 jun jun 178 Jun 4 09:43 conf drwxr-xr-x. 12 jun jun 4096 Jun 4 12:34 docs drwxr-xr-x. 7 jun jun 80 Jun 4 12:19 hbase-webapps -rw-rw-r--. 1 jun jun 261 Jun 4 12:36 LEGAL drwxrwxr-x. 3 jun jun 8192 Jul 23 14:11 lib -rw-rw-r--. 1 jun jun 143082 Jun 4 12:36 LICENSE.txt -rw-rw-r--. 1 jun jun 42141 Jun 4 12:36 NOTICE.txt -rw-r--r--. 1 jun jun 1477 Jun 4 09:43 README.txt
2.修改conf文件夹下的环境变量hbase-env.sh
# The java implementation to use. Java 1.7+ required. export JAVA_HOME=/usr/java/jdk1.8.0_171/
由于使用的jdk1.8，因此还需要将下面两行注释掉：
# Configure PermSize. Only needed in JDK7. You can safely remove it for JDK8+ #export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m" #export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m"
3.修改conf文件夹下的配置文件hbase-site.xml
<configuration>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://master:9000/hbase</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>master</value>
</property>
<property>
<name>hbase.master.info.port</name>
<value>60010</value>
</property>
</configuration>
4.设置conf文件夹下的regionservers
regionservers文件类似Hadoop的slaves文件，其中保存了rigionserver的列表，启动HBase的时候，系统将根据该文件建立HBase集群。需要先删除localhost并添加两条配置
slave0
slave1
5.设置linux环境变量并source生效
#hbase export HBASE_HOME=/home/jun/hbase-1.2.6.1/ export PATH=$HBASE_HOME/bin:$PATH export HADOOP_CLASSPATH=$HBASE_HOME/lib/*
6.将HBase安装文件复制到slave节点
[jun@master conf]$ scp -r /home/jun/hbase-1.2.6.1/ slave0:~/ [jun@master conf]$ scp -r /home/jun/hbase-1.2.6.1/ slave1:~/
7.启动并验证HBase
（1）执行start-hbase.sh启动HBase，可以看到，系统首先启动HBase自带的ZooKeeper，然后启动HBase的HMaster，接着分别启动slave节点上的regionservers。
[jun@master conf]$ start-hbase.sh master: starting zookeeper, logging to /home/jun/hbase-1.2.6.1/bin/../logs/hbase-jun-zookeeper-master.out starting master, logging to /home/jun/hbase-1.2.6.1//logs/hbase-jun-master-master.out slave0: starting regionserver, logging to /home/jun/hbase-1.2.6.1/bin/../logs/hbase-jun-regionserver-slave0.out slave1: starting regionserver, logging to /home/jun/hbase-1.2.6.1/bin/../logs/hbase-jun-regionserver-slave1.out
（2）查看各结点的jps，其中HMaster是HBase的主控节点进程，HQuorumPeer是HBase的ZooKeeper进程（即HBase内置的ZooKeeper）。
[jun@master ~]$ jps 6273 Jps 3587 SecondaryNameNode 3796 ResourceManager 5989 HMaster 5912 HQuorumPeer 3357 NameNode [jun@slave0 ~]$ jps 3441 NodeManager 4177 HRegionServer 3621 QuorumPeerMain 3271 DataNode 4375 Jps [jun@slave1 ~]$ jps 3232 DataNode 4370 Jps 3606 QuorumPeerMain 4169 HRegionServer 3407 NodeManager
（3）使用Web方式查看HBase系统的运行状态，访问http://master:60010
三、HBase的Shell命令
执行hbase shell即可进入HBase Shell
1.创建表，其中第一个参数是表名，f1和f12是列名
hbase(main):001:0> create "test1", {NAME => 'f1', VERSION => 5} Unknown argument ignored for column family f1: 1.8.7 0 row(s) in 2.6300 seconds => Hbase::Table - test1 hbase(main):002:0> create 'test2', 'f12' 0 row(s) in 2.2360 seconds => Hbase::Table - test2
2.查看列表
hbase(main):003:0> list TABLE test1 test2 2 row(s) in 0.0270 seconds => ["test1", "test2"]
3.插入数据，其中test1是表名，id001是行键（row key），f1是列族名，uid是属性名，001是属性值
hbase(main):004:0> put 'test1', 'id001', 'f1:uid', '001' 0 row(s) in 0.2300 seconds
4.扫描查询数据
hbase(main):005:0> scan 'test1' ROW COLUMN+CELL id001 column=f1:uid, timestamp=1532330789700, value=001 1 row(s) in 0.0750 seconds
5.查看表结构
hbase(main):007:0> describe 'test1' Table test1 is ENABLED test1 COLUMN FAMILIES DESCRIPTION {NAME => 'f1', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => ' FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 1 row(s) in 0.0310 seconds hbase(main):008:0> describe 'test2' Table test2 is ENABLED test2 COLUMN FAMILIES DESCRIPTION {NAME => 'f12', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 1 row(s) in 0.0140 seconds
6.删除表
hbase(main):009:0> list TABLE test1 test2 2 row(s) in 0.0100 seconds => ["test1", "test2"] hbase(main):010:0> disable 'test2' 0 row(s) in 2.2890 seconds hbase(main):011:0> drop 'test2' 0 row(s) in 1.2660 seconds hbase(main):012:0> list TABLE test1 1 row(s) in 0.0110 seconds => ["test1"]
7.退出HBase Shell
hbase(main):013:0> exit
四、HBase和ZooKeeper的关系
如果用户在使用HBase自带的ZooKeeper的时候出现异常，导致HBase不能正常工作，这时可以停止使用自带的Zookeeper，转而启动独立安装的ZooKeeper，并重启HBase，可以快速解决上述问题。
要想让HBase使用独立安装的Zookeeper，需要对HBase进行一些配置上的修改。
1.修改conf/hbase-env.sh文件，添加如下代码（true表示使用自带的ZooKeeper）
export HBASE_MANAGES_ZK=false
2.将Zookeeper配置文件复制到HBase的CLASSPATH
cp /home/jun/zookeeper-3.4.12/conf/zoo.cfg /home/jun/hbase-1.2.6.1/conf
3.重启HBase即可
儿女情长什么的，最影响我们闯荡江湖了。
