Zookeeper环境搭建

　　zookeeper支持windows、linux、mac等操作系统，其搭建方式也有集群、伪集群、单机环境。下面研究三种方式的搭建。

　　单机环境:windows操作系统

　　伪集群:windows

　　集群:linux

1.单机环境

　　下面在windows下面搭建zookeeper的单机环境。windows下面也适合做开发。但是不适合生产环境的部署安装。

1.Java安装

　　这个就不说了，zookeeper依赖于Java环境，所以先自行安装JDK。

2.zookeeper下载以及安装

(1)下载zookeeperxxx.tar.gz

　　下载地址:http://zookeeper.apache.org/releases.html 下载xxx.tar.gz解压即可，里面有sh脚本，也有cmd脚本。

(2)解压上面下载的文件

(3)在conf目录下面创建zoo.cfg (在conf目录下面提供了一分样本文件zoo_sample.cfg),或者复制一分样本文件改名为zoo.cfg

tickTime = 2000
dataDir = E:/zookeeper/zookeeper-3.4.13/data
clientPort = 2181
initLimit = 5
syncLimit = 2

解释:

　　tickTime：这个时间是作为 Zookeeper 服务器之间或客户端与服务器之间维持心跳的时间间隔，也就是每个 tickTime 时间就会发送一个心跳。
　　dataDir：顾名思义就是 Zookeeper 保存数据的目录，默认情况下，Zookeeper 将写数据的日志文件也保存在这个目录里。
　　dataLogDir：log目录, 同样可以是任意目录. 如果没有设置该参数, 将使用和dataDir相同的设置.
　　clientPort：这个端口就是客户端连接 Zookeeper 服务器的端口，Zookeeper 会监听这个端口，接受客户端的访问请求。

　　我在自己设置的时候同事配置dataDir和dataLogDir反而启动报错，所以我只设置了dataDir

(4)在对应目录建立上面的data目录即可

(5)启动zookeeper

E:\zookeeper\zookeeper-3.4.13\bin> .\zkServer.cmd

E:\zookeeper\zookeeper-3.4.13\bin>call "C:\Program Files\Java\jdk1.8.0_121"\bin\java "-Dzookeeper.log.dir=E:\zookeeper\zookeeper-3.4.13\bin\.." "-Dzookeeper.root.logger=INFO,CONSOLE" -cp "E:\zookeeper\zookeeper-3.4.13\bin\..\build\classes;E:\zookeeper\zookeeper-3.4.13\bin\..\build\lib\*;E:\zookeeper\zookeeper-3.4.13\bin\..\*;E:\zookeeper\zookeeper-3.4.13\bin\..\lib\*;E:\zookeeper\zookeeper-3.4.13\bin\..\conf" org.apache.zookeeper.server.quorum.QuorumPeerMain "E:\zookeeper\zookeeper-3.4.13\bin\..\conf\zoo.cfg"
2019-03-07 18:14:37,163 [myid:] - INFO  [main:QuorumPeerConfig@136] - Reading configuration from: E:\zookeeper\zookeeper-3.4.13\bin\..\conf\zoo.cfg
2019-03-07 18:14:37,170 [myid:] - INFO  [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2019-03-07 18:14:37,171 [myid:] - INFO  [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2019-03-07 18:14:37,171 [myid:] - INFO  [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2019-03-07 18:14:37,173 [myid:] - WARN  [main:QuorumPeerMain@116] - Either no config or no quorum defined in config, running  in standalone mode
2019-03-07 18:14:37,229 [myid:] - INFO  [main:QuorumPeerConfig@136] - Reading configuration from: E:\zookeeper\zookeeper-3.4.13\bin\..\conf\zoo.cfg
2019-03-07 18:14:37,230 [myid:] - INFO  [main:ZooKeeperServerMain@98] - Starting server
2019-03-07 18:14:37,271 [myid:] - INFO  [main:Environment@100] - Server environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 04:05 GMT
2019-03-07 18:14:37,271 [myid:] - INFO  [main:Environment@100] - Server environment:host.name=MicroWin10-1535
2019-03-07 18:14:37,272 [myid:] - INFO  [main:Environment@100] - Server environment:java.version=1.8.0_121
2019-03-07 18:14:37,272 [myid:] - INFO  [main:Environment@100] - Server environment:java.vendor=Oracle Corporation
2019-03-07 18:14:37,272 [myid:] - INFO  [main:Environment@100] - Server environment:java.home=C:\Program Files\Java\jdk1.8.0_121\jre
2019-03-07 18:14:37,273 [myid:] - INFO  [main:Environment@100] - Server environment:java.class.path=E:\zookeeper\zookeeper-3.4.13\bin\..\build\classes;E:\zookeeper\zookeeper-3.4.13\bin\..\build\lib\*;E:\zookeeper\zookeeper-3.4.13\bin\..\zookeeper-3.4.13.jar;E:\zookeeper\zookeeper-3.4.13\bin\..\lib\audience-annotations-0.5.0.jar;E:\zookeeper\zookeeper-3.4.13\bin\..\lib\jline-0.9.94.jar;E:\zookeeper\zookeeper-3.4.13\bin\..\lib\log4j-1.2.17.jar;E:\zookeeper\zookeeper-3.4.13\bin\..\lib\netty-3.10.6.Final.jar;E:\zookeeper\zookeeper-3.4.13\bin\..\lib\slf4j-api-1.7.25.jar;E:\zookeeper\zookeeper-3.4.13\bin\..\lib\slf4j-log4j12-1.7.25.jar;E:\zookeeper\zookeeper-3.4.13\bin\..\conf
2019-03-07 18:14:37,274 [myid:] - INFO  [main:Environment@100] - Server environment:java.library.path=C:\Program Files\Java\jdk1.8.0_121\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;E:\Anaconda3\exe;E:\Anaconda3\exe\Library\mingw-w64\bin;E:\Anaconda3\exe\Library\usr\bin;E:\Anaconda3\exe\Library\bin;E:\Anaconda3\exe\Scripts;E:\ImageMagick-7.0.8-Q16;C:\oraclexe\app\oracle\product\11.2.0\server\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;E:\soft\apache-maven-3.5.3\bin;C:\WINDOWS\System32\OpenSSH\;E:\git\Git\cmd;E:\SVN\bin;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files\Microsoft SQL Server\110\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\ManagementStudio\;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\;C:\Users\Administrator\AppData\Local\Microsoft\WindowsApps;E:\soft\maven\apache-maven-3.3.9\bin;C:\Program Files\MySQL\MySQL Server 5.7\bin;C:\Program Files\Java\jdk1.8.0_121\bin;E:\git\Git\bin;E:\git\Git\usr\bin;E:\git\Git;C:\Program Files\Java\jdk1.8.0_121\jre\bin;D:\zdcontomcat\zdc8\lo\program;E:\tesseract4\Tesseract-OCR;E:\redis\redis-win;;.
2019-03-07 18:14:37,276 [myid:] - INFO  [main:Environment@100] - Server environment:java.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\
2019-03-07 18:14:37,276 [myid:] - INFO  [main:Environment@100] - Server environment:java.compiler=<NA>
2019-03-07 18:14:37,277 [myid:] - INFO  [main:Environment@100] - Server environment:os.name=Windows 10
2019-03-07 18:14:37,278 [myid:] - INFO  [main:Environment@100] - Server environment:os.arch=amd64
2019-03-07 18:14:37,279 [myid:] - INFO  [main:Environment@100] - Server environment:os.version=10.0
2019-03-07 18:14:37,285 [myid:] - INFO  [main:Environment@100] - Server environment:user.name=Administrator
2019-03-07 18:14:37,287 [myid:] - INFO  [main:Environment@100] - Server environment:user.home=C:\Users\Administrator
2019-03-07 18:14:37,289 [myid:] - INFO  [main:Environment@100] - Server environment:user.dir=E:\zookeeper\zookeeper-3.4.13\bin
2019-03-07 18:14:37,296 [myid:] - INFO  [main:ZooKeeperServer@836] - tickTime set to 2000
2019-03-07 18:14:37,297 [myid:] - INFO  [main:ZooKeeperServer@845] - minSessionTimeout set to -1
2019-03-07 18:14:37,302 [myid:] - INFO  [main:ZooKeeperServer@854] - maxSessionTimeout set to -1
2019-03-07 18:14:37,585 [myid:] - INFO  [main:ServerCnxnFactory@117] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
2019-03-07 18:14:37,588 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181

3.测试

(1)用netstat或者jps查看都可以

C:\Users\Administrator>netstat -ano|findstr 2181
  TCP    0.0.0.0:2181           0.0.0.0:0              LISTENING       15532
  TCP    [::]:2181              [::]:0                 LISTENING       15532

C:\Users\Administrator>jps -l
19320 sun.tools.jps.Jps
15532 org.apache.zookeeper.server.quorum.QuorumPeerMain
4780

(2)zkCli.cmd连接进行测试:

PS E:\zookeeper\zookeeper-3.4.13\bin> .\zkCli.cmd
Connecting to localhost:2181
.........
[zk: localhost:2181(CONNECTING) 0] 2019-03-07 18:23:07,894 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1303] - Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x10035685eb40002, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null

[zk: localhost:2181(CONNECTED) 0]

[zk: localhost:2181(CONNECTED) 0] help
ZooKeeper -server host:port cmd args
        stat path [watch]
        set path data [version]
        ls path [watch]
        delquota [-n|-b] path
        ls2 path [watch]
        setAcl path acl
        setquota -n|-b val path
        history
        redo cmdno
        printwatches on|off
        delete path [version]
        sync path
        listquota path
        rmr path
        get path [watch]
        create [-s] [-e] path data acl
        addauth scheme auth
        quit
        getAcl path
        close
        connect host:port
[zk: localhost:2181(CONNECTED) 1] quit
Quitting...
2019-03-07 18:24:38,781 [myid:] - INFO  [main:ZooKeeper@693] - Session: 0x10035685eb40002 closed
2019-03-07 18:24:38,783 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x10035685eb40002
PS E:\zookeeper\zookeeper-3.4.13\bin>

2.伪集群安装-windows安装

　　伪集群，就是一台机器开启多个端口然后进行集群配置，最常用的学习方式。。。这个集群有点类似于ActiveMQ的伪集群。

1.将E:\zookeeper\zookeeper-3.4.13\conf\zoo.cfg复制出三分，分别命名为zoo1.cfg，zoo2.cfg，zoo3.cfg

对应内容:

zoo1.cfg

tickTime = 2000
dataDir = E:/zookeeper/zookeeper-3.4.13/data1
clientPort = 2181
initLimit = 5
syncLimit = 2

server.1=localhost:2887:3887
server.2=localhost:2888:3888
server.3=localhost:2889:3889

zoo2.cfg

tickTime = 2000
dataDir = E:/zookeeper/zookeeper-3.4.13/data2
clientPort = 2182
initLimit = 5
syncLimit = 2

server.1=localhost:2887:3887
server.2=localhost:2888:3888
server.3=localhost:2889:3889

zoo3.cfg

tickTime = 2000
dataDir = E:/zookeeper/zookeeper-3.4.13/data3
clientPort = 2183
initLimit = 5
syncLimit = 2

server.1=localhost:2887:3887
server.2=localhost:2888:3888
server.3=localhost:2889:3889

解释： server.{num}=ip/domain:Port1:Port2

　　num：表示数字表示第几号服务器；

　　ip/domain ：是服务器域名或者ip地址。

　　 Port1：表示这个服务器和集群中的Leader服务器交换信息的端口；

　　 Port2：表示万一集群中的Leader服务器挂了，需要一个端口重新进行选举，选出一个新的Leader，这个端口就是用来执行选举时服务器相互通信的端口。

由于我们是伪集群，所以ip或者域名是一样的，所以要分配不同的端口号

2.创建对应的目录

3.进入E:\zookeeper\zookeeper-3.4.13\bin下复制文件zkServer.cmd为zkServer-1.cmd,zkServer-2.cmd,zkServer-3.cmd

zkServer-1.cmd内容修改为如下：

@echo off

setlocal
call "%~dp0zkEnv.cmd"

set ZOOCFG=E:/zookeeper/zookeeper-3.4.13/conf/zoo1.cfg
set ZOOMAIN=org.apache.zookeeper.server.quorum.QuorumPeerMain
echo on
call %JAVA% "-Dzookeeper.log.dir=%ZOO_LOG_DIR%" "-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%" -cp "%CLASSPATH%" %ZOOMAIN% "%ZOOCFG%" %*

endlocal

zkServer-2.cmd内容修改为如下：

@echo off

setlocal
call "%~dp0zkEnv.cmd"

set ZOOCFG=E:/zookeeper/zookeeper-3.4.13/conf/zoo2.cfg
set ZOOMAIN=org.apache.zookeeper.server.quorum.QuorumPeerMain
echo on
call %JAVA% "-Dzookeeper.log.dir=%ZOO_LOG_DIR%" "-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%" -cp "%CLASSPATH%" %ZOOMAIN% "%ZOOCFG%" %*

endlocal

zkServer-3.cmd内容修改为如下：

@echo off

setlocal
call "%~dp0zkEnv.cmd"

set ZOOCFG=E:/zookeeper/zookeeper-3.4.13/conf/zoo3.cfg
set ZOOMAIN=org.apache.zookeeper.server.quorum.QuorumPeerMain
echo on
call %JAVA% "-Dzookeeper.log.dir=%ZOO_LOG_DIR%" "-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%" -cp "%CLASSPATH%" %ZOOMAIN% "%ZOOCFG%" %*

endlocal

4.添加myid文件

　　除了修改 zoo.cfg 配置文件，集群模式下还要配置一个标识自己身份也就是自己的ID值文件 myid，这个文件在zoo.cfg里dataDir指定的目录下，这个文件里面就只有一个数字，这个数字和server.n的n保持一致，该值范围可以是1-255之间，Zookeeper 启动时会读取这个文件，拿到里面的数据与 zoo.cfg 里面的配置信息比较从而判断到底是那个 server。

$ pwd
/e/zookeeper/zookeeper-3.4.13/data1

Administrator@MicroWin10-1535 MINGW64 /e/zookeeper/zookeeper-3.4.13/data1
$ echo "1">myid

Administrator@MicroWin10-1535 MINGW64 /e/zookeeper/zookee                            per-3.4.13/data2
$ pwd
/e/zookeeper/zookeeper-3.4.13/data2

Administrator@MicroWin10-1535 MINGW64 /e/zookeeper/zookeeper-3.4.13/data2
$ echo "2">myid

Administrator@MicroWin10-1535 MINGW64 /e/zookeeper/zookeeper-3.4.13/data3
$ echo "3">myid

5.启动即可

PS E:\zookeeper\zookeeper-3.4.13\bin> .\zkServer3.cmd

E:\zookeeper\zookeeper-3.4.13\bin>call "C:\Program Files\Java\jdk1.8.0_121"\bin\java "-Dzookeeper.log.dir=E:\zookeeper\zookeeper-3.4.13\bin\.." "-Dzookeeper.root.logger=INFO,CONSOLE" -cp "E:\zookeeper\zookeeper-3.4.13\bin\..\build\classes;E:\zookeeper\zookeeper-3.4.13\bin\..\build\lib\*;E:\zookeeper\zookeeper-3.4.13\bin\..\*;E:\zookeeper\zookeeper-3.4.13\bin\..\lib\*;E:\zookeeper\zookeeper-3.4.13\bin\..\conf" org.apache.zookeeper.server.quorum.QuorumPeerMain "E:/zookeeper/zookeeper-3.4.13/conf/zoo3.cfg"
。。。。。。
2019-03-07 19:03:31,939 [myid:3] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2183:ZooKeeperServer@174] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir E:\zookeeper\zookeeper-3.4.13\data3\version-2 snapdir E:\zookeeper\zookeeper-3.4.13\data3\version-2
2019-03-07 19:03:31,942 [myid:3] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2183:Follower@65] - FOLLOWING - LEADER ELECTION TOOK - 112
2019-03-07 19:03:31,945 [myid:3] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2183:QuorumPeer$QuorumServer@184] - Resolved hostname: localhost to address: localhost/127.0.0.1
2019-03-07 19:03:32,048 [myid:3] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2183:Learner@336] - Getting a snapshot from leader 0x100000000
2019-03-07 19:03:32,056 [myid:3] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2183:FileTxnSnapLog@301] - Snapshotting: 0x100000000 to E:\zookeeper\zookeeper-3.4.13\data3\version-2\snapshot.100000000
2019-03-07 19:03:47,177 [myid:3] - WARN  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2183:Follower@119] - Got zxid 0x100000001 expected 0x1
2019-03-07 19:03:47,180 [myid:3] - INFO  [SyncThread:3:FileTxnLog@213] - Creating new log file: log.100000001

　　启动前两个服务器的时候报错链接拒绝，因为在通过端口选举leader的时候没找到端口。在console也看到一些关于选举leader等信息。

6.查看启动状态:

(1)jps查看

C:\Users\Administrator>jps
21984 QuorumPeerMain
1576 QuorumPeerMain
22280 QuorumPeerMain
11132 Jps
4780

(2)连接集群的1281节点并创建第一条数据从1282节点查看数据

 .\zkCli.cmd -server 127.0.0.1:2181

创建节点:

[zk: 127.0.0.1:2181(CONNECTED) 3] create /FirstZnode "Myfirstzookeeper-app"
Created /FirstZnode
[zk: 127.0.0.1:2181(CONNECTED) 4] ls
[zk: 127.0.0.1:2181(CONNECTED) 5] ls /
[zookeeper, FirstZnode]
[zk: 127.0.0.1:2181(CONNECTED) 6] get /FirstZnode
Myfirstzookeeper-app
cZxid = 0x100000003
ctime = Thu Mar 07 19:30:14 CST 2019
mZxid = 0x100000003
mtime = Thu Mar 07 19:30:14 CST 2019
pZxid = 0x100000003
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 20
numChildren = 0

　　接下来链接到1282并继续查看节点信息同上证明集群搭建成功。

(3)linux下面有zkServer.sh status查看节点状态，但是cmd脚本不支持传入参数，这个在研究源码之后编写的bat脚本可以实现windows下面查看zookeeper集群节点状态:

　　通过研究sh的源码发现其进行查询状态的时候是启动的org.apache.zookeeper.client.FourLetterWordMain类，我们查看org.apache.zookeeper.client.FourLetterWordMain的main函数需要的参数如下:

    public static void main(String[] args)
            throws IOException
    {
        if (args.length != 3) {
            System.out.println("Usage: FourLetterWordMain <host> <port> <cmd>");
        } else {
            System.out.println(send4LetterWord(args[0], Integer.parseInt(args[1]), args[2]));
        }
    }

　　所以我们自己写java命令也可以查看状态了。

于是自己编写的查看1281端口的bat脚本如下: 主要做的就是清空classpath的值。然后重新赋值classpath的值，赋值classpath是为了引入zookeeper依赖的jar包

status1281.bat (需要放置到E:\zookeeper\zookeeper-3.4.13，也就是放置到与zookeeperxxx.jar同级目录)

e:
cd zookeeper\zookeeper-3.4.13
set classpath=
set classpath=./*;./lib/*;
java -cp zookeeper-3.4.13.jar -classpath %classpath% org.apache.zookeeper.client.FourLetterWordMain 127.0.0.1 2181 status
pause

结果:

我们修改上面bat里面的脚本查看2182和2183的状态如下:

3.集群版

　　这里研究正式linux环境集群，使用虚拟机进行配置。

　　操作工具我们使用SecureCRT，方便同时向linux发送相同的命令。关于如何开启发送相同命令如下:

一、首先在SecureCRT里同时打开多个服务器session
二、选择菜单栏View -->Chat Windows  对号，此时所有服务器连接下方应该有个空白的部分
三、在空白的部分（Chat Windows）右键鼠标, 选上Send Chat to All Tabs, 这样Chat Windows里会有"<Send chat to all tabs>"的标志

0.当然首先在三个机器装JDK环境

1.准备三台centos操作系统，ip分别为:

　　192.168.1.130 我们称为A机器

　　 192.168.1.131 　　我们称为B机器

　　 192.168.1.133　　我们称为C机器

2.建立相同的目录并上传zookeeperxxx.tar.gz

3.解压上面的目录 (三个做相同操作)

4.复制conf\zoo_sample.cfg到con\zoo.cfg目录下:

[root@localhost conf]# cp ./zoo_sample.cfg ./zoo.cfg
[root@localhost conf]# ls
configuration.xsl  log4j.properties  zoo.cfg  zoo_sample.cfg

5.修改 zoo.cfg

A\B\C机器的内容都如下:

tickTime = 2000
dataDir = /opt/zookeeper/zookeeper-3.4.13/data
clientPort = 2181
initLimit = 5
syncLimit = 2

server.1=192.168.1.130:2888:3888
server.2=192.168.1.131:2888:3888
server.3=192.168.1.133:2888:3888

6.同时创建三个服务器对应的数据目录

mkdir /opt/zookeeper/zookeeper-3.4.13/data

7.在三个机器的对应的data目录下面分别创建对应的id文件(重要)

A机器:

[root@localhost data]# echo "1">myid
[root@localhost data]# cat myid
1

B机器:

[root@localhost data]# echo "2">myid
[root@localhost data]# cat myid
2

C机器:

[root@localhost data]# echo "3">myid
[root@localhost data]# cat myid
3

8.接下来启动三个服务器即可(同时发送命令)

/opt/zookeeper/zookeeper-3.4.13/bin/zkServer.sh start

9.JPS查看JVM

　　三个服务器都有主类为QuorumPeerMain的PID即启动成功。

10查看集群状态

/opt/zookeeper/zookeeper-3.4.13/bin/zkServer.sh status