分布式架构高可用架构篇_01_zookeeper集群的安装、配置、高可用测试
Dubbo 建议使用 Zookeeper 作为服务的注册中心
Zookeeper 集群中只要有过半的节点是正常的情况下，那么整个集群对外就是可用的。正是基于这个 特性，要将 ZK 集群的节点数量要为奇数（2n+1：如 3、5、7 个节点）较为合适。
ZooKeeper 与 Dubbo 服务集群架构图
服务器 1：192.168.1.81 端口：2181、2881、3881
服务器 2：192.168.1.82 端口：2182、2882、3882
服务器 3：192.168.1.83 端口：2183、2883、3883
1、 修改操作系统的/etc/hosts 文件，添加 IP 与主机名映射：
# zookeeper cluster servers
192.168.1.81 edu-zk-01
192.168.1.82 edu-zk-02
192.168.1.83 edu-zk-03
[root@edu-zk-01 root]# cat /etc/hosts | grep edu
192.168.1.81 edu-zk-01
[root@edu-zk-01 root]#
[root@edu-zk-02 ~]# cat /etc/hosts | grep edu
192.168.1.82 edu-zk-02
[root@edu-zk-02 ~]#
2、 下载或上传 zookeeper-3.4.6.tar.gz 到/home/yxq/zookeeper 目录
[root@edu-zk-01 root]# clear
[root@edu-zk-01 root]# cd /home/yxq/zookeeper
[root@edu-zk-01 zookeeper]# wget http://apache.fayea.com/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
[root@edu-zk-02 ~]# mkdir /home/yxq/zookeeper
[root@edu-zk-02 ~]# cd /home/yxq/zookeeper/
[root@edu-zk-02 zookeeper]# wget http://apache.fayea.com/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
3、 解压 zookeeper 安装包，并按节点号对 zookeeper 目录重命名：
服务器1
[root@edu-zk-01 zookeeper]# tar -zxvf zookeeper-3.4.6.tar.gz
[root@edu-zk-01 zookeeper]# mv zookeeper-3.4.6 node-01
服务器2:
[root@edu-zk-02 zookeeper]# tar -zxvf zookeeper-3.4.6.tar.gz
[root@edu-zk-02 zookeeper]# mv zookeeper-3.4.6 node-02
服务器3:
[root@edu-zk-03 ~]# cd /home/yxq/zookeeper
[root@edu-zk-03 zookeeper]# wget http://apache.fayea.com/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
[root@edu-zk-03 zookeeper]# tar -zxvf zookeeper-3.4.6.tar.gz
[root@edu-zk-03 zookeeper]# mv zookeeper-3.4.6 node-03
4、 在各 zookeeper 节点目录下创建以下目录：
[root@edu-zk-01 zookeeper]# cd node-01/
[root@edu-zk-01 node-01]# mkdir data
[root@edu-zk-01 node-01]# mkdir logs
[root@edu-zk-02 zookeeper]# cd node-02/
[root@edu-zk-02 node-02]# mkdir data
[root@edu-zk-02 node-02]# mkdir logs
[root@edu-zk-03 zookeeper]# cd node-03/
[root@edu-zk-03 node-03]# mkdir data
[root@edu-zk-03 node-03]# mkdir logs
将 zookeeper/node-0X/conf 目录下的 zoo_sample.cfg 文件拷贝一份，命名为 zoo.cfg
[root@edu-zk-01 node-01]# cd conf
[root@edu-zk-01 conf]# cp zoo_sample.cfg zoo.cfg
[root@edu-zk-02 node-02]# cd conf
[root@edu-zk-02 conf]# cp zoo_sample.cfg zoo.cfg
[root@edu-zk-03 node-03]# cd conf
[root@edu-zk-03 conf]# cp zoo_sample.cfg zoo.cfg
6、 修改 zoo.cfg 配置文件：
zookeeper/node-01 的配置（/home/yxq/zookeeper/node-01/conf/zoo.cfg）如下：
zookeeper/node-02的配置（/home/yxq/zookeeper/node-02/conf/zoo.cfg）如下：
zookeeper/node-03 的配置（/home/yxq/zookeeper/node-03/conf/zoo.cfg）如下：
参数说明:
tickTime=2000
tickTime 这个时间是作为 Zookeeper 服务器之间或客户端与服务器之间维持心跳的时间间隔,也就是每
个 tickTime 时间就会发送一个心跳。
initLimit=10
initLimit 这个配置项是用来配置 Zookeeper 接受客户端（这里所说的客户端不是用户连接 Zookeeper
服务器的客户端,而是 Zookeeper 服务器集群中连接到 Leader 的 Follower 服务器）初始化连接时最长
能忍受多少个心跳时间间隔数。当已经超过 10 个心跳的时间（也就是 tickTime）长度后 Zookeeper 服
务器还没有收到客户端的返回信息,那么表明这个客户端连接失败。总的时间长度就是 10*2000=20 秒。
syncLimit=5
syncLimit 这个配置项标识 Leader 与 Follower 之间发送消息,请求和应答时间长度,最长不能超过多少
个 tickTime 的时间长度,总的时间长度就是 5*2000=10 秒。
dataDir=/home/yxq/zookeeper/node-01/data
dataDir 顾名思义就是 Zookeeper 保存数据的目录,默认情况下 Zookeeper 将写数据的日志文件也保存在 这个目录里。
clientPort=2181
clientPort 这个端口就是客户端（应用程序）连接 Zookeeper 服务器的端口,Zookeeper 会监听这个端
server.A=B：C：D
server.1=192.168.1.81:2881:3881
server.2=192.168.1.82:2882:3882
server.3=192.168.1.83:2883:3883
A 是一个数字,表示这个是第几号服务器；
B 是这个服务器的 IP 地址（或者是与 IP 地址做了映射的主机名）；
C 第一个端口用来集群成员的信息交换,表示这个服务器与集群中的 Leader 服务器交换信息的端口；
D 是在 leader 挂掉时专门用来进行选举 leader 所用的端口。
注意：如果是伪集群的配置方式，不同的 Zookeeper 实例通信端口号不能一样，所以要给它们分配不 同的端口号。
7、 在 dataDir=/home/yxq/zookeeper/node-0X/data 下创建 myid 文件
编辑 myid 文件，并在对应的 IP 的机器上输入对应的编号。如在 node-01 上，myid 文件内容就是 1,node-02 上就是 2，node-03 上就是 3：
[root@edu-zk-01 node-01]# cd data
[root@edu-zk-01 data]# vi myid
[root@edu-zk-01 data]# cat myid
1
[root@edu-zk-01 data]#
[root@edu-zk-02 conf]# cd ..
[root@edu-zk-02 node-02]# cd data
[root@edu-zk-02 data]# vi myid
[root@edu-zk-02 data]# cat myid
2
[root@edu-zk-03 conf]# cd ..
[root@edu-zk-03 node-03]# cd data
[root@edu-zk-03 data]# vi myid
[root@edu-zk-03 data]# cat myid
3
[root@edu-zk-03 data]#
8、 在防火墙中打开要用到的端口 218X、288X、388X
切换到 root 用户权限，执行以下命令：
[root@edu-zk-01 data]# vi /etc/sysconfig/iptables
[root@edu-zk-01 data]# service iptables restart
edu-zk-03,edu-zk-03同理
[root@edu-zk-02 data]# vi /etc/sysconfig/iptables
[root@edu-zk-02 data]# service iptables restart
[root@edu-zk-03 data]# vi /etc/sysconfig/iptables
[root@edu-zk-03 data]# service iptables restart
9、 启动并测试 zookeeper（要用 yxq用户启动，不要用 root）:
先启用edu-zk-01.集群环境下，只有过半的节点运行正常才不报错，这里报错是正常的
[root@edu-zk-01 root]# clear
[root@edu-zk-01 root]# who
yxq pts/0 2016-04-15 00:48 (192.168.1.61)
[root@edu-zk-01 root]# /home/yxq/zookeeper/node-01/bin/zkServer.sh start
JMX enabled by default
Using config: /home/yxq/zookeeper/node-01/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
查看是否正常启动
其中，QuorumPeerMain 是 zookeeper 进程，说明启动正常
[root@edu-zk-01 root]# jps
3293 Jps
3240 QuorumPeerMain
[root@edu-zk-01 root]# tail -500f zookeeper.out
2016-04-15 01:11:57,970 [myid:] - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /home/yxq/zookeeper/node-01/bin/../conf/zoo.cfg
2016-04-15 01:11:57,974 [myid:] - INFO [main:QuorumPeerConfig@340] - Defaulting to majority quorums
2016-04-15 01:11:57,976 [myid:1] - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2016-04-15 01:11:57,976 [myid:1] - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2016-04-15 01:11:57,977 [myid:1] - INFO [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2016-04-15 01:11:57,985 [myid:1] - INFO [main:QuorumPeerMain@127] - Starting quorum peer
2016-04-15 01:11:57,993 [myid:1] - INFO [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:2181
2016-04-15 01:11:58,015 [myid:1] - INFO [main:QuorumPeer@959] - tickTime set to 2000
2016-04-15 01:11:58,015 [myid:1] - INFO [main:QuorumPeer@979] - minSessionTimeout set to -1
2016-04-15 01:11:58,015 [myid:1] - INFO [main:QuorumPeer@990] - maxSessionTimeout set to -1
2016-04-15 01:11:58,015 [myid:1] - INFO [main:QuorumPeer@1005] - initLimit set to 10
2016-04-15 01:11:58,028 [myid:1] - INFO [main:FileSnap@83] - Reading snapshot /home/yxq/zookeeper/node-01/data/version-2/snapshot.0
2016-04-15 01:11:58,037 [myid:1] - INFO [Thread-1:QuorumCnxManager$Listener@504] - My election bind port: /192.168.1.81:3881
2016-04-15 01:11:58,044 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumPeer@714] - LOOKING
2016-04-15 01:11:58,045 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@815] - New election. My id = 1, proposed zxid=0x0
2016-04-15 01:11:58,047 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x1 (n.peerEpoch) LOOKING (my state)
2016-04-15 01:11:58,050 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@382] - Cannot open channel to 2 at election address /192.168.1.82:3882
Java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:341)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:449)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:430)
at java.lang.Thread.run(Thread.java:744)
2016-04-15 01:11:58,053 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@382] - Cannot open channel to 3 at election address /192.168.1.83:3883
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:341)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:449)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:430)
at java.lang.Thread.run(Thread.java:744)
2016-04-15 01:11:58,256 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 2 at election address /192.168.1.82:3882
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2016-04-15 01:11:58,259 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 3 at election address /192.168.1.83:3883
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2016-04-15 01:11:58,260 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@849] - Notification time out: 400
2016-04-15 01:11:58,663 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 2 at election address /192.168.1.82:3882
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2016-04-15 01:11:58,666 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 3 at election address /192.168.1.83:3883
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2016-04-15 01:11:58,667 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@849] - Notification time out: 800
2016-04-15 01:11:59,470 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 2 at election address /192.168.1.82:3882
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2016-04-15 01:11:59,472 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 3 at election address /192.168.1.83:3883
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2016-04-15 01:11:59,478 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@849] - Notification time out: 1600
2016-04-15 01:12:01,080 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 2 at election address /192.168.1.82:3882
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2016-04-15 01:12:01,084 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 3 at election address /192.168.1.83:3883
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2016-04-15 01:12:01,085 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@849] - Notification time out: 3200
2016-04-15 01:12:04,287 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 2 at election address /192.168.1.82:3882
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2016-04-15 01:12:04,290 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 3 at election address /192.168.1.83:3883
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2016-04-15 01:12:04,291 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@849] - Notification time out: 6400
启动edu-zk-02
这里三个节点中有两个节点启动，所以日志中不会显示错误信息
[root@edu-zk-02 root]# who
yxq pts/0 2016-04-15 00:50 (192.168.1.61)
[root@edu-zk-02 root]# /home/yxq/zookeeper/node-02/bin/zkServer.sh start
JMX enabled by default
Using config: /home/yxq/zookeeper/node-02/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@edu-zk-02 root]# ls
anaconda-ks.cfg Desktop Documents Downloads install.log install.log.syslog Music Pictures Public Templates Videos zookeeper.out
查看是否正常启动
其中，QuorumPeerMain 是 zookeeper 进程，说明启动正常
[root@edu-zk-02 root]# jps
2929 QuorumPeerMain
2986 Jps
[root@edu-zk-02 root]# tail -500f zookeeper.out
2016-04-15 01:15:24,042 [myid:] - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /home/yxq/zookeeper/node-02/bin/../conf/zoo.cfg
2016-04-15 01:15:24,051 [myid:] - INFO [main:QuorumPeerConfig@340] - Defaulting to majority quorums
2016-04-15 01:15:24,055 [myid:2] - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2016-04-15 01:15:24,056 [myid:2] - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2016-04-15 01:15:24,056 [myid:2] - INFO [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2016-04-15 01:15:24,064 [myid:2] - INFO [main:QuorumPeerMain@127] - Starting quorum peer
2016-04-15 01:15:24,078 [myid:2] - INFO [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:2182
2016-04-15 01:15:24,100 [myid:2] - INFO [main:QuorumPeer@959] - tickTime set to 2000
2016-04-15 01:15:24,100 [myid:2] - INFO [main:QuorumPeer@979] - minSessionTimeout set to -1
2016-04-15 01:15:24,100 [myid:2] - INFO [main:QuorumPeer@990] - maxSessionTimeout set to -1
2016-04-15 01:15:24,100 [myid:2] - INFO [main:QuorumPeer@1005] - initLimit set to 10
2016-04-15 01:15:24,118 [myid:2] - INFO [Thread-1:QuorumCnxManager$Listener@504] - My election bind port: /192.168.1.82:3882
2016-04-15 01:15:24,126 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:QuorumPeer@714] - LOOKING
2016-04-15 01:15:24,127 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:FastLeaderElection@815] - New election. My id = 2, proposed zxid=0x0
2016-04-15 01:15:24,132 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@597] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x1 (n.peerEpoch) LOOKING (my state)
2016-04-15 01:15:24,134 [myid:2] - WARN [WorkerSender[myid=2]:QuorumCnxManager@382] - Cannot open channel to 3 at election address /192.168.1.83:3883
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:341)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:449)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:430)
at java.lang.Thread.run(Thread.java:744)
2016-04-15 01:15:24,136 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@597] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x1 (n.peerEpoch) LOOKING (my state)
2016-04-15 01:15:24,138 [myid:2] - INFO [WorkerReceiver[myid=2]:FastLeaderElection@597] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x1 (n.peerEpoch) LOOKING (my state)
2016-04-15 01:15:24,340 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:QuorumPeer@796] - LEADING
2016-04-15 01:15:24,345 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Leader@60] - TCP NoDelay set to: true
2016-04-15 01:15:24,360 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2016-04-15 01:15:24,361 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:host.name=edu-zk-02
2016-04-15 01:15:24,361 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:java.version=1.7.0_45
2016-04-15 01:15:24,361 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:java.vendor=Oracle Corporation
2016-04-15 01:15:24,362 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:java.home=/usr/java/jdk1.7.0_45/jre
2016-04-15 01:15:24,362 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:java.class.path=/home/yxq/zookeeper/node-02/bin/../build/classes:/home/yxq/zookeeper/node-02/bin/../build/lib/*.jar:/home/yxq/zookeeper/node-02/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/yxq/zookeeper/node-02/bin/../lib/slf4j-api-1.6.1.jar:/home/yxq/zookeeper/node-02/bin/../lib/netty-3.7.0.Final.jar:/home/yxq/zookeeper/node-02/bin/../lib/log4j-1.2.16.jar:/home/yxq/zookeeper/node-02/bin/../lib/jline-0.9.94.jar:/home/yxq/zookeeper/node-02/bin/../zookeeper-3.4.6.jar:/home/yxq/zookeeper/node-02/bin/../src/java/lib/*.jar:/home/yxq/zookeeper/node-02/bin/../conf:/usr/jdk/lib/
2016-04-15 01:15:24,362 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2016-04-15 01:15:24,362 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:java.io.tmpdir=/tmp
2016-04-15 01:15:24,363 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:java.compiler=<NA>
2016-04-15 01:15:24,363 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:os.name=Linux
2016-04-15 01:15:24,363 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:os.arch=amd64
2016-04-15 01:15:24,363 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:os.version=2.6.32-573.18.1.el6.x86_64
2016-04-15 01:15:24,364 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:user.name=root
2016-04-15 01:15:24,364 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:user.home=/root
2016-04-15 01:15:24,364 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Environment@100] - Server environment:user.dir=/root
2016-04-15 01:15:24,367 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:ZooKeeperServer@162] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /home/yxq/zookeeper/node-02/logs/version-2 snapdir /home/yxq/zookeeper/node-02/data/version-2
2016-04-15 01:15:24,369 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Leader@358] - LEADING - LEADER ELECTION TOOK - 241
2016-04-15 01:15:24,380 [myid:2] - INFO [LearnerHandler-/192.168.1.81:48238:LearnerHandler@330] - Follower sid: 1 : info : org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@43cfd989
2016-04-15 01:15:24,395 [myid:2] - INFO [LearnerHandler-/192.168.1.81:48238:LearnerHandler@385] - Synchronizing with Follower sid: 1 maxCommittedLog=0x0 minCommittedLog=0x0 peerLastZxid=0x0
2016-04-15 01:15:24,396 [myid:2] - INFO [LearnerHandler-/192.168.1.81:48238:LearnerHandler@462] - Sending DIFF
2016-04-15 01:15:24,403 [myid:2] - INFO [LearnerHandler-/192.168.1.81:48238:LearnerHandler@522] - Received NEWLEADER-ACK message from 1
2016-04-15 01:15:24,410 [myid:2] - INFO [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2182:Leader@943] - Have quorum of supporters, sids: [ 1,2 ]; starting up and setting last processed zxid: 0x200000000
先不启动edu-zk-03,
这里我们使用zkServer.sh status查看主从节点
leader:主节点
[root@edu-zk-02 root]# /home/yxq/zookeeper/node-02/bin/zkServer.sh status
JMX enabled by default
Using config: /home/yxq/zookeeper/node-02/bin/../conf/zoo.cfg
Mode: leader
[root@edu-zk-02 root]#
follower从节点
[root@edu-zk-01 root]# /home/yxq/zookeeper/node-01/bin/zkServer.sh status
JMX enabled by default
Using config: /home/yxq/zookeeper/node-01/bin/../conf/zoo.cfg
Mode: follower
[root@edu-zk-01 root]#
11、配置 zookeeper 开机使用yxq用户启动：
[root@edu-zk-01 root]# vi /etc/rc.local
[root@edu-zk-01 root]# cat /etc/rc.local | grep yxq
su - yxq -c '/home/yxq/zookeeper/node-01/bin/zkServer.sh start'
[root@edu-zk-01 root]#
[root@edu-zk-02 root]# vi /etc/rc.local
[root@edu-zk-02 root]# cat /etc/rc.local | grep yxq
su - yxq -c '/home/yxq/zookeeper/node-02/bin/zkServer.sh start'
[root@edu-zk-02 root]#
[root@edu-zk-03 ~]# vi /etc/rc.local
[root@edu-zk-03 ~]# cat /etc/rc.local | grep yxq
su - yxq -c '/home/yxq/zookeeper/node-03/bin/zkServer.sh start'
[root@edu-zk-03 ~]#
二、安装 Dubbo 管控台（基础篇有讲,此处重点讲管控台如何链接集群）：
Dubbo 管控台可以对注册到 zookeeper 注册中心的服务或服务消费者进行管理，但管控台是否正常对Dubbo 服务没有影响，管控台也不需要高可用，因此可以单节点部署。
IP: 192.168.1.81
部署容器：Tomcat7
端口：8080
root@edu-zk-01 webapps]# cd /root
[root@edu-zk-01 root]# ls
anaconda-ks.cfg Desktop Documents Downloads dubbo-governance.log install.log install.log.syslog Music Pictures Public Templates Videos zookeeper.out
[root@edu-zk-01 root]# cd /home/yxq/zookeeper/tomcat7/webapps/ROOT/
[root@edu-zk-01 ROOT]# ls
crossdomain.xml css dubbo-governance.log favicon.ico images js META-INF SpryAssets WEB-INF
[root@edu-zk-01 ROOT]# cd WEB-INF/
[root@edu-zk-01 WEB-INF]# ls
classes common dubbo.properties forms i18n lib log4j.xml templates webx-governance.xml webx-home.xml web.xml webx-personal.xml webx-sysinfo.xml webx-sysmanage.xml webx.xml
[root@edu-zk-01 WEB-INF]# vi dubbo.properties
[root@edu-zk-01 WEB-INF]# cat dubbo.properties | grep re
dubbo.registry.address=zookeeper://192.168.1.81:2181?backup=192.168.1.82:2182,192.168.1.83:2183
三、测试集群高可用
启用edu-zk-03,这里是从节点
[root@edu-zk-03 conf]# /home/yxq/zookeeper/node-03/bin/zkServer.sh start
JMX enabled by default
Using config: /home/yxq/zookeeper/node-03/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@edu-zk-03 conf]# /home/yxq/zookeeper/node-03/bin/zkServer.sh status
JMX enabled by default
Using config: /home/yxq/zookeeper/node-03/bin/../conf/zoo.cfg
Mode: follower
我们再关掉edu-zk-02的leader节点
[root@edu-zk-02 root]# /home/yxq/zookeeper/node-02/bin/zkServer.sh stop
JMX enabled by default
Using config: /home/yxq/zookeeper/node-02/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
我们再查看edu-zk-03 status
[root@edu-zk-03 conf]# /home/yxq/zookeeper/node-03/bin/zkServer.sh status
JMX enabled by default
Using config: /home/yxq/zookeeper/node-03/bin/../conf/zoo.cfg
Mode: leader
这时edu-zk-03变成了leader,
provider控制台信息
016-04-15 16:53:15,516 WARN [ClientCnxn.java:1089] : Session 0x15418fd25210000 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2016-04-15 16:53:15,517 DEBUG [ClientCnxnSocketNIO.java:192] : Ignoring exception during shutdown input
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:798)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:426)
at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:189)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1097)
2016-04-15 16:53:15,518 DEBUG [ClientCnxnSocketNIO.java:199] : Ignoring exception during shutdown output
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:815)
at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:434)
at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:196)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1097)
2016-04-15 16:53:16,506 INFO [ClientCnxn.java:966] : Opening socket connection to server 192.168.1.83/192.168.1.83:2183. Will not attempt to authenticate using SASL (unknown error)
2016-04-15 16:53:16,507 INFO [ClientCnxn.java:849] : Socket connection established to 192.168.1.83/192.168.1.83:2183, initiating session
2016-04-15 16:53:16,510 DEBUG [ClientCnxn.java:889] : Session establishment request sent on 192.168.1.83/192.168.1.83:2183
2016-04-15 16:53:16,515 INFO [ClientCnxn.java:1207] : Session establishment complete on server 192.168.1.83/192.168.1.83:2183, sessionid = 0x15418fd25210000, negotiated timeout = 30000
2016-04-15 16:53:16,515 DEBUG [ZkClient.java:351] : Received event: WatchedEvent state:SyncConnected type:None path:null
2016-04-15 16:53:16,516 INFO [ZkClient.java:449] : zookeeper state changed (SyncConnected)
2016-04-15 16:53:16,516 DEBUG [ZkEventThread.java:88] : New event: ZkEvent[State changed to SyncConnected sent to com.alibaba.dubbo.remoting.zookeeper.zkclient.ZkclientZookeeperClient$1@20f8f517]
2016-04-15 16:53:16,516 DEBUG [ZkClient.java:395] : Leaving process event
2016-04-15 16:53:16,516 DEBUG [ZkEventThread.java:69] : Delivering event #2 ZkEvent[State changed to SyncConnected sent to com.alibaba.dubbo.remoting.zookeeper.zkclient.ZkclientZookeeperClient$1@20f8f517]
2016-04-15 16:53:16,517 DEBUG [ZkEventThread.java:79] : Delivering event #2 done
JMX enabled by default
Using config: /home/yxq/zookeeper/node-03/bin/../conf/zoo.cfg
Mode: leader
我们再关闭edu-zk-01节点
这里provider控制台信息报错(集群节点没有过半),这时dubbo管理台是可以正常运行的，但是集群是无用，提供者服务是缓存的，如果重启dubbo管理控制台，会发现启动不了，但是如果再启动一个节点edu-zk-01或edu-zk-02,管理控制台又立即可以运行起来。
java.net.SocketException: Connection reset by peer: shutdown
at sun.nio.ch.Net.shutdown(Native Method)
at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:819)
at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:434)
at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:196)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1097)
2016-04-15 16:58:09,848 INFO [ClientCnxn.java:966] : Opening socket connection to server 192.168.1.81/192.168.1.81:2181. Will not attempt to authenticate using SASL (unknown error)
2016-04-15 16:58:10,848 WARN [ClientCnxn.java:1089] : Session 0x15418fd25210000 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2016-04-15 16:58:10,848 DEBUG [ClientCnxnSocketNIO.java:192] : Ignoring exception during shutdown input
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:798)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:426)
at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:189)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1097)
2016-04-15 16:58:10,849 DEBUG [ClientCnxnSocketNIO.java:199] : Ignoring exception during shutdown output
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:815)
at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:434)
at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:196)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1097)
2016-04-15 16:58:11,552 INFO [ClientCnxn.java:966] : Opening socket connection to server 192.168.1.82/192.168.1.82:2182. Will not attempt to authenticate using SASL (unknown error)
2016-04-15 16:58:12,553 WARN [ClientCnxn.java:1089] : Session 0x15418fd25210000 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2016-04-15 16:58:12,554 DEBUG [ClientCnxnSocketNIO.java:192] : Ignoring exception during shutdown input
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:798)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:426)
at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:189)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1097)
2016-04-15 16:58:12,555 DEBUG [ClientCnxnSocketNIO.java:199] : Ignoring exception during shutdown output
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:815)
at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:434)
at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:196)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1157)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1097)
2016-04-15 16:58:13,909 INFO [ClientCnxn.java:966] : Opening socket connection to server 192.168.1.83/192.168.1.83:2183. Will not attempt to authenticate using SASL (unknown error)
2016-04-15 16:58:13,910 INFO [ClientCnxn.java:849] : Socket connection established to 192.168.1.83/192.168.1.83:2183, initiating session
2016-04-15 16:58:13,911 DEBUG [ClientCnxn.java:889] : Session establishment request sent on 192.168.1.83/192.168.1.83:2183
2016-04-15 16:58:13,914 INFO [ClientCnxn.java:1085] : Unable to read additional data from server sessionid 0x15418fd25210000, likely server has closed socket, closing socket connection and attempting reconnect
2016-04-15 16:58:13,915 DEBUG [ClientCnxnSocketNIO.java:199] : Ignoring exception during shutdown output
