Zookeeper笔记

Zookeeper

服务器

腾讯服务器

安装目录结构

github地址

ProcessOn地址:https://www.processon.com/mindmap/61b04da95653bb4f1e5ec85e

安装

解压文件:

[root@VM-0-9-centos zookeeper]# tar -zxvf apache-zookeeper-3.5.8-bin.tar.gz 

信息:

[root@VM-0-9-centos ~]# echo 太难了吧
太难了吧
[root@VM-0-9-centos ~]# cd usr/local
-bash: cd: usr/local: 没有那个文件或目录
[root@VM-0-9-centos ~]# cd /usr/local
[root@VM-0-9-centos local]# mkdir zookeeper
[root@VM-0-9-centos local]# cd zookeeper/
[root@VM-0-9-centos zookeeper]# ll
总用量 0

删除文件:

[root@VM-0-9-centos zookeeper]# rm -rf apache-zookeeper-3.5.8-bin.tar.gz 
[root@VM-0-9-centos zookeeper]# ll
总用量 4
drwxr-xr-x 6 root root 4096 12月  7 11:05 apache-zookeeper-3.5.8-bin

展示文件夹目录:

[root@VM-0-9-centos apache-zookeeper-3.5.8-bin]# ll
总用量 40
drwxr-xr-x 2 root root  4096 5月   4 2020 bin
drwxr-xr-x 2 root root  4096 5月   4 2020 conf
drwxr-xr-x 5 root root  4096 5月   4 2020 docs
drwxr-xr-x 2 root root  4096 12月  7 11:05 lib
-rw-r--r-- 1 root root 11358 5月   4 2020 LICENSE.txt
-rw-r--r-- 1 root root   432 5月   4 2020 NOTICE.txt
-rw-r--r-- 1 root root  1560 5月   4 2020 README.md
-rw-r--r-- 1 root root  1347 5月   4 2020 README_packaging.txt
[root@VM-0-9-centos apache-zookeeper-3.5.8-bin]# cd bin
[root@VM-0-9-centos bin]# ls
README.txt    zkCli.cmd  zkEnv.cmd  zkServer.cmd            zkServer.sh          zkTxnLogToolkit.sh
zkCleanup.sh  zkCli.sh   zkEnv.sh   zkServer-initialize.sh  zkTxnLogToolkit.cmd

查看文件内容(读写方式)

vim + 文件名

按Esc ,后输入:

:w 保存但不退出

:wq 保存并退出

:q 退出

:q! 强制退出，不保存

:e! 放弃所有修改，从上次保存文件开始再编辑命令历史

将

#zookeper 数据存储目录及日志保存目录（如果没有指明dataLogDir，则日志也保存在这个文件中）
dataDir=/tmp/zookeeper

改为:

#zookeper 数据存储目录及日志保存目录（如果没有指明dataLogDir，则日志也保存在这个文件中）
dataDir=/usr/local/zookeeper/zkdata

重命名:

[root@VM-0-9-centos conf]# mv zoo_sample.cfg zoo.cfg
[root@VM-0-9-centos conf]# ll
总用量 12
-rw-r--r-- 1 root root  535 5月   4 2020 configuration.xsl
-rw-r--r-- 1 root root 2712 5月   4 2020 log4j.properties
-rw-r--r-- 1 root root  935 12月  7 12:59 zoo.cfg

启动ZK服务器: bin路径下# ./zkServer.sh start …/conf/zoo.cfg

[root@VM-0-9-centos bin]# ./zkServer.sh
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/apache-zookeeper-3.5.8-bin/bin/../conf/zoo.cfg
Usage: ./zkServer.sh [--config <conf-dir>] {start|start-foreground|stop|restart|status|print-cmd}
[root@VM-0-9-centos bin]# ./zkServer.sh start ../conf/zoo.cfg
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: ../conf/zoo.cfg
Starting zookeeper ... STARTED

检查状态:

[root@VM-0-9-centos bin]# ./zkServer.sh status
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/apache-zookeeper-3.5.8-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: standalone  

Mode: standalone --表示单节点

停止命令:

[root@VM-0-9-centos bin]# ./zkServer.sh stop ../conf/zoo.cfg
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: ../conf/zoo.cfg
Stopping zookeeper ... STOPPED

启动客户端zkCli (需先启动zk服务)

[root@VM-0-9-centos bin]# ./zkServer.sh start ../conf/zoo.cfg 
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: ../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@VM-0-9-centos bin]# ./zkCli.sh
/usr/bin/java
Connecting to localhost:2181
....

内部命令:

ZooKeeper -server host:port cmd args
        addauth scheme auth
        close 
        config [-c] [-w] [-s]
        connect host:port
        create [-s] [-e] [-c] [-t ttl] path [data] [acl]
        delete [-v version] path
        deleteall path
        delquota [-n|-b] path
        get [-s] [-w] path
        getAcl [-s] path
        history 
        listquota path
        ls [-s] [-w] [-R] path
        ls2 path [watch]
        printwatches on|off
        quit 
        reconfig [-s] [-v version] [[-file path] | [-members serverID=host:port1:port2;port3[,...]*]] | [-add serverId=host:port1:port2;port3[,...]]* [-remove serverId[,...]*]
        redo cmdno
        removewatches path [-c|-d|-a] [-l]
        rmr path
        set [-s] [-v version] path data
        setAcl [-s] [-v version] [-R] path acl
        setquota -n|-b val path
        stat [-w] path
        sync path

ZK内部以节点(树)的方式保存数据

创建节点与查看节点:

[zk: localhost:2181(CONNECTED) 3] create /test1
Created /test1
[zk: localhost:2181(CONNECTED) 4] ls /
[test1, zookeeper]
[zk: localhost:2181(CONNECTED) 5] create /test1/sub1
Created /test1/sub1
[zk: localhost:2181(CONNECTED) 6] ls /
[test1, zookeeper]
[zk: localhost:2181(CONNECTED) 7] ls /test1
[sub1]

路径必须以根路径斜杠"/"开头

zk中的znode是什么样的结构
zk中的znode,包含了四个部分：
data：保存数据
acl：权限，定义了什么样的用户能够操作这个节点，且能够进行怎样的操作
c：create 创建权限，允许在该节点下创建子节点
w：write 更新权限，允许更新该节点的数据
r: read 读取权限，允许读取该节点的内容以及子节点的列表信息
d: delete 删除权限，允许删除该节点的子节点
a：admin管理者权限，允许对该节点进行acl权限设置
stat： 描述当前znode的元数据
child ：当前节点的子节点

创建带数据的节点 , 查看节点下数据及其详细信息

[zk: localhost:2181(CONNECTED) 12] create /test2 abc
Created /test2
[zk: localhost:2181(CONNECTED) 13] get /test2
abc
[zk: localhost:2181(CONNECTED) 14] get -s /test2
abc
cZxid = 0x4
ctime = Tue Dec 07 13:41:10 CST 2021
mZxid = 0x4
mtime = Tue Dec 07 13:41:10 CST 2021
pZxid = 0x4
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 3
numChildren = 0

解释:

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-IC3AfyvW-1640140815389)(D:\python\笔记\img\image-20211207134322088.png)]

创建持久序号节点: (防止并发) 特点:单调递增

[zk: localhost:2181(CONNECTED) 15] ls /
[test1, test2, zookeeper]
[zk: localhost:2181(CONNECTED) 16] create /test3
Created /test3
[zk: localhost:2181(CONNECTED) 17] create /test3
Node already exists: /test3
[zk: localhost:2181(CONNECTED) 18] create -s /test3
Created /test30000000003
[zk: localhost:2181(CONNECTED) 19] create -s /test3
Created /test30000000004
[zk: localhost:2181(CONNECTED) 20] create -s /test3
Created /test30000000005
[zk: localhost:2181(CONNECTED) 21] ls /
[test1, test2, test3, test30000000003, test30000000004, test30000000005, zookeeper]

临时节点: 随着会话的结束,自动删除(大约10秒延迟)

[zk: localhost:2181(CONNECTED) 22] create -e /test4
Created /test4
[zk: localhost:2181(CONNECTED) 23] ls /
[test1, test2, test3, test30000000003, test30000000004, test30000000005, test4, zookeeper]
[zk: localhost:2181(CONNECTED) 1] ls /
[test1, test2, test3, test30000000003, test30000000004, test30000000005, test4, zookeeper]
[zk: localhost:2181(CONNECTED) 2] ls /
[test1, test2, test3, test30000000003, test30000000004, test30000000005, zookeeper]

临时序号节点:

[zk: localhost:2181(CONNECTED) 3] create -e -s /test6
Created /test60000000007
[zk: localhost:2181(CONNECTED) 4] get -s /test60000000007
null
cZxid = 0xd
ctime = Tue Dec 07 14:49:20 CST 2021
mZxid = 0xd
mtime = Tue Dec 07 14:49:20 CST 2021
pZxid = 0xd
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x10000858b9f0001
dataLength = 0
numChildren = 0

ephemeralOwner = 0x10000858b9f0001 持久节点值为0,临时节点为其sessionid

创建容器节点: -c 特点:无子节点后会被定期(60s)删除

删除节点

[zk: localhost:2181(CONNECTED) 5] create -c /Mycontainer
Created /Mycontainer
[zk: localhost:2181(CONNECTED) 6] ls /
[Mycontainer, test1, test2, test3, test30000000003, test30000000004, test30000000005, test60000000007, zookeeper]
[zk: localhost:2181(CONNECTED) 7] create -c /Mycontainer/sub1
Created /Mycontainer/sub1
[zk: localhost:2181(CONNECTED) 8] create -c /Mycontainer/sub2
Created /Mycontainer/sub2
[zk: localhost:2181(CONNECTED) 9] ls /Mycontainer
[sub1, sub2]
[zk: localhost:2181(CONNECTED) 10] delete /Mycontainer/sub1
[zk: localhost:2181(CONNECTED) 11] ls /Mycontainer
[sub2]
[zk: localhost:2181(CONNECTED) 12] delete /Mycontainer/sub2
[zk: localhost:2181(CONNECTED) 13] ls /Mycontainer
[]
[zk: localhost:2181(CONNECTED) 14] ls /
[test1, test2, test3, test30000000003, test30000000004, test30000000005, test60000000007, zookeeper]

TTL节点,指定到期时间,到期自动销毁

略

日志与快照

[root@VM-0-9-centos local]# cd zookeeper
[root@VM-0-9-centos zookeeper]# ll
总用量 8
drwxr-xr-x 7 root root 4096 12月  7 13:02 apache-zookeeper-3.5.8-bin
drwxr-xr-x 3 root root 4096 12月  7 13:18 zkdata
[root@VM-0-9-centos zookeeper]# cd zkdada
-bash: cd: zkdada: 没有那个文件或目录
[root@VM-0-9-centos zookeeper]# cd zkdata/
[root@VM-0-9-centos zkdata]# ll
总用量 8
drwxr-xr-x 2 root root 4096 12月  7 13:18 version-2
-rw-r--r-- 1 root root    4 12月  7 13:18 zookeeper_server.pid
[root@VM-0-9-centos zkdata]# cd version-2/
[root@VM-0-9-centos version-2]# ll
总用量 12
-rw-r--r-- 1 root root 67108880 12月  7 14:56 log.1   (日志)
-rw-r--r-- 1 root root      424 12月  7 13:18 snapshot.0  (快照)

zk通过
在恢复时先恢复快
由
度更快

递归查询 : 查询下所有子节点

[zk: localhost:2181(CONNECTED) 2] ls -R /test1
/test1
/test1/sub1

删除所有节点

删除单节点

对节点赋值

[zk: localhost:2181(CONNECTED) 3] deleteall /test1
[zk: localhost:2181(CONNECTED) 4] ls /
[test2, test3, test30000000003, test30000000004, test30000000005, zookeeper]
[zk: localhost:2181(CONNECTED) 6] delete /test2
[zk: localhost:2181(CONNECTED) 7] ls /
[test3, test30000000003, test30000000004, test30000000005, zookeeper]
[zk: localhost:2181(CONNECTED) 8] get /test3
null
[zk: localhost:2181(CONNECTED) 9] set /test3 Hello
[zk: localhost:2181(CONNECTED) 10] get /test3
Hello

乐观锁删除:

只有当 dataVersion = 1 与参数 -v 1 对应才会删除成功

[zk: localhost:2181(CONNECTED) 11] get -s /test3
Hello
cZxid = 0x5
ctime = Tue Dec 07 13:52:37 CST 2021
mZxid = 0x19
mtime = Tue Dec 07 15:25:58 CST 2021
pZxid = 0x5
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
[zk: localhost:2181(CONNECTED) 12] delete -v 0 /test3
version No is not valid : /test3
[zk: localhost:2181(CONNECTED) 13] delete -v 1 /test3
[zk: localhost:2181(CONNECTED) 14] ls /
[test30000000003, test30000000004, test30000000005, zookeeper]

权限管理:

会话1

[zk: localhost:2181(CONNECTED) 15] addauth digest Tiam:123:cdrwa
[zk: localhost:2181(CONNECTED) 16] create /test-node
Created /test-node
[zk: localhost:2181(CONNECTED) 17] set /test-node Hello
[zk: localhost:2181(CONNECTED) 18] get /test-node
Hello
[zk: localhost:2181(CONNECTED) 19] create /test-auth hahaha auth:Tiam:123:cdrwa
Created /test-auth
[zk: localhost:2181(CONNECTED) 20] get /test-auth
hahaha

会话2: 无权限查看

WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /
[test-node, test30000000003, test30000000004, test30000000005, zookeeper]
[zk: localhost:2181(CONNECTED) 1] get /test-node
Hello
[zk: localhost:2181(CONNECTED) 2] ls /
[test-auth, test-node, test30000000003, test30000000004, test30000000005, zookeeper]
[zk: localhost:2181(CONNECTED) 3] get /test-auth
org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /test-auth

Zookeeper客户端

Zookeeper官方

shell命令 客户端

cd /usr/local/zookeeper/apache-zookeeper-3.5.8-bin/bin

./zkCli.sh

Curator客户端 是对zookeeper支持最好的一款客户端工具

watch 监听机制

**会话1:**监听内容,监听目录 (只监听子文件夹)

[zk: localhost:2181(CONNECTED) 2] get -w /test-node
Hello
[zk: localhost:2181(CONNECTED) 3] ls -w /test-node
[]

**会话2:**一但其他会话改变值

[zk: localhost:2181(CONNECTED) 0] set /test-node centos
[zk: localhost:2181(CONNECTED) 1] create /test-node/watch-node
Created /test-node/watch-node

**会话1:**会话监听提示值改变

[zk: localhost:2181(CONNECTED) 4] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeDataChanged path:/test-node

WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/test-node

监听目录下所有文件:

[zk: localhost:2181(CONNECTED) 5] ls -R -w /test-node
/test-node
/test-node/sub1
/test-node/watch-node
/test-node/sub1/ss1
[zk: localhost:2181(CONNECTED) 6] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/test-node/sub1

关闭，开启，防火墙

# 查看状态 ==> active (running)  运行中
[root@VM-0-9-centos ~]# sudo systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: active (running) since 二 2021-12-07 10:52:33 CST; 1 day 1h ago
     Docs: man:firewalld(1)
 Main PID: 576 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─576 /usr/bin/python2 -Es /usr/sbin/firewalld --nofork --nopid

12月 07 10:52:31 VM-0-9-centos systemd[1]: Starting firewalld - dynamic firewa....
12月 07 10:52:33 VM-0-9-centos systemd[1]: Started firewalld - dynamic firewal....
12月 07 10:52:33 VM-0-9-centos firewalld[576]: WARNING: AllowZoneDrifting is e....
Hint: Some lines were ellipsized, use -l to show in full.
[root@VM-0-9-centos ~]# sudo systemctl stop firewalld
#  inactive (dead) ==> 防火墙已关闭
[root@VM-0-9-centos ~]# sudo systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since 三 2021-12-08 12:14:13 CST; 12s ago
     Docs: man:firewalld(1)
  Process: 576 ExecStart=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS (code=exited, status=0/SUCCESS)
 Main PID: 576 (code=exited, status=0/SUCCESS)

12月 07 10:52:31 VM-0-9-centos systemd[1]: Starting firewalld - dynamic firewa....
12月 07 10:52:33 VM-0-9-centos systemd[1]: Started firewalld - dynamic firewal....
12月 07 10:52:33 VM-0-9-centos firewalld[576]: WARNING: AllowZoneDrifting is e....
12月 08 12:14:13 VM-0-9-centos systemd[1]: Stopping firewalld - dynamic firewa....
12月 08 12:14:13 VM-0-9-centos systemd[1]: Stopped firewalld - dynamic firewal....
Hint: Some lines were ellipsized, use -l to show in full.

打开:sudo systemctl start firewalld

搭建服务器节点

[root@VM-0-9-centos zookeeper]# cd zkdata/
[root@VM-0-9-centos zkdata]# ll
总用量 4
drwxr-xr-x 2 root root 4096 12月  7 13:18 version-2
[root@VM-0-9-centos zkdata]# mkdir zk1
[root@VM-0-9-centos zkdata]# ll
总用量 8
drwxr-xr-x 2 root root 4096 12月  7 13:18 version-2
drwxr-xr-x 2 root root 4096 12月  8 13:25 zk1
[root@VM-0-9-centos zkdata]# ll
总用量 8
drwxr-xr-x 2 root root 4096 12月  7 13:18 version-2
drwxr-xr-x 2 root root 4096 12月  8 13:25 zk1
[root@VM-0-9-centos zkdata]# mkdir zk2
[root@VM-0-9-centos zkdata]# mkdir zk3
[root@VM-0-9-centos zkdata]# mkdir zk4
[root@VM-0-9-centos zkdata]# ll
总用量 20
drwxr-xr-x 2 root root 4096 12月  7 13:18 version-2
drwxr-xr-x 2 root root 4096 12月  8 13:25 zk1
drwxr-xr-x 2 root root 4096 12月  8 13:25 zk2
drwxr-xr-x 2 root root 4096 12月  8 13:25 zk3
drwxr-xr-x 2 root root 4096 12月  8 13:26 zk4

对每个集群添加myid

[root@VM-0-9-centos zkdata]# echo 1 > ./zk1/myid
[root@VM-0-9-centos zkdata]# cd zk1
[root@VM-0-9-centos zk1]# ll
总用量 4
-rw-r--r-- 1 root root 2 12月  8 13:28 myid
[root@VM-0-9-centos zk1]# vim
[root@VM-0-9-centos zk1]# vi myid
[root@VM-0-9-centos zk1]# cd ..
[root@VM-0-9-centos zkdata]# echo 2 > ./zk2/myid
[root@VM-0-9-centos zkdata]# echo 3 > ./zk3/myid
[root@VM-0-9-centos zkdata]# echo 4 > ./zk4/myid
[root@VM-0-9-centos zkdata]# cd zk4
[root@VM-0-9-centos zk4]# ll
总用量 4
-rw-r--r-- 1 root root 2 12月  8 13:32 myid
[root@VM-0-9-centos zk4]# cat myid 
4

配置文件内容:zoo1.cfg

复制文件

[root@VM-0-9-centos conf]# cp zoo.cfg zoo1.cfg
[root@VM-0-9-centos conf]# vim zoo1.cfg 
[root@VM-0-9-centos conf]# cp zoo1.cfg zoo2.cfg
[root@VM-0-9-centos conf]# cp zoo1.cfg zoo3.cfg
[root@VM-0-9-centos conf]# cp zoo1.cfg zoo4.cfg
[root@VM-0-9-centos conf]# ll
总用量 28
-rw-r--r-- 1 root root  535 5月   4 2020 configuration.xsl
-rw-r--r-- 1 root root 2712 5月   4 2020 log4j.properties
-rw-r--r-- 1 root root 1171 12月  8 13:50 zoo1.cfg
-rw-r--r-- 1 root root 1171 12月  8 13:51 zoo2.cfg
-rw-r--r-- 1 root root 1171 12月  8 13:51 zoo3.cfg
-rw-r--r-- 1 root root 1171 12月  8 13:51 zoo4.cfg
-rw-r--r-- 1 root root  935 12月  7 12:59 zoo.cfg

Zookeeper IP

10.0.0.9

集群建立失败–原因未知

SpringCloud整合视频进度:https://www.bilibili.com/video/BV18E411x7eT?p=28

Zookeeper视频进度:https://www.bilibili.com/video/BV1Ph411n7Ep?p=26&spm_id_from=pageDriver

待以后解决2021/12/08