linux运维、架构之路-Zookeeper集群部署
一、Zookeeper介绍
- 是一个针对大型分布式系统的可靠协调系统；
- 提供的功能包括：配置维护、名字服务、分布式同步、组服务等；
- 目标就是封装好复杂易出错的关键职务，将简单易用的接口和性能高效、功能稳定的系统提供给用户；
- Zookeeper已经成为Hadoop生态系统中的基础组件。
架构图：
二、Zookeeper特点
- 最终一致性：为客户端展示同一视图，这是Zookeeper最重要的性能；
- 可靠性：如果消息被一台服务器接受，那么它将被所有的服务器接受；
- 原子性：更新只能成功或失败，没有中间状态。
三、用到Zookeeper的一些开源软件
- HDFS
- YARN
- Storm
- HBase
- Flume
- Dubbo
- metaq
四、Zookeeper集群部署
Zookeeper官网文档：http://zookeeper.apache.org/doc/current/zookeeperAdmin.html Zookeeper下载地址：http://archive.apache.org/dist/zookeeper/
1、服务器规划
|
主机名 |
软件 |
JAVA版本 |
系统 |
IP地址 |
|
zk-1 |
zookeeper-3.4.10.tar.gz |
jdk1.8.0_181 |
CentOS7.5 |
192.168.56.11 |
|
zk-2 |
zookeeper-3.4.10.tar.gz |
jdk1.8.0_181 |
CentOS7.5 |
192.168.56.12 |
|
zk-3 |
zookeeper-3.4.10.tar.gz |
jdk1.8.0_181 |
CentOS7.5 |
192.168.56.13 |
2、系统基础配置
#关闭防火墙# systemctl disable firewalld.service systemctl stop firewalld.service systemctl disable NetworkManager #关闭selinux# sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config setenforce 0
3、安装JDK环境
tar xf jdk-8u161-linux-x64.tar.gz -C /usr/local/ ln -s /usr/local/jdk1.8.0_181 /usr/local/jdk ln -s /usr/local/jdk/bin/java /usr/bin/java echo 'export JAVA_HOME=/usr/local/jdk export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar export PATH=$JAVA_HOME/bin:$PATH' >>~/.bash_profile source ~/.bash_profile
4、安装zookeeper
mkdir /app #所有节点都创建软件安装目录 tar xf zookeeper-3.4.10.tar.gz -C /app/ ln -s /app/zookeeper-3.4.10/ /app/zookeeper
5、修改配置文件
#创建快照日志存放目录： mkdir /app/zookeeper/dataDir #创建事务日志存放目录： mkdir /app/zookeeper/dataLogDir egrep -v "#|^$" /app/zookeeper/conf/zoo.cfg tickTime=2000 initLimit=10 syncLimit=5 dataDir=/app/zookeeper/dataDir dataLogDir=/app/zookeeper/dataLogDir clientPort=2181 server.1=192.168.56.11:2182:2183 server.2=192.168.56.12:2182:2183 server.3=192.168.56.13:2182:2183 maxClientCnxns=0 autopurge.snapRetainCount=3 autopurge.purgeInterval=1
6、分发.bash_profile JDK && zookeeper 文件
scp ~/.bash_profile 192.168.56.12:~/ scp ~/.bash_profile 192.168.56.13:~/ scp -rp /usr/local/jdk1.8.0_181/ 192.168.56.12:/usr/local/ scp -rp /usr/local/jdk1.8.0_181/ 192.168.56.13:/usr/local/ scp -rp /app/zookeeper-3.4.10 192.168.56.12:/app/ scp -rp /app/zookeeper-3.4.10 192.168.56.13:/app/ 创建JDK和zookeeper软链接方便以后升级 ln -s /usr/local/jdk1.8.0_181 /usr/local/jdk ln -s /usr/local/jdk/bin/java /usr/bin/java ln -s /app/zookeeper-3.4.10/ /app/zookeeper
7、创建一个myid文件
echo "1" > /app/zookeeper/dataDir/myid echo "2" > /app/zookeeper/dataDir/myid echo "3" > /app/zookeeper/dataDir/myid #里面内容为一个数字，用来标识当前主机，conf/zoo.cfg文件中配置的server.X中X为什么数字，则myid文件中就输入这个数字
8、启动和关闭zookeeper服务
#放在后台运行：nohup /app/zookeeper/bin/zkServer.sh start >/dev/null 2>&1 & #停止： /app/zookeeper/bin/zkServer.sh stop #查看进程 [root@zk-1 ~]# jps 40772 QuorumPeerMain 40856 Jps
9、测试zookeeper集群
①查看集群状态
/app/zookeeper/bin/zkServer.sh status #分别在集群三个节点执行,集群中一个是Leader，其余的两个结点是Follower ZooKeeper JMX enabled by default Using config: /app/zookeeper/bin/../conf/zoo.cfg Mode: follower ZooKeeper JMX enabled by default Using config: /app/zookeeper/bin/../conf/zoo.cfg Mode: leader ZooKeeper JMX enabled by default Using config: /app/zookeeper/bin/../conf/zoo.cfg Mode: follower
②测试连接zookeeper集群
[root@zk-3 bin]# /app/zookeeper/bin/zkCli.sh -server 192.168.56.11:2181 Connecting to 192.168.56.11:2181 2020-04-22 15:01:24,055 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT 2020-04-22 15:01:24,059 [myid:] - INFO [main:Environment@100] - Client environment:host.name=zk-3 2020-04-22 15:01:24,059 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_181 2020-04-22 15:01:24,062 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2020-04-22 15:01:24,062 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/local/jdk1.8.0_181/jre 2020-04-22 15:01:24,062 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/app/zookeeper/bin/../build/classes:/app/zookeeper/bin/../build/lib/*.jar:/app/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/app/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/app/zookeeper/bin/../lib/netty-3.10.5.Final.jar:/app/zookeeper/bin/../lib/log4j-1.2.16.jar:/app/zookeeper/bin/../lib/jline-0.9.94.jar:/app/zookeeper/bin/../zookeeper-3.4.10.jar:/app/zookeeper/bin/../src/java/lib/*.jar:/app/zookeeper/bin/../conf:.:/usr/local/jdk/lib/dt.jar:/usr/local/jdk/lib/tools.jar 2020-04-22 15:01:24,062 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2020-04-22 15:01:24,062 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2020-04-22 15:01:24,062 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2020-04-22 15:01:24,063 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2020-04-22 15:01:24,063 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2020-04-22 15:01:24,063 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-862.el7.x86_64 2020-04-22 15:01:24,063 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2020-04-22 15:01:24,063 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2020-04-22 15:01:24,063 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/app/zookeeper-3.4.10/bin 2020-04-22 15:01:24,064 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=192.168.56.11:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@42110406 Welcome to ZooKeeper! 2020-04-22 15:01:24,085 [myid:] - INFO [main-SendThread(192.168.56.11:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server 192.168.56.11/192.168.56.11:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2020-04-22 15:01:24,212 [myid:] - INFO [main-SendThread(192.168.56.11:2181):ClientCnxn$SendThread@876] - Socket connection established to 192.168.56.11/192.168.56.11:2181, initiating session 2020-04-22 15:01:24,239 [myid:] - INFO [main-SendThread(192.168.56.11:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server 192.168.56.11/192.168.56.11:2181, sessionid = 0x171a0ae23360000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: 192.168.56.11:2181(CONNECTED) 0]
10、脚本定期清理zk快照和日志文件
#!/bin/bash dataDir=/app/zookeeper/dataDir/version-2 dataLogDir=/app/zookeeper/dataLogDir/version-2 # Reserved 5 files. COUNT=5 ls -t $dataDir/snapshot.* | tail -n +$[$COUNT+1] | xargs rm -f ls -t $dataLogDir/log.* | tail -n +$[$COUNT+1] | xargs rm -f ##每周日的0点0分执行： 0 0 * * 0 /app/scripts/clean_zklog.sh
五、zookeeper单机版配置
1、3.5版本以上需下载带bin的包，不然启动报错
apache-zookeeper-3.5.5-bin.tar.gz
2、Zookeeper 3.5启动时 8080端口被占用
#这是Zookeeper AdminServer，默认使用8080端口 admin.serverPort=8081
3、单机配置文件
[root@CTC-CaaS-Master01 ~]# egrep -v "#|^$" /data/zk/zookeeper-3.5.5/conf/zoo.cfg tickTime=2000 initLimit=10 syncLimit=5 dataDir=/data/zk/zookeeper-3.5.5/dataDir dataLogDir=/data/zk/zookeeper-3.5.5/dataLogDir clientPort=2181 maxClientCnxns=0 autopurge.snapRetainCount=3 autopurge.purgeInterval=1 admin.serverPort=8081
成功最有效的方法就是向有经验的人学习！
