openEuler Linux 部署 Zookeeper 集群

openEuler Linux 部署 Zookeeper 集群

openEuler Linux 部署 Zookeeper 集群

主机名	IP地址
spark01	192.168.28.11
spark02	192.168.28.12
spark03	192.168.28.13

升级操作系统和软件

yum -y update 

升级后建议重启

安装常⽤软件

 yum -y install gcc gcc-c++ autoconf  automake cmake make rsync vim man zip unzip net-tools  zlib zlib-devel openssl openssl-devel pcre-devel tcpdump  lrzsz tar wget 

修改主机名

hostnamectl set-hostname spark01
# 或者
vim /etc/hostname 
spark01 

Reboot

修改IP地址

vim /etc/sysconfig/network-scripts/ifcfgens160 

⽹卡 配置⽂件示例

TYPE=Ethernet 
PROXY_METHOD=none
BROWSER_ONLY=no 
BOOTPROTO=none 
DEFROUTE=yes 
IPV4_FAILURE_FATAL=no
IPV6INIT=yes IPV6_AUTOCONF=yes 
IPV6_DEFROUTE=yes 
IPV6_FAILURE_FATAL=no 
NAME=ens160 
UUID=943779e9-249c-44bb-b272-d49ea5831ed4 
DEVICE=ens160 
ONBOOT=yes 
IPADDR=192.168.226.11 
PREFIX=24 
GATEWAY=192.168.226.2 
DNS1=192.168.226.2

关闭防⽕墙

systemctl stop firewalld 
systemctl disable firewalld 

vim /etc/selinux/config  

# This file controls the state of SELinux on the system. 
# SELINUX= can take one of these three values: 
# 	enforcing - SELinux security policy is enforced. 
# 	permissive - SELinux prints warnings instead of  enforcing. 
# 	disabled - No SELinux policy is loaded. 
# SELINUX=enforcing 
# SELINUXTYPE= can take one of three two values: 
# 	targeted - Targeted processes are protected, 
# 	minimum - Modification of targeted policy. Only  selected processes are protected.  
# 	mls - Multi Level Security protection. 
# SELINUXTYPE=targeted 

SELINUX=disabled 

执⾏这条命令

setenforce 0 

或者

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
setenforce 0

创建软件安装目录并上传软件，配置环境变量

mkdir -p /opt/soft
cd /opt/soft
# 上传jdk zookeeper
tar -zxvf jdk-8u361-linux-x64.tar.gz
mv jdk1.8.0_361 jdk8
tar -zxvf apache-zookeeper-3.7.1-bin.tar.gz
mv apache-zookeeper-3.7.1-bin zookeeper

vim /etc/profile
	
export JAVA_HOME=/opt/soft/jdk8
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export ZOOKEEPER_HOME=/opt/soft/zookeeper

export PATH=$PATH:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin

编辑完成后使用source命令使文件/etc/profile生效执行以下命令
source /etc/profile
检查环境变量
printenv

修改域名映射

vim /etc/hosts
192.168.226.11 spark01
192.168.226.12 spark02
192.168.226.13 spark03

修改后建议重启

配置ssh免密钥登录

创建本地秘钥并将公共秘钥写入认证文件

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
# 或者
ssh-copy-id spark01
ssh-copy-id spark02
ssh-copy-id spark03
scp -rv  ~/.ssh root@spark02:~/
scp -rv  ~/.ssh root@spark03:~/

# 远程登录自己
ssh spark01
# Are you sure you want to continue connecting (yes/no)? 此处输入yes
# 登录成功后exit或者logout返回
exit

拷贝配置文件分发到其它服务器或者使用脚本分发

scp -v  ~/.bashrc root@spark02:~/
scp -v  ~/.bashrc root@spark03:~/
scp -rv /opt/soft/hadoop3/etc/hadoop/* root@spark02:/opt/soft/hadoop3/etc/hadoop/
scp -rv /opt/soft/hadoop3/etc/hadoop/* root@spark03:/opt/soft/hadoop3/etc/hadoop/

在各服务器上使环境变量生效

编辑配置文件

cd /opt/soft/zookeeper/conf
vim zoo.cfg

# 心跳单位，2s
tickTime=2000
# zookeeper初始化的同步超时时间，10个心跳单位，也即20s
initLimit=10
# 普通同步：发送一个请求并得到响应的超时时间，5个心跳单位也即10s
syncLimit=5
# 内存快照数据的存储位置
dataDir=/home/zookeeper/data
# 事务日志的存储位置
dataLogDir=/home/zookeeper/datalog
# 当前zookeeper节点的端口 
clientPort=2181
# 单个客户端到集群中单个节点的并发连接数，通过ip判断是否同一个客户端，默认60
maxClientCnxns=1000
# 保留7个内存快照文件在dataDir中，默认保留3个
autopurge.snapRetainCount=7
# 清除快照的定时任务，默认1小时，如果设置为0，标识关闭清除任务
autopurge.purgeInterval=1
#允许客户端连接设置的最小超时时间，默认2个心跳单位
minSessionTimeout=4000
#允许客户端连接设置的最大超时时间，默认是20个心跳单位，也即40s，
maxSessionTimeout=40000
#zookeeper 3.5.5启动默认会把AdminService服务启动，这个服务默认是8080端口
admin.serverPort=9001
#集群地址配置
server.1=spark01:2888:3888
server.2=spark02:2888:3888
server.3=spark03:2888:3888

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/zookeeper/data
dataLogDir=/home/zookeeper/datalog 
clientPort=2181
maxClientCnxns=1000
autopurge.snapRetainCount=7
autopurge.purgeInterval=1
minSessionTimeout=4000
maxSessionTimeout=40000
admin.serverPort=9001
server.1=spark01:2888:3888
server.2=spark02:2888:3888
server.3=spark03:2888:3888

保存后根据配置文件创建目录

mkdir -p /home/zookeeper/data
mkdir -p /home/zookeeper/datalog

echo 1 > /home/zookeeper/data/myid
more /home/zookeeper/data/myid

编写zookeeper开机启动脚本

在/etc/systemd/system/文件夹下创建一个启动脚本zookeeper.service

cd /etc/systemd/system
vim zookeeper.service

内容如下：

[Unit]
Description=zookeeper
After=syslog.target network.target

[Service]
Type=forking
# 指定zookeeper 日志文件路径，也可以在zkServer.sh 中定义
Environment=ZOO_LOG_DIR=/home/zookeeper/datalog
# 指定JDK路径，也可以在zkServer.sh 中定义
Environment=JAVA_HOME=/opt/soft/jdk8
ExecStart=/opt/soft/zookeeper/bin/zkServer.sh start
ExecStop=/opt/soft/zookeeper/bin/zkServer.sh stop
Restart=always
User=root
Group=root

[Install]
WantedBy=multi-user.target

粘贴如下：

[Unit]
Description=zookeeper
After=syslog.target network.target

[Service]
Type=forking
Environment=ZOO_LOG_DIR=/home/zookeeper/datalog
Environment=JAVA_HOME=/opt/soft/jdk8
ExecStart=/opt/soft/zookeeper/bin/zkServer.sh start
ExecStop=/opt/soft/zookeeper/bin/zkServer.sh stop
Restart=always
User=root
Group=root

[Install]
WantedBy=multi-user.target

# 重新加载服务
systemctl daemon-reload

# 启动zookeeper
systemctl start zookeeper

# 开机自启动
systemctl enable zookeeper

# 查看zookeeper状态
systemctl status zookeeper

systemctl daemon-reload
systemctl start zookeeper
systemctl enable zookeeper
systemctl status zookeeper

其它节点按照上面的配置再配置一遍后修改对应的myid