Etcd循序渐进
前言
规划
1.主机信息
| 节点名称 | 节点IP | 软件版本 | OS版本 |
|---|---|---|---|
| server50 | 192.168.4.50 | etcd | coentos 4.19 |
| server51 | 192.168.4.51 | etcd | coentos 4.19 |
| server52 | 192.168.4.52 | etcd | coentos 4.19 |
2.软件版本
- etcd：v3.5.6
地址：https://github.com/etcd-io/etcd/releases
部署
初始化
1、关闭防火墙和selinux
systemctl stop firewalld
setenforce 0
sed -i 's/=enforcing/=disabled/g' /etc/selinux/config
2、所有节点同步时间
rpm -ivh http://mirrors.wlnmp.com/centos/wlnmp-release-centos.noarch.rpm
yum install ntpdate -y
所有节点同步时间。时间同步配置如下：
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
echo 'Asia/Shanghai' >/etc/timezone
ntpdate time2.aliyun.com
# 加入到crontab
*/5 * * * * /usr/sbin/ntpdate time2.aliyun.com
3、所有节点配置limit
ulimit -SHn 65535
vim /etc/security/limits.conf
# 末尾添加如下内容
* soft nofile 65536
* hard nofile 131072
* soft nproc 65535
* hard nproc 655350
* soft memlock unlimited
* hard memlock unlimited
新建目录
mkdir /data/app/etcd/{data,conf} -p
下载安装
wget https://github.com/etcd-io/etcd/releases/download/v3.5.6/etcd-v3.5.6-linux-amd64.tar.gz
tar -xvf etcd-v3.5.6-linux-amd64.tar.gz
cd etcd-v3.5.6-linux-amd64
cp etcd etcdctl /usr/bin/
etcd配置文件
提示：vi /data/app/etcd/conf/etcd.conf.yml
1、节点一
name: etcd1
data-dir: /data/app/etcd/data
listen-client-urls: http://192.168.4.50:2379
advertise-client-urls: http://192.168.4.50:2379
listen-peer-urls: http://192.168.4.50:2380
initial-advertise-peer-urls: http://192.168.4.50:2380
initial-cluster: etcd1=http://192.168.4.50:2380,etcd2=http://192.168.4.51:2380,etcd3=http://192.168.4.52:2380
initial-cluster-token: etcd-cluster-token
initial-cluster-state: new
2、节点二
name: etcd2
data-dir: /data/app/etcd/data
listen-client-urls: http://192.168.4.51:2379
advertise-client-urls: http://192.168.4.51:2379
listen-peer-urls: http://192.168.4.51:2380
initial-advertise-peer-urls: http://192.168.4.51:2380
initial-cluster: etcd1=http://192.168.4.50:2380,etcd2=http://192.168.4.51:2380,etcd3=http://192.168.4.52:2380
initial-cluster-token: etcd-cluster-token
initial-cluster-state: new
3、节点三
name: etcd3
data-dir: /data/app/etcd/data
listen-client-urls: http://192.168.4.52:2379,http://127.0.0.1:2379
advertise-client-urls: http://192.168.4.52:2379
listen-peer-urls: http://192.168.4.52:2380
initial-advertise-peer-urls: http://192.168.4.52:2380
initial-cluster: etcd1=http://192.168.4.50:2380,etcd2=http://192.168.4.51:2380,etcd3=http://192.168.4.52:2380
initial-cluster-token: etcd-cluster-token
initial-cluster-state: new
备注：
- data-dir 指定节点的数据存储目录
- listen-peer-urls 监听URL，用于与其他节点通讯
- listen-client-urls 对外提供服务的地址：比如 http://ip:2379,http://127.0.0.1:2379 ，客户端会连接到这里和 etcd 交互
- initial-advertise-peer-urls 该节点member(同伴)监听地址，这个值会告诉集群中其他节点
- initial-cluster 集群中所有节点的信息，格式为 node1=http://ip1:2380,node2=http://ip2:2380,… 。注意：这里的 node1 是节点的 --name 指定的名字；后面的 ip1:2380 是 --initial-advertise-peer-urls 指定的值
- initial-cluster-state 新建集群的时候，这个值为 new ；假如已经存在的集群，这个值为 existing
- initial-cluster-token 创建集群的 token，这个值每个集群保持唯一。这样的话，如果你要重新创建集群，即使配置和之前一样，也会再次生成新的集群和节点 uuid；否则会导致多个集群之间的冲突，造成未知的错误
- advertise-client-urls 对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点
配置systemd服务文件
vi /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
WorkingDirectory=/data/app/etcd/data
# User=etcd
ExecStart=/usr/bin/etcd --config-file=/data/app/etcd/conf/etcd.conf.yml
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
启动
systemctl daemon-reload
systemctl start etcd.service
systemctl enable etcd.service
systemctl restart etcd.service
systemctl stop etcd.service
systemctl status etcd.service
查看集群状态
[root@server50 ~]# etcdctl -w table endpoint status --cluster --endpoints=192.168.4.50:2379
+--------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+--------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| http://192.168.4.52:2379 | b96dea56371d2c55 | 3.5.6 | 20 kB | false | false | 3 | 14 | 14 | |
| http://192.168.4.50:2379 | d2b23f19d7a28c12 | 3.5.6 | 20 kB | false | false | 3 | 14 | 14 | |
| http://192.168.4.51:2379 | fc95a547ddbee4f2 | 3.5.6 | 20 kB | true | false | 3 | 14 | 14 | |
+--------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
自签证书
监控
常见命令
1.查看etcd的版本
curl -L http://127.0.0.1:2379/version
2.查看etcd暴露出来的prometheus指标
curl -L http://127.0.0.1:2379/metrics
3.查看etcd集群的成员
etcdctl --ca-file=ca.pem --cert-file=etcd.pem --key-file=etcd-key.pem member list
etcdctl member list
4.查看etcd集群的健康状态
etcdctl --ca-file=ca.pem --cert-file=etcd.pem --key-file=etcd-key.pem cluster-health
etcdctl endpoint health
5.备份etcd的整个数据目录
默认的数据目录为/var/lib/etcd/
etcdctl --ca-file=ca.pem --cert-file=etcd.pem --key-file=etcd-key.pem backup --data-dir /var/lib/etcd --backup-dir /tmp/etcd
etcdctl backup --data-dir /var/lib/etcd --backup-dir /tmp/etcd
--data-dir：指明数据目录的位置
--backup-dir：指明备份的位置
6.对etcd进行快照备份
ETCDCTL_API=3 /etc/kubernetes/bin/etcdctl snapshot save snapshot.db
Snapshot saved at snapshot.db
ETCDCTL_API=3：表示使用etcd的v3版本的API接口
注：一定要添加ETCDCTL_API=3才能正常备份；如果不添加将无法备份
7.通过快照恢复etcd集群
每个节点都要执行
ETCDCTL_API=3 etcdctl snapshot restore snapshot.db
--name {{ NODE_NAME }}
--initial-cluster {{ ETCD_NODES }}
--initial-cluster-token etcd-cluster-0
--initial-advertise-peer-urls https://{{ inventory_hostname }}:2380
--name：表示当前etcd节点的名字（非主机名）
--initial-cluster：集群中所有节点的peer访问地址；例：etcd01=https://10.10.5.217:2380 ,etcd02=https://10.10.5.219:2380,etcd03=https://10.10.5.220:2380
--initial-cluster-token：集群中各节点通信的token
--initial-advertise-peer-urls：当前节点对其它节点的通信地址
