ETCD安装与简单使用

原创文档编写不易，未经许可请勿转载。文档中有疑问的可以邮件联系我。 邮箱：yinwanit@163.com

说明

ETCD集群是一个分布式系统,使用Raft协议来维护集群内各个节点状态的一致性。etcd主机状态分为 Leader, Follower, Candidate ，当集群初始化时，每个节点都是Follower角色，然后会把自己的角色改成Candidate并发起一次选举投票，被选为主的节点角色为Leader，其他节点角色转为Follower。

当Leader主节点故障过后，Follower节点没有收到Leader的心跳数据，Follower节点会自动改变角色为Candidate状态，然后进行投票选出新的Leader。配置etcd集群，建议尽可能是奇数个节点，而不要偶数个节点。

etcd使用raft协议通信，对内使用2380端口（集群同步），对外提供服务器2379（客户端访问）。数据写入Leader中，数据读取从follower中。

该文档只演示etcd单机及集群环境安装配置、基本操作、快照及恢复。

环境说明

主机名 ip地址 操作系统版本 集群类别
etcd_s 192.168.100.50/24 Centos 7.8 单机
etcd_c01 192.168.100.51/24 Centos 7.8 集群
etcd_c02 192.168.100.52/24 Centos 7.8 集群
etcd_c03 192.168.100.53/24 Centos 7.8 集群

 

步骤概述

  1. 服务器操作系统配置
  2. etcd常用配置文件说明
  3. etcd常用命令介绍
  4. etcd单机环境安装
  5. etcd集群环境安装
  6. etcd单机环境快照及恢复
  7. etcd集群环境快照及恢复

一、服务器操作系统配置

  • 服务器操作系统设置本文不做介绍，安装etcd前需要满足以下条件，可自行百度进行相关配置。
  • 关闭selinux
  • 关闭firewalld
  • 安装bash-completion、vim
  • 完成ip地址及主机名设置、dns配置
  • 关闭swap
  • 配置yum源
  • 服务器需要联网

yum源文件

[base]
name=CentOS-$releasever – Base – mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/os/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7

#released updates
[updates]
name=CentOS-$releasever – Updates – mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/updates/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7

#additional packages that may be useful
[extras]
name=CentOS-$releasever – Extras – mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/extras/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7
View Code

二、etcd配置文件详解

etcd默认的配置文件路径为/etc/etcd.conf。

ETCD_DATA_DIR="/var/lib/etcd/cluster.etcd"                                           #设置etcd服务文件存储路径
ETCD_LISTEN_PEER_URLS="http://192.168.100.50:2380,http://localhost:2380"             #设置集群监听地址，供集群内节点同步数据用
ETCD_LISTEN_CLIENT_URLS="http://192.168.100.50:2379,http://localhost:2379"           #服务监听地址，对外客户端提供访问接口。
ETCD_NAME="etcd_c01"                                                                 #节点名称，默认未default，在多集群环境下集群内节点NAME均需要设置成唯一值。
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.100.50:2380"                        #对集群内主机通告数据同步地址，该值会同步到同集群其他节点。
ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379,http://192.168.100.50:2379"        #对外公告的本节点对客户端提供服务的地址，该地址会同步到同集群其他节点。
ETCD_INITIAL_CLUSTER="etcd_c01=http://192.168.100.50:2380，etcd_c01=http://192.168.100.50:2380，etcd_c01=http://192.168.100.50:2380"  #集群内有多少个节点，多个节点时需要都写上。
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"                                            #自定一的token，相当于节点之间同步用的口令，同集群内该值必须一致。
ETCD_INITIAL_CLUSTER_STATE="new"                                                     #设置是新集群还是已有集群，有两个“new”和“existing”可选分别代表新集群和已有集群。

三、etcd常见命令

etcd分为两个版本v2和v3，v3使用键值对存储数据。使用export  ETCDCTL_API=3切换etcd的V2和V3版本。

3.1 v2

 自行翻译

etcdctl     backup          backup an etcd directory
etcdctl     cluster-health  check the health of the etcd cluster
etcdctl     mk              make a new key with a given value
etcdctl     mkdir           make a new directory
etcdctl     rm              remove a key or a directory
etcdctl     rmdir           removes the key if it is an empty directory or a key-value pair
etcdctl     get             retrieve the value of a key
etcdctl     ls              retrieve a directory
etcdctl     set             set the value of a key
etcdctl     setdir          create a new directory or update an existing directory TTL
etcdctl     update          update an existing key with a given value
etcdctl     updatedir       update an existing directory
etcdctl     watch           watch a key for changes
etcdctl     exec-watch      watch a key for changes and exec an executable
etcdctl     member          member add, remove and list subcommands
etcdctl     user            user add, grant and revoke subcommands
etcdctl     role            role add, grant and revoke subcommands
etcdctl     auth            overall auth controls
etcdctl     help, h         Shows a list of commands or help for one command

3.2 v3

 自行翻译

etcdctl        get                     Gets the key or a range of keys
etcdctl        put                     Puts the given key into the store
etcdctl        del                     Removes the specified key or range of keys [key, range_end)
etcdctl        txn                     Txn processes all the requests in one transaction
etcdctl        compaction              Compacts the event history in etcd
etcdctl        alarm disarm            Disarms all alarms
etcdctl        alarm list              Lists all alarms
etcdctl        defrag                  Defragments the storage of the etcd members with given endpoints
etcdctl        endpoint health         Checks the healthiness of endpoints specified in `--endpoints` flag
etcdctl        endpoint status         Prints out the status of endpoints specified in `--endpoints` flag
etcdctl        endpoint hashkv         Prints the KV history hash for each endpoint in --endpoints
etcdctl        move-leader             Transfers leadership to another etcd cluster member.
etcdctl        watch                   Watches events stream on keys or prefixes
etcdctl        version                 Prints the version of etcdctl
etcdctl        lease grant             Creates leases
etcdctl        lease revoke            Revokes leases
etcdctl        lease timetolive        Get lease information
etcdctl        lease list              List all active leases
etcdctl        lease keep-alive        Keeps leases alive (renew)
etcdctl        member add              Adds a member into the cluster
etcdctl        member remove           Removes a member from the cluster
etcdctl        member update           Updates a member in the cluster
etcdctl        member list             Lists all members in the cluster
etcdctl        snapshot save           Stores an etcd node backend snapshot to a given file
etcdctl        snapshot restore        Restores an etcd member snapshot to an etcd directory
etcdctl        snapshot status         Gets backend snapshot status of a given file
etcdctl        make-mirror             Makes a mirror at the destination etcd cluster
etcdctl        migrate                 Migrates keys in a v2 store to a mvcc store
etcdctl        lock                    Acquires a named lock
etcdctl        elect                   Observes and participates in leader election
etcdctl        auth enable             Enables authentication
etcdctl        auth disable            Disables authentication
etcdctl        user add                Adds a new user
etcdctl        user delete             Deletes a user
etcdctl        user get                Gets detailed information of a user
etcdctl        user list               Lists all users
etcdctl        user passwd             Changes password of user
etcdctl        user grant-role         Grants a role to a user
etcdctl        user revoke-role        Revokes a role from a user
etcdctl        role add                Adds a new role
etcdctl        role delete             Deletes a role
etcdctl        role get                Gets detailed information of a role
etcdctl        role list               Lists all roles
etcdctl        role grant-permission   Grants a key to a role
etcdctl        role revoke-permission  Revokes a key from a role
etcdctl        check perf              Check the performance of the etcd cluster
etcdctl        help                    Help about any command

四、单机环境部署

4.1 安装etcd

# yum  install etcd -y

4.2 配置etcd

etcd默认配置文件路径/etc/etcd/etcd.conf。执行以下命令，请根据实际情况替换红色字体部分内容

# cat >> /etc/etcd/etcd.conf << EOF
ETCD_DATA_DIR="/var/lib/etcd/etcd-50.etcd"
ETCD_LISTEN_PEER_URLS="http://192.168.100.50:2380,http://localhost:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.100.50:2379,http://localhost:2379"
ETCD_NAME="etcd-50"
ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379,http://192.168.100.50:2379"
EOF

确认etcd配置文件配置正常

# cat /etc/etcd/etcd.conf

启动etcd服务并设置开机启动

# systemctl  enable etcd --now
# systemctl status etcd

4.3 测试etcd

查看当前etcd服务状态。

# etcdctl  member list

 使用V2版本测试

# #查看集群状态
# etcdctl  cluster-health
# #新建一个目录
# etcdctl  mkdir test1_dir
# #查看目录是否创建完成
# etcdctl ls

使用V3版本测试

使用V3版本前需要设置以下环境变量

# export  ETCDCTL_API=3
# etcdctl  put test_value 888888
# etcdctl  get test_value

五、集群环境部署

 5.1 安装etcd软件

三个节点上均需要进行etcd软件安装。

# yum  install etcd -y

5.2 配置etcd

etcd默认配置文件路径/etc/etcd/etcd.conf。执行以下命令，请根据实际情况替换红色字体部分内容

etcd_01

# cat >> /etc/etcd/etcd.conf << EOF
ETCD_DATA_DIR="/var/lib/etcd/cluster.etcd"
ETCD_LISTEN_PEER_URLS="http://192.168.100.51:2380,http://localhost:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.100.51:2379,http://localhost:2379"
ETCD_NAME="etcd_c01"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.100.51:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379,http://192.168.100.51:2379"
ETCD_INITIAL_CLUSTER="etcd_c01=http://192.168.100.51:2380,etcd_c02=http://192.168.100.52:2380,etcd_c03=http://192.168.100.53:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

etcd_02

# cat >> /etc/etcd/etcd.conf << EOF
ETCD_DATA_DIR="/var/lib/etcd/cluster.etcd"
ETCD_LISTEN_PEER_URLS="http://192.168.100.52:2380,http://localhost:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.100.52:2379,http://localhost:2379"
ETCD_NAME="etcd_c02"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.100.52:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379,http://192.168.100.52:2379"
ETCD_INITIAL_CLUSTER="etcd_c01=http://192.168.100.51:2380,etcd_c02=http://192.168.100.52:2380,etcd_c03=http://192.168.100.53:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

etcd_03

# cat >> /etc/etcd/etcd.conf << EOF
ETCD_DATA_DIR="/var/lib/etcd/cluster.etcd"
ETCD_LISTEN_PEER_URLS="http://192.168.100.53:2380,http://localhost:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.100.53:2379,http://localhost:2379"
ETCD_NAME="etcd_c03"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.100.53:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379,http://192.168.100.53:2379"
ETCD_INITIAL_CLUSTER="etcd_c01=http://192.168.100.51:2380,etcd_c02=http://192.168.100.52:2380,etcd_c03=http://192.168.100.53:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

5.3 启动etcd集群服务

对于etcd集群需要同时启动两个以上的节点整体服务才能才能启动。所有节点均需要执行。

# systemctl enable etcd --now

查看集群状态

# etcdctl member list

 最后一列isLeader表示当前节点是否为Leader，V2版本会显示，V3版本没有isLeaser列

5.4 测试etcd

在任意一台机器上创建数据，在其他同集群机器上进行查看删除。

先在节点一上操作创建两个key

# export ETCDCTL_API=3
# etcdctl put cluster_test 51_52_53
# etcdctl put cluster_test2 lvan

节点二上操作查看可以获取到节点一上创建的数据

# etcdctl get cluster_test
# etcdctl get cluster_test2

节点三上删除一个key

# export ETCDCTL_API=3
# etcdctl del cluster_test2

测试结果，任意节点操作过后其他节点能够正常获取到数据变化。

 

六、etcd单机环境快照及恢复

 etcd还原数据时，一定要确定etcd是关闭的。只有ETCD V3版本才有快照功能。

 1、设置etcd版本

# export ETCDCTL_API=3

2、对当前数据进行快照，备份当前etcd中的数据为snap1.data。相当于做快照。

# etcdctl snapshot  save snap1.data

3、停止etcd集群

# systemctl stop etcd

4、删除原有数据(cat /etc/etcd/etcd.conf  | grep ETCD_DATA_DIR)

# rm -rf /var/lib/etcd/cluster.etcd

5、恢复数据

需要以下信息（集群未关闭时可用 etcdctl member list 命令查看）：
.本节点的etcd名字 --name
.数据恢复到哪里--etcd存储数据的位置 --data-dir
.初始化集群所用的节点 --initial-cluster

# etcdctl snapshot restore snap1.data  --name=default   --data-dir=/var/lib/etcd/cluster.etcd  --initial-cluster="default=http://192.168.100.50:2380"

6、修改该恢复出来的数据的所有者和所有组

# chown  -R  etcd.etcd  /var/lib/etcd/*

7、启动etcd集群

# systemctl start  etcd

8、查看测试数据是否恢复

# export ETCDCTL_API=3
# etcdctl get aa

七、etcd单机环境快照及恢复

 etcd集群环境进行快照恢复时，快照在任意节点创建就行，恢复时所有etcd同集群节点都要有备份的快照文件。

 

1、执行创建快照

任意节点上执行快照就行，生成的文件复制到其他节点上。文章使用第一个节点创建快照。

# export ETCDCTL_API=3
# etcdctl snapshot save snap2.data
# scp snap2.data root@192.168.100.52:/root/
# scp snap2.data root@192.168.100.53:/root/

2、获取etcd服务信息

获取当前etcd集群服务信息

# etcdctl  member list
3963c7393c1f01db, started, etcd_c03, http://192.168.100.53:2380, http://192.168.100.53:2379,http://localhost:2379
7d382b8093b82b47, started, etcd_c01, http://192.168.100.51:2380, http://192.168.100.51:2379,http://localhost:2379
b09a930d31068447, started, etcd_c02, http://192.168.100.52:2380, http://192.168.100.52:2379,http://localhost:2379

3、停止etcd服务

统计群所有节点上上都要停止。

# systemctl stop etcd

4、删除etcd数据文件

通过/etc/etcd/etcd.conf文件中的ETCD_DATA_DIR行找到etcd的数据文件存放地址，然后删除这个文件夹，所有节点上都需要操作。

# cat /etc/etcd/etcd.conf | grep -i ETCD_DATA_DIR | awk -F'"' '{print $2}'
# rm -rf  /var/lib/etcd/cluster.etcd

5、初始化集群

三个节点都要初始化集群，每个节点初始化时peer字段ip地址和name字段不同，请注意甄别。

数据文件恢复过后还需要对恢复的文件修改权限。

第一个节点，ip地址192.168.100.51

# export ETCDCTL_API=3
# etcdctl snapshot restore snap2.data --data-dir=/var/lib/etcd/cluster.etcd --initial-cluster=etcd_c01=http://192.168.100.51:2380,etcd_c02=http://192.168.100.52:2380,etcd_c03=http://192.168.100.53:2380 --initial-advertise-peer-urls http://192.168.100.51:2380 --name=etcd_c01
# chown -R etcd.etcd /var/lib/etcd/*

第二个节点，ip地址192.168.100.52

# export ETCDCTL_API=3
# etcdctl snapshot restore snap2.data --data-dir=/var/lib/etcd/cluster.etcd --initial-cluster=etcd_c01=http://192.168.100.51:2380,etcd_c02=http://192.168.100.52:2380,etcd_c03=http://192.168.100.53:2380 --initial-advertise-peer-urls http://192.168.100.52:2380 --name=etcd_c02
# chown -R etcd.etcd /var/lib/etcd/*

第三个节点，ip地址192.168.100.53

# export ETCDCTL_API=3
# etcdctl snapshot restore snap2.data --data-dir=/var/lib/etcd/cluster.etcd --initial-cluster=etcd_c01=http://192.168.100.51:2380,etcd_c02=http://192.168.100.52:2380,etcd_c03=http://192.168.100.53:2380 --initial-advertise-peer-urls http://192.168.100.53:2380 --name=etcd_c03
# chown -R etcd.etcd /var/lib/etcd/*

6、启动etcd并测试

三个节点上均需要启动etcd服务。

# systemctl start etcd

测试数据是否能够获取到

# export ETCDCTL_API=3
# etcdctl get test2
# etcdctl get test3
# etcdctl get test4

 

posted @ 2023-08-01 23:07  飞翔的小胖猪  阅读(1300)  评论(0)  编辑  收藏  举报