etcd 分布式集群搭建和使用

目录

• 1.多节点服务集群搭建
  • 启动节点1
  • 启动节点2
  • 启动节点3
• 2.使用
  • 查看节点信息
  • 集群状态
  • 键值对 读写
  • 分布式锁
• 3.参考

上篇文章etcd 入门 中已经介绍了etcd基本功能，单节点服务的安装和使用。

本文将介绍etcd 分布式多节点的安装、配置和使用。

由于资源限制，本文以单机多节点服务为例进行阐述。

在实际应用场景中，为了高可用和容灾，是需要多机器安装的。

1.多节点服务集群搭建

操作系统使用的 centos 6.8。

默认情况下，端口2379用于提供HTTP API服务，端口2380用于节点间通信。

在单机情况下，多节点服务需要使用不同端口。

三节点集群信息如下：

name	ip	port
etcd-01	127.0.0.1	2379,2380
etcd-02	127.0.0.1	2479,2480
etcd-03	127.0.0.1	2579,2580

启动节点1

定义启动脚本start_etcd1.sh

TOKEN=token-01
CLUSTER_STATE=new
NAME_1=etcd-01
NAME_2=etcd-02
NAME_3=etcd-03
HOST_1=127.0.0.1
HOST_2=127.0.0.1
HOST_3=127.0.0.1
PORT_API_1=2379
PORT_PEER_1=2380
PORT_API_2=2479
PORT_PEER_2=2480
PORT_API_3=2579
PORT_PEER_3=2580

CLUSTER=${NAME_1}=http://${HOST_1}:${PORT_PEER_1},${NAME_2}=http://${HOST_2}:${PORT_PEER_2},${NAME_3}=http://${HOST_3}:${PORT_PEER_3}

# For every machine
THIS_NAME=${NAME_1}
THIS_IP=${HOST_1}
THIS_PORT_API=${PORT_API_1}
THIS_PORT_PEER=${PORT_PEER_1}

./etcd --data-dir=data.${THIS_NAME} --name ${THIS_NAME} \
	--initial-advertise-peer-urls http://${THIS_IP}:${THIS_PORT_PEER} --listen-peer-urls http://${THIS_IP}:${THIS_PORT_PEER} \
	--advertise-client-urls http://${THIS_IP}:${THIS_PORT_API} --listen-client-urls http://${THIS_IP}:${THIS_PORT_API} \
	--initial-cluster ${CLUSTER} \
	--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

启动

bash start_etcd2.sh

启动节点2

定义启动脚本start_etcd2.sh,
只需将启动脚本start_etcd1.sh中的配置项修改：

THIS_NAME=${NAME_2}
THIS_IP=${HOST_2}
THIS_PORT_API=${PORT_API_2}
THIS_PORT_PEER=${PORT_PEER_2}

启动

bash start_etcd2.sh

启动节点3

定义启动脚本start_etcd3.sh，
只需将启动脚本start_etcd1.sh中的配置项修改：

THIS_NAME=${NAME_3}
THIS_IP=${HOST_3}
THIS_PORT_API=${PORT_API_3}
THIS_PORT_PEER=${PORT_PEER_3}

启动

bash start_etcd3.sh

2.使用

查看节点信息

export ETCDCTL_API=3
HOST_1=127.0.0.1
HOST_2=127.0.0.1
HOST_3=127.0.0.1
PORT_API_1=2379
PORT_API_2=2479
PORT_API_3=2579


ENDPOINTS=$HOST_1:${PORT_API_1},$HOST_2:${PORT_API_2},$HOST_3:${PORT_API_3}

./etcdctl --endpoints=$ENDPOINTS member list

$ ./etcdctl --endpoints=$ENDPOINTS member list
264ae6bc59e99892, started, etcd-01, http://127.0.0.1:2380, http://127.0.0.1:2379, false
dbafe5ad6b652eda, started, etcd-02, http://127.0.0.1:2480, http://127.0.0.1:2479, false
f570ae41f524bdcb, started, etcd-03, http://127.0.0.1:2580, http://127.0.0.1:2579, false

或者

$ ./etcdctl --endpoints=$ENDPOINTS --write-out=table member list
+------------------+---------+---------+-----------------------+-----------------------+------------+
|        ID        | STATUS  |  NAME   |      PEER ADDRS       |     CLIENT ADDRS      | IS LEARNER |
+------------------+---------+---------+-----------------------+-----------------------+------------+
| 264ae6bc59e99892 | started | etcd-01 | http://127.0.0.1:2380 | http://127.0.0.1:2379 |      false |
| dbafe5ad6b652eda | started | etcd-02 | http://127.0.0.1:2480 | http://127.0.0.1:2479 |      false |
| f570ae41f524bdcb | started | etcd-03 | http://127.0.0.1:2580 | http://127.0.0.1:2579 |      false |

集群状态

$ ./etcdctl --endpoints=$ENDPOINTS endpoint status
127.0.0.1:2379, 264ae6bc59e99892, 3.4.13, 20 kB, false, false, 8, 10, 10, 
127.0.0.1:2479, dbafe5ad6b652eda, 3.4.13, 33 kB, true, false, 8, 10, 10, 
127.0.0.1:2579, f570ae41f524bdcb, 3.4.13, 20 kB, false, false, 8, 10, 10, 

$ ./etcdctl --endpoints=$ENDPOINTS --write-out=table  endpoint status
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|    ENDPOINT    |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 127.0.0.1:2379 | 264ae6bc59e99892 |  3.4.13 |   20 kB |     false |      false |         8 |         10 |                 10 |        |
| 127.0.0.1:2479 | dbafe5ad6b652eda |  3.4.13 |   33 kB |      true |      false |         8 |         10 |                 10 |        |
| 127.0.0.1:2579 | f570ae41f524bdcb |  3.4.13 |   20 kB |     false |      false |         8 |         10 |                 10 |        |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

可以看到，127.0.0.1:2479 这个节点是leader。

查看节点健康状况

$ ./etcdctl --endpoints=$ENDPOINTS endpoint health
127.0.0.1:2379 is healthy: successfully committed proposal: took = 5.843376ms
127.0.0.1:2479 is healthy: successfully committed proposal: took = 4.92724ms
127.0.0.1:2579 is healthy: successfully committed proposal: took = 7.661623ms

$ ./etcdctl --endpoints=$ENDPOINTS --write-out=table  endpoint health
+----------------+--------+------------+-------+
|    ENDPOINT    | HEALTH |    TOOK    | ERROR |
+----------------+--------+------------+-------+
| 127.0.0.1:2379 |   true | 8.470348ms |       |
| 127.0.0.1:2479 |   true | 4.540441ms |       |
| 127.0.0.1:2579 |   true | 8.666543ms |       |
+----------------+--------+------------+-------+

键值对 读写

$ ./etcdctl --endpoints=$ENDPOINTS put foo "Hello World"
OK
$ ./etcdctl --endpoints=$ENDPOINTS get foo
foo
Hello World

分布式锁

在终端1中获取锁

$  ./etcdctl --endpoints=$ENDPOINTS lock mutex1
mutex1/3dcb7516bf48a904

在终端2中获取同一把锁，会被卡住

$  ./etcdctl --endpoints=$ENDPOINTS lock mutex1

此时，停止终端1的操作，释放锁，终端2才可以获取到锁:

$ ./etcdctl --endpoints=$ENDPOINTS lock mutex1
mutex1/3dcb7516bf48a907

3.参考

github etcd 官方文档