etcd集群搭建
etcd介绍，以及适用场景，参考：http://www.infoq.com/cn/articles/etcd-interpretation-application-scenario-implement-principle
etcd的项目：https://github.com/coreos/etcd/releases/tag/v3.2.10 并且下载最新版本etcd
环境：
master：192.168.101.14，node1：192.168.101.15，node2：192.168.101.19
1、在三个节点上下载etcd-v3.2.10-linux-amd64.tar.gz，并提供etcd命令，具体操作如下
[root@docker ~]# tar xf etcd-v3.2.10-linux-amd64.tar.gz [root@docker ~]# cd etcd-v3.2.10-linux-amd64 [root@docker etcd-v3.2.10-linux-amd64]# cp etcd etcdctl /usr/local/bin/
2、在三个节点上创建etcd数据目录：
# mkdir -p /var/lib/etcd
3、在每个节点上创建etcd的systemd unit文件/usr/lib/systemd/system/etcd.service
master节点： [root@docker ~]# cat /usr/lib/systemd/system/etcd.service [Unit] Description=etcd server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify WorkingDirectory=/var/lib/etcd/ EnvironmentFile=-/etc/etcd/etcd.conf ExecStart=/usr/local/bin/etcd --name master --initial-advertise-peer-urls http://192.168.101.14:2380 --listen-peer-urls http://192.168.101.14:2380 --listen-client-urls http://192.168.101.14:2379,http://127.0.0.1:2379 --advertise-client-urls http://192.168.101.14:2379 --initial-cluster-token etcd-cluster-1 --initial-cluster master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380,node2=http://192.168.101.19:2380 --initial-cluster-state new --data-dir=/var/lib/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target
node1节点： [root@localhost ~]# cat /usr/lib/systemd/system/etcd.service [Unit] Description=etcd server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify WorkingDirectory=/var/lib/etcd/ EnvironmentFile=-/etc/etcd/etcd.conf ExecStart=/usr/local/bin/etcd --name node1 --initial-advertise-peer-urls http://192.168.101.15:2380 --listen-peer-urls http://192.168.101.15:2380 --listen-client-urls http://192.168.101.15:2379,http://127.0.0.1:2379 --advertise-client-urls http://192.168.101.15:2379 --initial-cluster-token etcd-cluster-1 --initial-cluster master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380,node2=http://192.168.101.19:2380 --initial-cluster-state new --data-dir=/var/lib/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target
node2节点： [root@localhost ~]# cat /usr/lib/systemd/system/etcd.service [Unit] Description=etcd server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify WorkingDirectory=/var/lib/etcd/ EnvironmentFile=-/etc/etcd/etcd.conf ExecStart=/usr/local/bin/etcd --name node2 --initial-advertise-peer-urls http://192.168.101.19:2380 --listen-peer-urls http://192.168.101.19:2380 --listen-client-urls http://192.168.101.19:2379,http://127.0.0.1:2379 --advertise-client-urls http://192.168.101.19:2379 --initial-cluster-token etcd-cluster-1 --initial-cluster master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380,node2=http://192.168.101.19:2380 --initial-cluster-state new --data-dir=/var/lib/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target
由于并没有配置TSL认证，所以都是http而不是https，etcd客户端监听在2379，服务端监听在2380
三个节点配置完成了服务文件，于是加载该文件并启动服务：
# systemctl daemon-reload
# systemctl enable etcd
# systemctl start etcd
# systemctl status etcd
查看三个节点的状态：
[root@docker ~]# systemctl status etcd ● etcd.service - etcd server Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2017-11-23 13:56:05 CST; 12s ago Main PID: 21562 (etcd) CGroup: /system.slice/etcd.service └─21562 /usr/local/bin/etcd --name master --initial-advertise-peer-urls http://192.168.101.14:2380 --listen-peer-urls http://192.168.101.... Nov 23 13:56:05 docker etcd[21562]: enabled capabilities for version 3.0 Nov 23 13:56:06 docker etcd[21562]: health check for peer 192d36c71643c39d could not connect: dial tcp 192.168.101.19:2380: getsockopt: con...n refused Nov 23 13:56:07 docker etcd[21562]: peer 192d36c71643c39d became active Nov 23 13:56:07 docker etcd[21562]: established a TCP streaming connection with peer 192d36c71643c39d (stream Message writer) Nov 23 13:56:07 docker etcd[21562]: established a TCP streaming connection with peer 192d36c71643c39d (stream MsgApp v2 writer) Nov 23 13:56:07 docker etcd[21562]: established a TCP streaming connection with peer 192d36c71643c39d (stream MsgApp v2 reader) Nov 23 13:56:07 docker etcd[21562]: established a TCP streaming connection with peer 192d36c71643c39d (stream Message reader) Nov 23 13:56:09 docker etcd[21562]: updating the cluster version from 3.0 to 3.2 Nov 23 13:56:09 docker etcd[21562]: updated the cluster version from 3.0 to 3.2 Nov 23 13:56:09 docker etcd[21562]: enabled capabilities for version 3.2 Hint: Some lines were ellipsized, use -l to show in full.
[root@localhost ~]# systemctl status etcd ● etcd.service - etcd server Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2017-11-23 13:56:05 CST; 15s ago Main PID: 21504 (etcd) CGroup: /system.slice/etcd.service └─21504 /usr/local/bin/etcd --name node1 --initial-advertise-peer-urls http://192.168.101.15:2380 --listen-peer-urls http://192.168.101.1... Nov 23 13:56:05 localhost.localdomain systemd[1]: Started etcd server. Nov 23 13:56:05 localhost.localdomain etcd[21504]: set the initial cluster version to 3.0 Nov 23 13:56:05 localhost.localdomain etcd[21504]: enabled capabilities for version 3.0 Nov 23 13:56:07 localhost.localdomain etcd[21504]: peer 192d36c71643c39d became active Nov 23 13:56:07 localhost.localdomain etcd[21504]: established a TCP streaming connection with peer 192d36c71643c39d (stream Message writer) Nov 23 13:56:07 localhost.localdomain etcd[21504]: established a TCP streaming connection with peer 192d36c71643c39d (stream MsgApp v2 reader) Nov 23 13:56:07 localhost.localdomain etcd[21504]: established a TCP streaming connection with peer 192d36c71643c39d (stream MsgApp v2 writer) Nov 23 13:56:07 localhost.localdomain etcd[21504]: established a TCP streaming connection with peer 192d36c71643c39d (stream Message reader) Nov 23 13:56:09 localhost.localdomain etcd[21504]: updated the cluster version from 3.0 to 3.2 Nov 23 13:56:09 localhost.localdomain etcd[21504]: enabled capabilities for version 3.2
[root@localhost ~]# systemctl status etcd ● etcd.service - etcd server Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2017-11-23 13:56:07 CST; 17s ago Main PID: 21508 (etcd) CGroup: /system.slice/etcd.service └─21508 /usr/local/bin/etcd --name node2 --initial-advertise-peer-urls http://192.168.101.19:2380 --listen-peer-urls http://192.168.101.1... Nov 23 13:56:07 localhost.localdomain etcd[21508]: dialing to target with scheme: "" Nov 23 13:56:07 localhost.localdomain etcd[21508]: could not get resolver for scheme: "" Nov 23 13:56:07 localhost.localdomain etcd[21508]: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged! Nov 23 13:56:07 localhost.localdomain etcd[21508]: ready to serve client requests Nov 23 13:56:07 localhost.localdomain etcd[21508]: dialing to target with scheme: "" Nov 23 13:56:07 localhost.localdomain etcd[21508]: could not get resolver for scheme: "" Nov 23 13:56:07 localhost.localdomain etcd[21508]: serving insecure client requests on 192.168.101.19:2379, this is strongly discouraged! Nov 23 13:56:07 localhost.localdomain systemd[1]: Started etcd server. Nov 23 13:56:09 localhost.localdomain etcd[21508]: updated the cluster version from 3.0 to 3.2 Nov 23 13:56:09 localhost.localdomain etcd[21508]: enabled capabilities for version 3.2
没有报错问题后查看集群中的成员，任意节点执行就行：
[root@docker ~]# etcdctl member list 192d36c71643c39d: name=node2 peerURLs=http://192.168.101.19:2380 clientURLs=http://192.168.101.19:2379 isLeader=false 5f3835545a5f41e4: name=master peerURLs=http://192.168.101.14:2380 clientURLs=http://192.168.101.14:2379 isLeader=true 77c1ac60c5100363: name=node1 peerURLs=http://192.168.101.15:2380 clientURLs=http://192.168.101.15:2379 isLeader=false
可以看见集群中自动推选了一个节点作为leader，然后查看集群健康状态：
[root@docker ~]# etcdctl cluster-health member 192d36c71643c39d is healthy: got healthy result from http://192.168.101.19:2379 member 5f3835545a5f41e4 is healthy: got healthy result from http://192.168.101.14:2379 member 77c1ac60c5100363 is healthy: got healthy result from http://192.168.101.15:2379 cluster is healthy
使用etcd进行操作数据：
[root@docker ~]# etcdctl set name wadeson
wadeson
在node1、node2节点上进行查看：
[root@localhost ~]# etcdctl get name
wadeson
[root@localhost ~]# etcdctl get name
wadeson
下面也是查看集群健康状态的一种：
[root@docker ~]# curl http://192.168.101.14:2379/health {"health": "true"}[root@docker ~]# curl http://192.168.101.15:2379/health {"health": "true"}[root@docker ~]# curl http://192.168.101.19:2379/health {"health": "true"}
4、对etcd集群中的member进行操作：
4.1、更新advertise client urls
直接修改参数--advertise-client-urls或者ETCD_ADVERTISE_CLIENT_URLS，然后重启该member
4.2、更新advertise peer urls：
$ etcdctl member update 192d36c71643c39d http://192.168.101.15:2380
更新要修改的peer urls的member id和peer urls，然后重启member
4.3、移除集群中的某一个节点member：
首先查看member列表：
[root@docker ~]# etcdctl member list 192d36c71643c39d: name=node2 peerURLs=http://192.168.101.19:2380 clientURLs=http://192.168.101.19:2379 isLeader=false 5f3835545a5f41e4: name=master peerURLs=http://192.168.101.14:2380 clientURLs=http://192.168.101.14:2379 isLeader=true 77c1ac60c5100363: name=node1 peerURLs=http://192.168.101.15:2380 clientURLs=http://192.168.101.15:2379 isLeader=false
现在将node2也就是192.168.101.19这个节点移除该集群：
[root@docker ~]# etcdctl member remove 192d36c71643c39d Removed member 192d36c71643c39d from cluster [root@docker ~]# etcdctl member list 5f3835545a5f41e4: name=master peerURLs=http://192.168.101.14:2380 clientURLs=http://192.168.101.14:2379 isLeader=true 77c1ac60c5100363: name=node1 peerURLs=http://192.168.101.15:2380 clientURLs=http://192.168.101.15:2379 isLeader=false
集群针对的操作是对member_id进行操作
4.4、向集群增加一个member，现在集群只有两个member：
[root@docker ~]# etcdctl member list 5f3835545a5f41e4: name=master peerURLs=http://192.168.101.14:2380 clientURLs=http://192.168.101.14:2379 isLeader=true 77c1ac60c5100363: name=node1 peerURLs=http://192.168.101.15:2380 clientURLs=http://192.168.101.15:2379 isLeader=false
现在向该集群增加一个member，也就是增加一个节点：
[root@docker ~]# etcdctl member add node2 http://192.168.101.19:2380 Added member named node2 with ID 4edc521f6598ba03 to cluster ETCD_NAME="node2" ETCD_INITIAL_CLUSTER="node2=http://192.168.101.19:2380,master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380" ETCD_INITIAL_CLUSTER_STATE="existing"
然后安装提示操作：
[root@docker ~]# export ETCD_NAME="node2" [root@docker ~]# export ETCD_INITIAL_CLUSTER="node2=http://192.168.101.19:2380,master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380" [root@docker ~]# export ETCD_INITIAL_CLUSTER_STATE="existing"
[root@docker ~]# etcd --listen-client-urls http://192.168.101.19:2379 --advertise-client-urls http://192.168.101.19:2379 --listen-peer-urls http://192.168.101.19:2380 --initial-advertise-peer-urls http://192.168.101.19:2380 --data-dir %data_dir% 2017-11-23 14:30:33.649640 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER=node2=http://192.168.101.19:2380,master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380 2017-11-23 14:30:33.649694 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER_STATE=existing 2017-11-23 14:30:33.649706 I | pkg/flags: recognized and used environment variable ETCD_NAME=node2 2017-11-23 14:30:33.649748 I | etcdmain: etcd Version: 3.2.10 2017-11-23 14:30:33.649752 I | etcdmain: Git SHA: 694728c 2017-11-23 14:30:33.649754 I | etcdmain: Go Version: go1.8.5 2017-11-23 14:30:33.649757 I | etcdmain: Go OS/Arch: linux/amd64 2017-11-23 14:30:33.649760 I | etcdmain: setting maximum number of CPUs to 2, total number of available CPUs is 2 2017-11-23 14:30:33.649850 C | etcdmain: listen tcp 192.168.101.19:2380: bind: cannot assign requested address
不知道为何最后绑定不到192.168.101.19上，这里是一个问题。。。。。。。待后面解决
