如何在已有的 K8S 集群中使用 etcdctl 工具操作 etcd 集群
1、集群信息
K8S 集群信息,集群中有三个 master 节点
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-m1 Ready master 55d v1.17.0
k8s-m2 Ready master 55d v1.17.0
k8s-m3 Ready master 55d v1.17.0
etcd 集群以 pod 方式运行在 K8S 集群之上
# kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
etcd-k8s-m1 1/1 Running 44 55d 172.0.2.139 k8s-m1 <none> <none>
etcd-k8s-m2 1/1 Running 2 26m 172.0.2.146 k8s-m2 <none> <none>
etcd-k8s-m3 1/1 Running 3779 55d 172.0.2.234 k8s-m3 <none> <none>
2、下载 etcdctl
查看 etcd 版本,需要下载 3.4.3 版本的 etcdctl
[root@k8s-m1 member]# kubectl describe pods etcd-k8s-m1 -n kube-system
Name: etcd-k8s-m1
Namespace: kube-system
Priority: 2000000000
Priority Class Name: system-cluster-critical
Node: k8s-m1/172.0.2.139
Start Time: Mon, 13 Apr 2020 02:28:39 -0400
Labels: component=etcd
tier=control-plane
Annotations: kubernetes.io/config.hash: 3d4819355a9752ba239aa13c1885dcc1
kubernetes.io/config.mirror: 3d4819355a9752ba239aa13c1885dcc1
kubernetes.io/config.seen: 2020-02-20T04:27:11.811231481-05:00
kubernetes.io/config.source: file
Status: Running
IP: 172.0.2.139
IPs:
IP: 172.0.2.139
Controlled By: Node/k8s-m1
Containers:
etcd:
Container ID: docker://c8722c4def309777ca9be9fb7a273521f6fe3cb3195105a10121f22c24310fe6
Image: k8s.gcr.io/etcd:3.4.3-0
下载 etcd 版本,解压,copy etcdctl 到 k8s master 节点的 /usr/bin 目录下
# wget https://github.com/etcd-io/etcd/releases/download/v3.4.3/etcd-v3.4.3-linux-amd64.tar.gz .
[root@k8s-m1 member]# ls -l /usr/bin/etcdctl
-rwxr-xr-x. 1 root root 17542688 Mar 4 03:09 /usr/bin/etcdctl
[root@k8s-m1 member]# etcdctl version
etcdctl version: 3.4.3
API version: 3.4
3、使用 etcdctl
3.1、获取 etcd 的 endpoint
endpoint 为 https://172.0.2.139:2379
# kubectl get pods etcd-k8s-m1 -o yaml -n kube-system
...
containers:
- command:
- etcd
- --advertise-client-urls=https://172.0.2.139:2379
- --cert-file=/etc/kubernetes/pki/etcd/server.crt
- --client-cert-auth=true
- --data-dir=/var/lib/etcd
- --initial-advertise-peer-urls=https://172.0.2.139:2380
- --initial-cluster=k8s-m1=https://172.0.2.139:2380
- --key-file=/etc/kubernetes/pki/etcd/server.key
- --listen-client-urls=https://127.0.0.1:2379,https://172.0.2.139:2379
- --listen-metrics-urls=http://127.0.0.1:2381
- --listen-peer-urls=https://172.0.2.139:2380
- --name=k8s-m1
- --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
- --peer-client-cert-auth=true
- --peer-key-file=/etc/kubernetes/pki/etcd/peer.key
- --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
- --snapshot-count=10000
- --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
image: k8s.gcr.io/etcd:3.4.3-0
...
3.2、准备 key 和 cert
etcd 的 endpoint 为 https 方式,所以要为 etcdctl 命令准备 key 和 cert
在 3.1 节输出信息中:
key 使用 /etc/kubernetes/pki/etcd/peer.key
cert 使用 /etc/kubernetes/pki/etcd/peer.crt
3.3、执行 etcdctl 命令
# etcdctl --cert /etc/kubernetes/pki/etcd/peer.crt --key /etc/kubernetes/pki/etcd/peer.key --endpoints https://172.0.2.139:2379 --insecure-skip-tls-verify member list
1e2fb9983e528532, started, k8s-m2, https://172.0.2.146:2380, https://172.0.2.146:2379, false
947c9889866d299a, started, k8s-m3, https://172.0.2.234:2380, https://172.0.2.234:2379, false
e97c0cc82d69a534, started, k8s-m1, https://172.0.2.139:2380, https://172.0.2.139:2379, false
注意:因为集群证书为自签发,所以这里需要加上–insecure-skip-tls-verify 参数,不然会报如下错误
# etcdctl --cert /etc/kubernetes/pki/etcd/peer.crt --key /etc/kubernetes/pki/etcd/peer.key --endpoints https://172.0.2.139:2379 member list
{"level":"warn","ts":"2020-04-16T05:00:52.085-0400","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"endpoint://client-c086c9e1-cb96-4c26-890e-b311b761b2c3/172.0.2.139:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest connection error: connection error: desc = \"transport: authentication handshake failed: x509: certificate signed by unknown authority\""}
Error: context deadline exceeded
摘抄自网络,便于检索查找。