使用etcdkeeper管理kubeoperator安装k8s集群使用的etcd

官方文档地址：https://registry.hub.docker.com/r/deltaprojects/etcdkeeper

下载地址：https://github.com/evildecay/etcdkeeper/releases

https://github.com/evildecay/etcdkeeper/releases/download/v0.7.6/etcdkeeper-v0.7.6-linux_x86_64.zip

# cd /usr/local/src
# wget https://github.com/evildecay/etcdkeeper/releases/download/v0.7.6/etcdkeeper-v0.7.6-linux_x86_64.zip
# unzip etcdkeeper-v0.7.6-linux_x86_64.zip
# cd /usr/local/src/etcdkeeper
# chmod a+x etcdkeeper
# ./etcdkeeper --help 
Usage of ./etcdkeeper:
  -auth
        use auth
  -cacert string
        verify certificates of TLS-enabled secure servers using this CA bundle (v3)
  -cert string
        identify secure client using this TLS certificate file (v3)
  -h string
        host name or ip address (default "0.0.0.0")
  -key string
        identify secure client using this TLS key file (v3)
  -p int
        port (default 8080)
  -sep string
        separator (default "/")
  -timeout int
        ETCD client connect timeout (default 5)
  -usetls
        use tls

查看kubeoperator安装k8s集群使用的etcd的有关信息,可以获取到如下的信息
使用的版本是V3,采用静态发现方式安装的，有二进制文件，有启动使用的service文件，通过查看该service文件信息，可以获取到有关证书信息

# etcdctl version
etcdctl version: 3.5.2
API version: 3.5

# etcdctl member list
948b763ececdb5ee, started, etcd-test-k8s-master-1, https://10.16.16.111:2380, https://10.16.16.111:2379, false

# ll /usr/local/bin/
-rwxr-xr-x. 1 root root  23588864 2月   1 2022 etcd
-rwxr-xr-x. 1 root root  17993728 2月   1 2022 etcdctl

# systemctl status etcd
● etcd.service - Etcd Server
   Loaded: loaded (/etc/systemd/system/etcd.service; enabled; vendor preset: disabled)
   Active: active (running) since 三 2023-03-15 17:36:10 CST; 1 weeks 6 days ago
     Docs: https://github.com/coreos
 Main PID: 1010 (etcd)
    Tasks: 12
   Memory: 507.0M
   CGroup: /system.slice/etcd.service
           └─1010 /usr/local/bin/etcd --name=etcd-test-k8s-master-1 --cert-file=/etc/kubernetes/pki/etcd/server.crt --key-file=/etc/kubernetes/pki/etcd/server.key --pe...

3月 29 10:36:12 test-k8s-master-1 etcd[1010]: {"level":"info","ts":"2023-03-29T10:36:12.761+0800","caller":"v3compactor/periodic.go:134","msg":"starting aut...:"1h0m0s"}
3月 29 10:36:12 test-k8s-master-1 etcd[1010]: {"level":"info","ts":"2023-03-29T10:36:12.762+0800","caller":"v3compactor/periodic.go:142","msg":"completed au…:"277.91µs"}
3月 29 10:36:34 test-k8s-master-1 etcd[1010]: {"level":"info","ts":"2023-03-29T10:36:34.584+0800","caller":"mvcc/index.go:214","msg":"compact tree index","r...":6095110}
3月 29 10:36:34 test-k8s-master-1 etcd[1010]: {"level":"info","ts":"2023-03-29T10:36:34.598+0800","caller":"mvcc/kvstore_compaction.go:57","msg":"finished s...227622ms"}
3月 29 10:41:34 test-k8s-master-1 etcd[1010]: {"level":"info","ts":"2023-03-29T10:41:34.592+0800","caller":"mvcc/index.go:214","msg":"compact tree index","r...":6095596}
3月 29 10:41:34 test-k8s-master-1 etcd[1010]: {"level":"info","ts":"2023-03-29T10:41:34.606+0800","caller":"mvcc/kvstore_compaction.go:57","msg":"finished s...020413ms"}
3月 29 10:46:34 test-k8s-master-1 etcd[1010]: {"level":"info","ts":"2023-03-29T10:46:34.595+0800","caller":"mvcc/index.go:214","msg":"compact tree index","r...":6096078}
3月 29 10:46:34 test-k8s-master-1 etcd[1010]: {"level":"info","ts":"2023-03-29T10:46:34.609+0800","caller":"mvcc/kvstore_compaction.go:57","msg":"finished s...939006ms"}
3月 29 10:51:34 test-k8s-master-1 etcd[1010]: {"level":"info","ts":"2023-03-29T10:51:34.599+0800","caller":"mvcc/index.go:214","msg":"compact tree index","r...":6096565}
3月 29 10:51:34 test-k8s-master-1 etcd[1010]: {"level":"info","ts":"2023-03-29T10:51:34.614+0800","caller":"mvcc/kvstore_compaction.go:57","msg":"finished s...893426ms"}
Hint: Some lines were ellipsized, use -l to show in full.

# cat /etc/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd

ExecStart=/usr/local/bin/etcd \
  --name=etcd-test-k8s-master-1 \
  --cert-file=/etc/kubernetes/pki/etcd/server.crt \
  --key-file=/etc/kubernetes/pki/etcd/server.key \
  --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt \
  --peer-key-file=/etc/kubernetes/pki/etcd/peer.key \
  --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt \
  --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt \
  --initial-advertise-peer-urls=https://10.16.16.111:2380 \
  --listen-peer-urls=https://10.16.16.111:2380 \
  --listen-client-urls=https://10.16.16.111:2379,http://127.0.0.1:2379 \
  --advertise-client-urls=https://10.16.16.111:2379 \
  --initial-cluster-token=etcd-cluster-token \
  --initial-cluster=etcd-test-k8s-master-1=https://10.16.16.111:2380 \
  --initial-cluster-state=new \
  --data-dir=/var/lib/etcd \
  --snapshot-count=50000 \
  --auto-compaction-retention=1 \
  --max-request-bytes=10485760 \
  --quota-backend-bytes=8589934592
Restart=always
RestartSec=15
LimitNOFILE=65536
OOMScoreAdjust=-999

[Install]
WantedBy=multi-user.target

etcdkeeper的命令参数跟etcd证书参数的匹配关系如下：(使用其中的三个证书)

启动命令：

# ./etcdkeeper -p=65530 -usetls -cacert=/etc/kubernetes/pki/etcd/ca.crt -key=/etc/kubernetes/pki/etcd/server.key -cert=/etc/kubernetes/pki/etcd/server.crt
2023-03-29 10:49:17.862945 I | listening on 0.0.0.0:65530

使用浏览器访问：http://10.16.16.111:65530/etcdkeeper

版本切换到V3，修改一下连接etcd的地址

查看日志；

2023-03-29 10:50:20.898429 I | POST v3 connect success.
2023-03-29 10:50:20.933013 I | GET v3 /
2023-03-29 10:50:27.052112 I | GET v3 /registry/clusterrolebindings/ingress-nginx

点击目录树上的内容，etcdkeeper的运行日志上会显示出具体的操作

问题：右边显示的值中有好多乱码的特殊字符

解决办法，不显示目录树，直接显示出所有的键值对

右下角分别显示出：当前键的过期时间（0为永不过期），etcd版本，占用的内存空间存储大小，etcd的节点名称，etcdkeeper的版本

目录树结构的也能正确显示，mode选择json，然后再点击“格式化数据”