Kubernetes etcd集群的安装配置
1.简介
etcd 是高可用的分布式的键值数据库，
2.配置证书
下载cfssl 工具 参考《Kubernetes CFSSL介绍 》安装配置
2.1生成ca机构
证书配置，设置过期时间为10年
cat > ca-config.json <<EOF { "signing": { "default": { "expiry": "87600h" }, "profiles": { "www": { "expiry": "87600h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] } } } } EOF
证书信息
cat > ca-csr.json <<EOF { "CN": "etcd CA", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "Beijing", "ST": "Beijing" } ] } EOF
生成证书
[root@dn01 etcd-cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca - [root@dn01 etcd-cert]# ll total 24 -rw-r--r--. 1 root root 287 Sep 10 18:33 ca-config.json -rw-r--r--. 1 root root 956 Sep 10 18:35 ca.csr -rw-r--r--. 1 root root 209 Sep 10 18:33 ca-csr.json -rw-------. 1 root root 1679 Sep 10 18:35 ca-key.pem -rw-r--r--. 1 root root 1265 Sep 10 18:35 ca.pem
2.2为etcd颁发证书
生成证书的json文件
cat > server-csr.json <<EOF { "CN": "etcd", "hosts": [ "10.10.100.30", "10.10.100.31", "10.10.100.32" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing" } ] } EOF
生成etcd 的证书
[root@dn01 etcd-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
注意：选项-profile对应的值是ca配置文件中的profiles值，其中ca-config.json中配置的profiles值可以是多个。
3.安装配置etcd集群
注意：安装配置前先关闭防火墙，否则会在后面启动etcd服务时报错
3.1下载程序包
https://github.com/etcd-io/etcd/releases
3.2 解压上传到服务器上的etcd压缩包
[root@dn01 k8s]# tar -zxf etcd-v3.3.10-linux-amd64.tar.gz
[root@dn01 k8s]# ls etcd-v3.3.10-linux-amd64
Documentation etcd etcdctl README-etcdctl.md README.md READMEv2-etcdctl.md
3.3 为etcd 创建安装目录
[root@dn01 k8s]# mkdir /opt/etcd/{cfg,bin,ssl} -p
[root@dn01 k8s]# ls /opt/etcd/
bin cfg ssl
3.4 将解压的etcd压缩包文件中的两个可执行文件移到/opt/etcd/bin目录下
[root@dn01 k8s]# mv etcd-v3.3.10-linux-amd64/etcd etcd-v3.3.10-linux-amd64/etcdctl /opt/etcd/bin/
3.5 将生成的etcd证书拷贝的etcd的配置目录下
[root@dn01 etcd-cert]# pwd
/root/k8s/etcd-cert
[root@dn01 etcd-cert]# cp *pem /opt/etcd/ssl/
[root@dn01 etcd-cert]# ls /opt/etcd/ssl/
ca-key.pem ca.pem server-key.pem server.pem
3.6 为etcd 的启动生成配置文件，配置启动选项，默认没有这些文件，需要手动创建
[root@dn01 k8s]# vi /opt/etcd/cfg/etcd
#[Member] ETCD_NAME="etcd01" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="https://10.10.100.30:2380" ETCD_LISTEN_CLIENT_URLS="https://10.10.100.30:2379" #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.10.100.30:2380" ETCD_ADVERTISE_CLIENT_URLS="https://10.10.100.30:2379" ETCD_INITIAL_CLUSTER="etcd01=https://10.10.100.30:2380,etcd02=https://10.10.100.31:2380,etcd03=https://10.10.100.32:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new
为系统system目录创建etcd 服务，本例中将system管理的etcd 服务放置在目录/usr/lib/systemd/system下
[root@dn01 ~]# cat /usr/lib/systemd/system/etcd.service [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify EnvironmentFile=/opt/etcd/cfg/etcd ExecStart=/opt/etcd/bin/etcd --name=${ETCD_NAME} --data-dir=${ETCD_DATA_DIR} --listen-peer-urls=${ETCD_LISTEN_PEER_URLS} --listen-client-urls=${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 --advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} --initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} --initial-cluster=${ETCD_INITIAL_CLUSTER} --initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} --initial-cluster-state=new --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem --peer-key-file=/opt/etcd/ssl/server-key.pem --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target
将etcd 所有配置文件拷贝到其他节点
[root@dn01 k8s]# scp -r /opt/etcd/ root@10.10.100.31:/opt/ The authenticity of host '10.10.100.31 (10.10.100.31)' can't be established. ECDSA key fingerprint is SHA256:pyiZjF3b1phvgSDt3+LU2LbME/tEfDsNOrZJCCZiicg. ECDSA key fingerprint is MD5:35:c1:58:24:d0:7f:a9:6c:d9:99:68:a2:98:b8:9a:8d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '10.10.100.31' (ECDSA) to the list of known hosts. root@10.10.100.31's password: etcd 100% 502 395.8KB/s 00:00 etcd 100% 18MB 38.4MB/s 00:00 etcdctl 100% 15MB 61.6MB/s 00:00 ca-key.pem 100% 1679 1.6MB/s 00:00 ca.pem 100% 1265 1.8MB/s 00:00 server-key.pem 100% 1679 3.1MB/s 00:00 server.pem 100% 1338 2.7MB/s 00:00 [root@dn01 k8s]#
节点3同样要拷贝
将system的服务文件拷贝到其他节点
[root@dn01 k8s]# scp /usr/lib/systemd/system/etcd.service root@10.10.100.31:/usr/lib/systemd/system root@10.10.100.31's password: etcd.service 100% 923 738.6KB/s 00:00
节点3同样要拷贝
修改节点2和节点3的etcd配置文件，将文件中涉及的ETCD_NAME值对应自身的节点名称，IP地址修改为自身的ip地址
例如节点2实例：
[root@dn02 cfg]# vi etcd #[Member] ETCD_NAME="etcd02" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="https://10.10.100.31:2380" ETCD_LISTEN_CLIENT_URLS="https://10.10.100.31:2379" #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.10.100.31:2380" ETCD_ADVERTISE_CLIENT_URLS="https://10.10.100.31:2379" ETCD_INITIAL_CLUSTER="etcd01=https://10.10.100.30:2380,etcd02=https://10.10.100.31:2380,etcd03=https://10.10.100.32:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new"
节点3的修改方式相同
当在每个节点修改完配置文件后，启动etcd ，会自动加入到进去，这时主节点日志不会在出现报错
本例在节点2上修改配置完成后启动etcd
加载启动配置 [root@dn02 k8s]# systemctl daemon-reload 启动etcd [root@dn02 k8s]# systemctl start etcd
同样节点3 修改后加载启动即可
各个节点启动之后，查看集群的健康状态
[root@dn01 ~]# /opt/etcd/bin/etcdctl --ca-file=/root/k8s/etcd-cert/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://10.10.100.30:2379,https://10.10.100.31:2379,https://10.10.100.32:2379" cluster-health member e47f480ee9f6f5c is healthy: got healthy result from https://10.10.100.32:2379 member 85cd1a03e146ee71 is healthy: got healthy result from https://10.10.100.30:2379 member bc5952299e602aaa is healthy: got healthy result from https://10.10.100.31:2379 cluster is healthy 注意ca证书的目录位置，和etcd证书的目录位置
