kubernetes单Master集群部署--Etcd集群部署(2)
1、工具准备
使用cfssl来生成自签证书，先下载cfssl工具：
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64 mv cfssl_linux-amd64 /usr/local/bin/cfssl mv cfssljson_linux-amd64 /usr/local/bin/cfssljson mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo
cfssl工具，子命令介绍：
-
bundle: 创建包含客户端证书的证书包 -
genkey: 生成一个key(私钥)和CSR(证书签名请求) -
scan: 扫描主机问题 -
revoke: 吊销证书 -
certinfo: 输出给定证书的证书信息， 跟cfssl-certinfo 工具作用一样 -
gencrl: 生成新的证书吊销列表 -
selfsign: 生成一个新的自签名密钥和 签名证书 -
print-defaults: 打印默认配置，这个默认配置可以用作模板config：生成配置模板
csr：生成证书请求模板
-
serve: 启动一个HTTP API服务 -
gencert: 生成新的key(密钥)和签名证书-
-ca：指明ca的证书
-
-ca-key：指明ca的私钥文件
-
-config：指明请求证书的json文件
-
-profile：与-config中的profile对应，是指根据config中的profile段来生成证书的相关信息
-
-
ocspdump -
ocspsign -
info: 获取有关远程签名者的信息 -
sign: 签名一个客户端证书，通过给定的CA和CA密钥，和主机名 -
ocsprefresh -
ocspserve
2、生成证书
创建以下三个文件：
cat ca-config.json
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"www": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
server auth表示client可以用该ca对server提供的证书进行验证
client auth表示server可以用该ca对client提供的证书进行验证
创建证书签名请求
cat ca-csr.json
{
"CN": "etcd CA",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing"
}
]
}
cat server-csr.json
{
"CN": "etcd",
"hosts": [
"10.11.97.191",
"10.11.97.192",
"10.11.97.71"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing"
}
]
}
生成CA证书和私钥
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
# ls *pem
3、部署Etcd
二进制包下载地址：https://github.com/coreos/etcd/releases/tag/v3.2.12
以下部署步骤在规划的三个etcd节点操作一样，唯一不同的是etcd配置文件中的服务器IP要写当前的！
解压二进制包：
# mkdir /opt/etcd/{bin,cfg,ssl} -p
# tar zxvf etcd-v3.2.12-linux-amd64.tar.gz
# mv etcd-v3.2.12-linux-amd64/{etcd,etcdctl} /opt/etcd/bin/
创建etcd配置文件：
# cat /opt/etcd/cfg/etcd
#[Member] ETCD_NAME="etcd01" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="https://10.11.97.191:2380" ETCD_LISTEN_CLIENT_URLS="https://10.11.97.191:2379" #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.11.97.191:2380" ETCD_ADVERTISE_CLIENT_URLS="https://10.11.97.191:2379" ETCD_INITIAL_CLUSTER="etcd01=https://10.11.97.191:2380,etcd02=https://10.11.97.192:2380,etcd03=https://10.11.97.71:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new"
- ETCD_NAME 节点名称
- ETCD_DATA_DIR 数据目录
- ETCD_LISTEN_PEER_URLS 集群通信监听地址
- ETCD_LISTEN_CLIENT_URLS 客户端访问监听地址
- ETCD_INITIAL_ADVERTISE_PEER_URLS 集群通告地址
- ETCD_ADVERTISE_CLIENT_URLS 客户端通告地址
- ETCD_INITIAL_CLUSTER 集群节点地址
- ETCD_INITIAL_CLUSTER_TOKEN 集群Token
- ETCD_INITIAL_CLUSTER_STATE 加入集群的当前状态，new是新集群，existing表示加入已有集群
4、systemd管理etcd
cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=/opt/etcd/cfg/etcd
ExecStart=/opt/etcd/bin/etcd \
--name=${ETCD_NAME} \
--data-dir=${ETCD_DATA_DIR} \
--listen-peer-urls=${ETCD_LISTEN_PEER_URLS} \
--listen-client-urls=${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
--advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} \
--initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
--initial-cluster=${ETCD_INITIAL_CLUSTER} \
--initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} \
--initial-cluster-state=new \
--cert-file=/opt/etcd/ssl/server.pem \
--key-file=/opt/etcd/ssl/server-key.pem \
--peer-cert-file=/opt/etcd/ssl/server.pem \
--peer-key-file=/opt/etcd/ssl/server-key.pem \
--trusted-ca-file=/opt/etcd/ssl/ca.pem \
--peer-trusted-ca-file=/opt/etcd/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
把刚才生成的证书拷贝到配置文件中的位置：
# cp ca*pem server*pem /opt/etcd/ssl
5、启动Etcd集群
# systemctl start etcd
# systemctl enable etcd
都部署完成后，检查etcd集群状态：
/opt/etcd/bin/etcdctl \ --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem \ --endpoints="https://10.11.97.191:2379,https://10.11.97.192:2379,https://10.11.97.71:2379" \ cluster-health
V3版本命令如下：
etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="https://10.11.97.181:2379,https://10.11.97.187:2379,https://10.11.97.71:2379" endpoint status
https://10.11.97.181:2379, 7dabc9878efbb430, 3.3.13, 1.9 MB, true, 1109, 20488746 https://10.11.97.187:2379, 9343710c8350509f, 3.3.13, 1.9 MB, false, 1109, 20488746 https://10.11.97.71:2379, c7d3ab01affc106, 3.3.13, 1.9 MB, false, 1109, 20488746
6、检查健康状况
如果输出上面信息，就说明集群部署成功。如果有问题第一步先看日志：/var/log/message 或 journalctl -u etcd
V3版本命令如下：
etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="https://10.11.97.181:2379,https://10.11.97.187:2379,https://10.11.97.71:2379" endpoint health
https://10.11.97.71:2379 is healthy: successfully committed proposal: took = 3.309008ms https://10.11.97.181:2379 is healthy: successfully committed proposal: took = 3.008708ms https://10.11.97.187:2379 is healthy: successfully committed proposal: took = 42.179839ms
7、安装脚本化
我们可以将Etcd的安装实行脚本化
证书生成脚本：etcd-cert.sh
服务和配置脚本：etcd.sh
具体地址：
