kubeadm部署高可用K8S证书过期
kubeadm部署高可用K8S证书过期
kubeadm 默认证书为一年,一年过期后,会导致 api service 不可用,使用过程中会出现:x509: certificate has expired or is not yet valid.
可以在初始化群集之前重新编译 kubeadm,证书有效期自动为 100 年。
兼容性:
已经测试适用于以下版本:1.17.0、1.18.0、1.19.0、1.20.0、1.21.0、1.22.0、1.23.0
1. 获取源码
访问 https://github.com/kubernetes/kubernetes/releases下载特定版本源码
wget https://github.com/kubernetes/kubernetes/archive/v1.22.2.tar.gz
tar -zxvf kubernetes-1.23.0.tar.gz
mv kubernetes-1.23.0 kubernetes
cd kubernetes
或者使用 git 获取
yum install git
git clone --depth=1 -b v1.22.2 https://github.com/kubernetes/kubernetes.git
2.修改证书有效期
修改证书有效期为100年(默认一年)
vim cmd/kubeadm/app/constants/constants.go
CertificateValidity = time.Hour * 24 * 365 * 99
修改CA有效期为100年(默认十年)
vim ./staging/src/k8s.io/client-go/util/cert/cert.go
const duration365d = time.Hour * 24 * 365
git diff验证,最后修改如下所示,已将证书都修改为100年
[root@master01 kubernetes]# git diff
diff --git a/cmd/kubeadm/app/constants/constants.go b/cmd/kubeadm/app/constants/constants.go
index b355973..c9be686 100644
--- a/cmd/kubeadm/app/constants/constants.go
+++ b/cmd/kubeadm/app/constants/constants.go
@@ -45,7 +45,7 @@ const (
TempDirForKubeadm = "tmp"
// CertificateValidity defines the validity for all the signed certificates generated by kubeadm
- CertificateValidity = time.Hour * 24 * 365
+ CertificateValidity = time.Hour * 24 * 365 * 99
// CACertAndKeyBaseName defines certificate authority base name
CACertAndKeyBaseName = "ca"
diff --git a/staging/src/k8s.io/client-go/util/cert/cert.go b/staging/src/k8s.io/client-go/util/cert/cert.go
index bffb152..9e20e3a 100644
--- a/staging/src/k8s.io/client-go/util/cert/cert.go
+++ b/staging/src/k8s.io/client-go/util/cert/cert.go
@@ -64,7 +64,7 @@ func NewSelfSignedCACert(cfg Config, key crypto.Signer) (*x509.Certificate, erro
},
DNSNames: []string{cfg.CommonName},
NotBefore: now.UTC(),
- NotAfter: now.Add(duration365d * 10).UTC(),
+ NotAfter: now.Add(duration365d * 100).UTC(),
KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCer
BasicConstraintsValid: true,
IsCA: true,
3.编译kubeadm
3.1本地环境编译
3.1.1 安装编译软件包
CentOS:
yum install gcc make -y
yum install rsync jq -y
Ubuntu:
sudo apt install build-essential
sudo apt install rsync jq -y
3.1.2 GoLang环境安装
查看kube-cross的TAG版本号
# cat ./build/build-image/cross/VERSION
v1.22.0-go1.16.8-buster.0
#注意:之前的版本显示这种数字格式 v1.16.8-1
安装Go环境
wget https://golang.google.cn/dl/go1.16.8.linux-amd64.tar.gz
tar zxvf go1.16.8.linux-amd64.tar.gz -C /usr/local
在 /etc/profile 文件添加如下:
#go setting
export GOROOT=/usr/local/go
export GOPATH=/usr/local/gopath
export PATH=$PATH:$GOROOT/bin
生效
#source /etc/profile
验证:
go version
# 输出如下
go version go1.16.8 linux/amd64
4.编译
#编译 kubeadm, 这里主要编译 kubeadm 即可
make all WHAT=cmd/kubelet GOFLAGS=-v
# 编译 kubectl
make all WHAT=cmd/kubectl GOFLAGS=-v
# 编译 kubelet
make all WHAT=cmd/kubelet GOFLAGS=-v
编译完成后,输出文件位于 _output/bin/kubeadm 目录下
其中 bin 是使用了软连接
真实路径是_output/local/bin/linux/amd64/kubeadm
备份源kubeadm文件
集群模式,所有master节点都要进行kubeadm替换与证书更新
mv /usr/bin/kubeadm /usr/bin/kubeadm_backup
替换kubeadm文件
cp _output/local/bin/linux/amd64/kubeadm /usr/bin/kubeadm
chmod +x /usr/bin/kubeadm
查看编译后的版本
kubeadm version
#输出如下
kubeadm version: &version.Info{Major:"1", Minor:"22+", GitVersion:"v1.22.2-dirty", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"dirty", BuildDate:"2022-09-14T01:43:43Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}
5.更新证书
如果是使用原版 kubeadm 安装之后,可以手动执行命令更新证书有效期到 100 年。
可以先备份证书,证书在 /etc/kubernetes/pki
5.1 检查证书到期时间
kubeadm certs check-expiration
# 早期版本 (1.19 及之前版本) 命令如下
#kubeadm alpha certs check-expiration
#输出如下
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED
admin.conf Oct 21, 2022 08:34 UTC 36d no
apiserver Oct 21, 2022 08:33 UTC 36d ca no
apiserver-etcd-client Oct 21, 2022 08:34 UTC 36d etcd-ca no
apiserver-kubelet-client Oct 21, 2022 08:34 UTC 36d ca no
controller-manager.conf Oct 21, 2022 08:34 UTC 36d no
etcd-healthcheck-client Oct 21, 2022 08:34 UTC 36d etcd-ca no
etcd-peer Oct 21, 2022 08:34 UTC 36d etcd-ca no
etcd-server Oct 21, 2022 08:34 UTC 36d etcd-ca no
front-proxy-client Oct 21, 2022 08:34 UTC 36d front-proxy-ca no
scheduler.conf Oct 21, 2022 08:34 UTC 36d no
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
ca Oct 19, 2031 08:33 UTC 9y no
etcd-ca Oct 19, 2031 08:34 UTC 9y no
front-proxy-ca Oct 19, 2031 08:34 UTC 9y no
kubernetes V1.20 开始废弃kubeadm alpha certs 命令。
kubernetes V1.21 开始弃用kubeadm alpha 命令。
5.2 续订所有证书
kubeadm certs renew all
#输出,提示重启控制平面Pod
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed
Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates
5.3 再次检查证书有效期
kubeadm certs check-expiration
#输出如下
[root@master01 ~]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED
admin.conf Aug 22, 2121 01:44 UTC 98y no
apiserver Aug 22, 2121 01:44 UTC 98y ca no
apiserver-etcd-client Aug 22, 2121 01:44 UTC 98y etcd-ca no
apiserver-kubelet-client Aug 22, 2121 01:44 UTC 98y ca no
controller-manager.conf Aug 22, 2121 01:44 UTC 98y no
etcd-healthcheck-client Aug 22, 2121 01:44 UTC 98y etcd-ca no
etcd-peer Aug 22, 2121 01:44 UTC 98y etcd-ca no
etcd-server Aug 22, 2121 01:44 UTC 98y etcd-ca no
front-proxy-client Aug 22, 2121 01:44 UTC 98y front-proxy-ca no
scheduler.conf Aug 22, 2121 01:44 UTC 98y no
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
ca Sep 11, 2032 09:09 UTC 9y no
etcd-ca Sep 11, 2032 09:09 UTC 9y no
front-proxy-ca Sep 11, 2032 09:09 UTC 9y no
6.实施后步骤
运行 kubeadm renew 命令后,我们应该重启控制平面Pod。目前并非所有组件和证书都支持动态证书重新加载。由于静态Pod由本地kubelet管理,而不是由API服务器管理,因此无法使用kubectl删除和重新启动它们。要重新启动静态Pod,我们可以暂时从/etc/kubernetes/manifests/中删除其清单文件,然后等待20秒。如果Pod不再位于manifest目录中,kubelet 将终止Pod。然后我们可以将文件移回,在另一个fileCheckFrequency周期之后,kubelet将重新创建 Pod,并且可以完成组件的证书更新。
mkdir -p /opt/temp
mv /etc/kubernetes/manifests/*.yaml /opt/temp
#等待30秒之后
cp /opt/temp/*.yaml /etc/kubernetes/manifests/
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】凌霞软件回馈社区,博客园 & 1Panel & Halo 联合会员上线
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】博客园社区专享云产品让利特惠,阿里云新客6.5折上折
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· CSnakes vs Python.NET:高效嵌入与灵活互通的跨语言方案对比
· DeepSeek “源神”启动!「GitHub 热点速览」
· 我与微信审核的“相爱相杀”看个人小程序副业
· Plotly.NET 一个为 .NET 打造的强大开源交互式图表库
· 上周热点回顾(2.17-2.23)