Kubernetes v1.25 编译 kubeadm 修改证书有效期到 100 年

请访问原文链接：https://sysin.org/blog/kubernetes-kubeadm-cert-100y/ 查看最新版。原创作品，转载请保留出处。

作者主页：sysin.org

kubernetes-flower

随着 Kubernetes 1.25 的发布（August 23, 2022），没写完的文章又要测试一下新版本才能完成。

kubeadm 默认证书为一年，一年过期后，会导致 api service 不可用，使用过程中会出现：x509: certificate has expired or is not yet valid.

Google 建议通过不停更新版本来自动更新证书，太坑 ^_

可以在初始化群集之前重新编译 kubeadm，证书有效期自动为 100 年。

兼容性：

已经测试适用于以下版本

1.17.0
1.18.0
1.19.0
1.20.0
1.21.0
1.22.0
1.23.0
1.24.0
1.25.0

1. 获取源码

访问：https://github.com/kubernetes/kubernetes/releases，下载特定版本源码

wget https://github.com/kubernetes/kubernetes/archive/v1.25.0.tar.gz
tar -zxvf kubernetes-1.25.0.tar.gz
mv kubernetes-1.25.0 kubernetes
cd kubernetes

或者使用 git 获取

# yum install git
# sudo apt install git
git clone https://github.com/kubernetes/kubernetes.git
cd kubernetes
git checkout -b remotes/origin/release-1.25 v1.25.0

2. 修改证书有效期

查看网上的资料主要有两个地方需要修改

2.1 修改 CA 有效期为 100 年（默认为 10 年）

vim ./staging/src/k8s.io/client-go/util/cert/cert.go

// 这个方法里面 NotAfter:              now.Add(duration365d * 10).UTC()
// 默认有效期就是 10 年，改成 100 年 (sysin)
// 输入 /NotAfter 查找，回车定位
func NewSelfSignedCACert(cfg Config, key crypto.Signer) (*x509.Certificate, error) {
        now := time.Now()
        tmpl := x509.Certificate{
                SerialNumber: new(big.Int).SetInt64(0),
                Subject: pkix.Name{
                        CommonName:   cfg.CommonName,
                        Organization: cfg.Organization,
                },
                NotBefore:             now.UTC(),
                // NotAfter:              now.Add(duration365d * 10).UTC(),
                NotAfter:              now.Add(duration365d * 100).UTC(),
                KeyUsage:              x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
                BasicConstraintsValid: true,
                IsCA:                  true,
        }

        certDERBytes, err := x509.CreateCertificate(cryptorand.Reader, &tmpl, &tmpl, key.Public(), key)
        if err != nil {
                return nil, err
        }
        return x509.ParseCertificate(certDERBytes)
}

2.2 修改证书有效期为 100 年（默认为 1 年）

vim ./cmd/kubeadm/app/constants/constants.go

// 就是这个常量定义 CertificateValidity，改成 * 100 年 (sysin)
// 输入 /CertificateValidity 查找，回车定位
const (
        // KubernetesDir is the directory Kubernetes owns for storing various configuration files
        KubernetesDir = "/etc/kubernetes"
        // ManifestsSubDirName defines directory name to store manifests
        ManifestsSubDirName = "manifests"
        // TempDirForKubeadm defines temporary directory for kubeadm
        // should be joined with KubernetesDir.
        TempDirForKubeadm = "tmp"

        // CertificateValidity defines the validity for all the signed certificates generated by kubeadm
        // CertificateValidity = time.Hour * 24 * 365
        CertificateValidity = time.Hour * 24 * 365 * 100

        // CACertAndKeyBaseName defines certificate authority base name
        CACertAndKeyBaseName = "ca"
        // CACertName defines certificate name
        CACertName = "ca.crt"
        // CAKeyName defines certificate name
        CAKeyName = "ca.key"

验证一下已经正确修改：

cat ./staging/src/k8s.io/client-go/util/cert/cert.go | grep NotAfter
cat ./cmd/kubeadm/app/constants/constants.go | grep CertificateValidity

git 验证（可选，适用于 git 获取的源码），修改的内容如下：

git diff

diff --git a/cmd/kubeadm/app/constants/constants.go b/cmd/kubeadm/app/constants/constants.go
index 75adf43..54f25fa 100644
--- a/cmd/kubeadm/app/constants/constants.go
+++ b/cmd/kubeadm/app/constants/constants.go
@@ -44,7 +44,7 @@ const (
        TempDirForKubeadm = "tmp"

        // CertificateValidity defines the validity for all the signed certificates generated by kubeadm
-       CertificateValidity = time.Hour * 24 * 365
+       CertificateValidity = time.Hour * 24 * 365 * 100

        // CACertAndKeyBaseName defines certificate authority base name
        CACertAndKeyBaseName = "ca"
diff --git a/staging/src/k8s.io/client-go/util/cert/cert.go b/staging/src/k8s.io/client-go/util/cert/cert.go
index 9fd097a..865d6bb 100644
--- a/staging/src/k8s.io/client-go/util/cert/cert.go
+++ b/staging/src/k8s.io/client-go/util/cert/cert.go
@@ -63,7 +63,7 @@ func NewSelfSignedCACert(cfg Config, key crypto.Signer) (*x509.Certificate, erro
                        Organization: cfg.Organization,
                },
                NotBefore:             now.UTC(),
-               NotAfter:              now.Add(duration365d * 10).UTC(),
+               NotAfter:              now.Add(duration365d * 100).UTC(),
                KeyUsage:              x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
                BasicConstraintsValid: true,
                IsCA:                  true,

源代码改好了，接下来就是编译 kubeadm 了。

3. 编译

3.1 Docker 镜像编译

注意：该方法仅适用早期版本，现在已经不可用。

使用该方法可以获得与官方一致的编译环境。

目前 gcr 上的最新版本仍然是 v1.13.6-1（665.3 MB），构建于 2020年1月16日。

查看 kube-cross 的 TAG 版本号

cat ./build/build-image/cross/VERSION

v1.25.0-go1.19-bullseye.0

这里我们可以使用官方容器对代码进行编译：k8s.gcr.io/kube-cross:v1.19-1

拉取镜像

docker pull k8s.gcr.io/kube-cross:v1.19-1

无法FQ可以用下面的替代镜像：比较遗憾都没有更新！

docker pull gcrcontainer/kube-cross:v1.19-1

或者：docker pull registry.aliyuncs.com/google_containers/kube-cross:v1.19-1

编译

# docker run --rm -v <你修改后的代码目录>:/go/src/k8s.io/kubernetes -it gcrcontainer/kube-cross bash
docker run --rm -v /root/kubernetes:/go/src/k8s.io/kubernetes -it k8s.gcr.io/kube-cross:v1.19-1 bash

cd /go/src/k8s.io/kubernetes

# 编译 kubeadm, 这里主要编译 kubeadm 即可
make all WHAT=cmd/kubeadm GOFLAGS=-v

# 编译 kubelet
# make all WHAT=cmd/kubelet GOFLAGS=-v

# 编译 kubectl
# make all WHAT=cmd/kubectl GOFLAGS=-v

# 退出容器
exit

#编译完产物在 _output/bin/kubeadm 目录下，
#其中 bin 是使用了软连接 (sysin)
#真实路径是_output/local/bin/linux/amd64/kubeadm
mv /usr/bin/kubeadm /usr/bin/kubeadm_backup
cp _output/local/bin/linux/amd64/kubeadm /usr/bin/kubeadm
#chmod +x /usr/bin/kubeadm

# 验证版本
kubeadm version

3.2 本机编译

环境需求参看官方文档。

本例在 AlmaLinux 9 x86_64 中编译完成。

3.2.1 软件包准备

CentOS：

yum groupinstall "Development Tools" -y #gcc, make etc.
yum install rsync jq -y

Ubuntu：

sudo apt install build-essential #(Following command will install essential commands like gcc, make etc.)
sudo apt install rsync jq -y

3.2.2 GoLang 环境

查看 kube-cross 的 TAG 版本号

# cat ./build/build-image/cross/VERSION
v1.25.0-go1.19-bullseye.0
#注意：之前的版本显示这种数字格式 v1.19-1

安装 Go 环境：

wget https://dl.google.com/go/go1.19.linux-amd64.tar.gz
## 或者
# wget https://golang.google.cn/dl/go1.19.linux-amd64.tar.gz
tar zxvf go1.19.linux-amd64.tar.gz  -C /usr/local

# 编辑 / etc/profile 文件添加如下：
#go setting (sysin)
#export GOROOT=/usr/local/go
#export GOPATH=/usr/local/gopath
#export PATH=$PATH:$GOROOT/bin
#生效
#source /etc/profile

# 这里一次性编译，直接执行如下命令即可
export PATH=$PATH:/usr/local/go/bin

验证：

go version
# 输出如下
go version go1.19 linux/amd64

编译：

# 编译 kubeadm, 这里主要编译 kubeadm 即可
make all WHAT=cmd/kubeadm GOFLAGS=-v

# 编译 kubelet
# make all WHAT=cmd/kubelet GOFLAGS=-v

# 编译 kubectl
# make all WHAT=cmd/kubectl GOFLAGS=-v

#编译完产物在 _output/bin/kubeadm 目录下，
#其中 bin 是使用了软连接 (sysin)
#真实路径是_output/local/bin/linux/amd64/kubeadm
mv /usr/bin/kubeadm /usr/bin/kubeadm_backup
cp _output/local/bin/linux/amd64/kubeadm /usr/bin/kubeadm
chmod +x /usr/bin/kubeadm

查看编译后的版本信息

kubeadm version
# 输出如下
kubeadm version: &version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.0", GitCommit:"a866cbe2e5bbaa01cfd5e969aa3e033f3282a8a2", GitTreeState:"archive", BuildDate:"2022-10-09T05:25:21Z", GoVersion:"go1.19", Compiler:"gc", Platform:"linux/amd64"}

保存备用

# 压缩保存到用户主目录下
tar zcvf ~/kubeadm-1.25.0.tgz -C/usr/bin/ kubeadm

# 在其他节点上替换原有版本
mv /usr/bin/kubeadm /usr/bin/kubeadm_bak
tar zxvf ./kubeadm-1.25.0.tgz -C /usr/bin/

4. 执行命令更新证书

如果是使用原版 kubeadm 安装之后，可以手动执行命令更新证书有效期到 100 年。

可以先备份证书，证书在 /etc/kubernetes/pki

检查证书到期时间

kubeadm certs check-expiration
# 早期版本 (1.19 及之前版本) 命令如下
#kubeadm alpha certs check-expiration

kubeadm alpha certs 命令 1.20 开始废弃。

kubeadm alpha 命令 1.21 开始彻底废弃。

参看 kubeadm alpha（替换 URL 中的版本号查看状态）。

输出如下：

[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Dec 08, 2121 05:35 UTC   99y                                     no
apiserver                  Dec 08, 2121 05:35 UTC   99y             ca                      no
apiserver-etcd-client      Dec 08, 2121 05:35 UTC   99y             etcd-ca                 no
apiserver-kubelet-client   Dec 08, 2121 05:35 UTC   99y             ca                      no
controller-manager.conf    Dec 08, 2121 05:35 UTC   99y                                     no
etcd-healthcheck-client    Dec 08, 2121 05:35 UTC   99y             etcd-ca                 no
etcd-peer                  Dec 08, 2121 05:35 UTC   99y             etcd-ca                 no
etcd-server                Dec 08, 2121 05:35 UTC   99y             etcd-ca                 no
front-proxy-client         Dec 08, 2121 05:35 UTC   99y             front-proxy-ca          no
scheduler.conf             Dec 08, 2121 05:35 UTC   99y                                     no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Dec 08, 2121 05:35 UTC   99y             no
etcd-ca                 Dec 08, 2121 05:35 UTC   99y             no
front-proxy-ca          Dec 08, 2121 05:35 UTC   99y             no

续订证书（查看可以使用的参数）

kubeadm certs renew --help
This command is not meant to be run on its own. See list of available subcommands.

Usage:
  kubeadm certs renew [flags]
  kubeadm certs renew [command]

Available Commands:
  admin.conf               Renew the certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself
  all                      Renew all available certificates
  apiserver                Renew the certificate for serving the Kubernetes API
  apiserver-etcd-client    Renew the certificate the apiserver uses to access etcd
  apiserver-kubelet-client Renew the certificate for the API server to connect to kubelet
  controller-manager.conf  Renew the certificate embedded in the kubeconfig file for the controller manager to use
  etcd-healthcheck-client  Renew the certificate for liveness probes to healthcheck etcd
  etcd-peer                Renew the certificate for etcd nodes to communicate with each other
  etcd-server              Renew the certificate for serving etcd
  front-proxy-client       Renew the certificate for the front proxy client
  scheduler.conf           Renew the certificate embedded in the kubeconfig file for the scheduler manager to use

Flags:
  -h, --help   help for renew

Global Flags:
      --add-dir-header           If true, adds the file directory to the header of the log messages
      --log-file string          If non-empty, use this log file
      --log-file-max-size uint   Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
      --one-output               If true, only write logs to their native severity level (vs also writing to each lower severity level)
      --rootfs string            [EXPERIMENTAL] The path to the 'real' host root filesystem.
      --skip-headers             If true, avoid header prefixes in the log messages
      --skip-log-headers         If true, avoid headers when opening log files
  -v, --v Level                  number for the log level verbosity

Use "kubeadm certs renew [command] --help" for more information about a command.

续订全部证书

kubeadm certs renew all

再次查看证书有效期，全部都 100 年了

kubeadm certs check-expiration

5. 下载

已经修改好的 kubeadm 下载（1.17.0、1.18.0、1.19.0，1.20.0 - 1.25.0 持续更新中）：

百度网盘链接：https://sysin.org/blog/kubernetes-kubeadm-cert-100y/

posted @ 2022-10-09 14:21 sysin 阅读(900) 评论(0) 编辑收藏举报

刷新页面返回顶部

sysin