kubernetes：v1.25 + containerd
由于kubernets从v1.24开始停止支持dockershim，kubernets不再支持通过docker来创建和管理容器。本文记录安装kubernetes v1.25 + containerd
1、环境
| IP | 主机名 | 节点角色 | 操作系统 |
| 10.0.8.101 | master01,k8s-master01,k8s-api.ilinux.io | master | Ubuntu 22.04.1 LTS |
| 10.0.8.111 | node01,k8s-node01 | node | Ubuntu 22.04.1 LTS |
| 10.0.8.112 | node02,k8s-node2 | node | Ubuntu 22.04.1 LTS |
| 10.0.8.113 | node03,k8s-node03 | node | Ubuntu 22.04.1 LTS |
2、时区调整
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
3、配置镜像源 这里是有aliyun
sed -i 's@http://cn.archive.ubuntu.com@https://mirrors.aliyun.com@g' /etc/apt/sources.list
apt-get update
网卡配置：
cat /etc/netplan/00-installer-config.yaml
network:
ethernets:
ens33:
dhcp4: false
addresses: [192.168.3.101/24]
nameservers:
addresses: [192.168.3.1]
gateway4: 192.168.3.1
ethernets:
ens37:
dhcp4: false
addresses: [10.0.8.101/24]
version: 2
4、配置域名
#cat /etc/hosts 10.0.8.101 master01 k8s-master01 k8s-api.ilinux.io 10.0.8.111 node01 k8s-node01 10.0.8.112 node02 k8s-node02 10.0.8.113 node03 k8s-node03
5、关闭防火墙
#systemctl disable --now ufw
#systemctl stop ufw
6、关闭swap
swapoff -a # 临时swap sed -ri 's/.*swap.*/#&/' /etc/fstab # 永久关闭swap
7、优化资源限制参数
ulimit -SHn 65535 cat >>/etc/security/limits.conf <<EOF * soft nofile 65536 * hard nofile 131072 * soft nproc 65535 * hard nproc 655350 * soft memlock unlimited * hard memlock unlimited EOF
8、安装ipvs等工具并优化内存
apt -y install ipvsadm ipset sysstat conntrack libseccomp-dev
加载ipvs模块以及配置：
modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack cat >> /etc/modules-load.d/ipvs.conf <<EOF ip_vs ip_vs_lc ip_vs_wlc ip_vs_rr ip_vs_wrr ip_vs_lblc ip_vs_lblcr ip_vs_dh ip_vs_sh ip_vs_fo ip_vs_nq ip_vs_sed ip_vs_ftp ip_vs_sh nf_conntrack ip_tables ip_set xt_set ipt_set ipt_rpfilter ipt_REJECT ipip EOF systemctl restart systemd-modules-load.service
9、内核参数调整
cat > /etc/sysctl.d/k8s.conf <<EOF net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 fs.may_detach_mounts = 1 vm.overcommit_memory=1 vm.panic_on_oom=0 fs.inotify.max_user_watches=89100 fs.file-max=52706963 fs.nr_open=52706963 net.netfilter.nf_conntrack_max=2310720 net.ipv4.tcp_keepalive_time = 600 net.ipv4.tcp_keepalive_probes = 3 net.ipv4.tcp_keepalive_intvl =15 net.ipv4.tcp_max_tw_buckets = 36000 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_max_orphans = 327680 net.ipv4.tcp_orphan_retries = 3 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_max_syn_backlog = 16384 net.ipv4.ip_conntrack_max = 65536 net.ipv4.tcp_max_syn_backlog = 16384 net.ipv4.tcp_timestamps = 0 net.core.somaxconn = 16384 EOF sysctl --system
10、配置containerd需要的模块
cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf overlay br_netfilter EOF 加载模块： modprobe -- overlay modprobe -- br_netfilter 配置Containerd所需的内核： cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 EOF 加载内核： sysctl --system
11、安装containerd
# step 1: 安装必要的一些系统工具 sudo apt-get update sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common # step 2: 安装GPG证书 curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add - # Step 3: 写入软件源信息 sudo add-apt-repository "deb [arch=amd64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable" # Step 4: 更新并安装containerd.io sudo apt-get -y update sudo apt-get -y install containerd.io critcl
12、配置containerd
生成配置文件：containerd config default > /etc/containerd/config.toml
修改配置文件 sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml grep 'SystemdCgroup' -B 11 /etc/containerd/config.toml #检查是否修改成功 将sandbox_image镜像地址更改为阿里云： sed -i 's@k8s.gcr.io@registry.aliyuncs.com/google_containers@g' /etc/containerd/config.toml grep 'sandbox_image' /etc/containerd/config.toml 镜像加速： 配置文件：/etc/containerd/config.toml，增加配置路径 [plugins."io.containerd.grpc.v1.cri".registry] config_path = "/etc/containerd/certs.d" 创建加速配置文件 如：docker.io mkdir -p /etc/containerd/certs.d/docker.io 配置文件：hosts.toml cat /etc/containerd/certs.d/docker.io/hosts.toml server = "https://docker.io" [host."http://hub-mirror.c.163.com"] capabilities = ["pull","resolve"]
#增加私有仓库： 如：harbor.myland.com mkdir -p /etc/containerd/certs.d/harbor.myland.com
#配置文件：hosts.toml,如果不需要验证服务端证书，skip_verify 设置为 true即可，ca.crt为签发harbor的server.crt的 CA的证书 cat /etc/containerd/certs.d/harbor.myland.com/hosts.toml
server = "https://harbor.myland.com" [host."https://harbor.myland.com"] capabilities = ["pull", "resolve", "push"] skip_verify = false ca = "/etc/containerd/certs.d/harbor.myland.com/ca.crt" systemctl daemon-reload && systemctl enable --now containerd systemctl restart containerd
#可以安装nerdctl工具管理containerd，改命令风格与docker命令相似
https://github.com/containerd/nerdctl
13、安装kubelet kubeadm kubectl
apt update && apt install -y apt-transport-https curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list apt update -y apt install -y kubelet kubeadm kubectl
14、kubernetes初始化
kubeadm init \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.25.2 \ --control-plane-endpoint k8s-api.ilinux.io \ --apiserver-advertise-address 10.0.8.101 \ --pod-network-cidr 10.201.0.0/16 \ --service-cidr 10.97.0.0/16 \ --token-ttl 0
13、验证
查看k8s集群
ctr查看containerd
nerdctl查看
netctl检查containerd配置，与docker一致：如检查PID
也可以设置默认的nerdctl使用的namespace
mkdir /etc/nerdctl && cat 'namespace = "k8s.io"' >/ etc/nerdctl/nerdctl.toml
