基于Containerd的k8s部署
一、基础环境准备(所有节点)
1.1、节点规划
| 主机名 | IP地址 | 节点规划 | 操作系统 |
| master | 192.168.43.37 | master | CentOS7.9 |
| node01 | 192.168.43.38 | node | CentOS7.9 |
| node02 | 192.168.43.39 | node | CentOS7.9 |
1.2、修改主机名（按照节点规划执行）
hostnamectl set-hostname master hostnamectl set-hostname node01 hostnamectl set-hostname node02
1.3、修改host文件
echo '192.168.43.37 master' >> /etc/hosts echo '192.168.43.38 node01' >> /etc/hosts echo '192.168.43.39 node02' >> /etc/hosts
1.4、设置防火墙
systemctl stop firewalld&&systemctl disable firewalld
1.5设置SELINUX
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config setenforce 0
1.6、配置yum源
cd /etc/yum.repos.d/
mkdir bak
mv *.repo bak
下载repo文件上传
http://mirrors.cloud.tencent.com/repo/centos7_base.repo
安装wget和ntpdate
yum install ntpdate wget -y
1.7、时区及时间同步设置
timedatectl set-timezone Asia/Shanghai
设置时间同步
ntpdate time.windows.com
设置时间同步定时任务
crontab -e */5 * * * * ntpdate time.windows.com
1.8、关闭swap分区
sed -i '/swap/s/^\(.*\)$/#\1/g' /etc/fstab swapoff -a
1.9、配置iptables桥接流量
cat > /etc/sysctl.d/kubernetes.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 EOF
加载br_netfilter
modprobe br_netfilter
sysctl -p /etc/sysctl.d/kubernetes.conf
设置开机启动
cat >> /etc/rc.d/rc.local << EOF /usr/sbin/modprobe br_netfilter EOF
修改权限
chmod +x /etc/rc.d/rc.local
立即启动
modprobe br_netfilter
查看是否加载成功
sysctl --system
1.10、开启IPVS
IPVS（IP Virtual Server）是一个用户负载均衡的Linux内核模块，它可以用来替代kube-proxy默认的iptables，IPVS提供了更高效和可扩展的负载均衡，特别适用于大规模的机器环境。
cat > /etc/sysconfig/modules/ipvs.modules <<EOF #!/bin/bash modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack_ipv4 EOF
授权
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4
查看是否加载成功
lsmod | grep -e ipvs -e nf_conntrack_ipv4
安装ipset、ipvsadm
yum install ipset ipvsadm -y
二、部署containerd（所有节点执行）
2.1、下载
wget https://github.com/containerd/containerd/releases/download/v1.7.7/cri-containerd-1.7.7-linux-amd64.tar.gz
解压
tar -zxvf cri-containerd-1.7.7-linux-amd64.tar.gz -C /
创建配置文件
mkdir /etc/containerd containerd config default > /etc/containerd/config.toml
修改配置文件（这一行不修改，导致后面初始化报错，依据报错修改这个3.X版本号）
vi /etc/containerd/config.toml sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.8" #修改这一行
修改SystemdCgroup
sed -i 's#SystemdCgroup = false#SystemdCgroup = true#g' /etc/contained/config.toml
设置开机自启动
systemctl daemon-reload
systemctl restart containerd.service
systemctl enable --now containerd
2.2、安装runc
下载 libseccomp（因为runc依赖于libseccomp计算库）
wget https://github.com/opencontainers/runc/releases/download/v1.1.7/libseccomp-2.5.4.tar.gz
解压
tar -zxvf libseccomp-2.5.4.tar.gz
安装依赖包
yum install -y gcc gcc-c++ gperf
安装
cd libseccomp-2.5.4 ./configure make && make install
查看是否安装成功
find / -name libseccomp.so
下载runc
wget https://github.com/opencontainers/runc/releases/download/v1.1.7/runc.amd64
为下载的runc赋权
chmod +x runc.amd64
删除contained自带的runc
[root@localhost libseccomp-2.5.4]# whereis runc runc: /usr/local/sbin/runc [root@localhost libseccomp-2.5.4]# rm -rf /usr/local/sbin/runc
将runc.amd移动
mv runc.amd64 /usr/local/sbin/runc
运行
重启所有服务器
三、安装Kubernetes
3.1、配置yum源
cat > /etc/yum.repos.d/k8s.repo <<EOF [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey-https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF
查看是否存储目标镜像
yum clean all yum makecache yum list kubeadm.x86_64 --showduplicates yum -y install kubeadm-1.28.0-0 kubelet-1.28.0-0 kubectl-1.28.0-0
3.2、安装
yum -y install kubeadm-1.28.0-0 kubelet-1.28.0-0 kubectl-1.28.0-0
安装完成启动kubelet
systemctl enable kubelet.service
systemctl start kubelet.service
3.3、配置cgroup驱动改为systemd
vi /etc/sysconfig/kubelet KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"
3.4、初始化（只在master节点执行）
此处修改master的IP地址及版本
kubeadm init \ --apiserver-advertise-address=192.168.43.37 \ --image-repository=registry.aliyuncs.com/google_containers \ --kubernetes-version=v1.28.0 \ --service-cidr=10.96.0.0/12 \ --pod-network-cidr=10.244.0.0/16 \ --cri-socket=unix:///var/run/containerd/containerd.sock
出现如下报错（https://blog.csdn.net/qq_47904804/article/details/131030273）
[root@master ~]# kubeadm init \ > --apiserver-advertise-address=192.168.43.37 \ > --image-repository=registry.aliyuncs.com/google_containers \ > --kubernetes-version=v1.28.0 \ > --service-cidr=10.96.0.0/12 \ > --pod-network-cidr=10.244.0.0/16 \ > --cri-socket=unix:///var/run/containerd/containerd.sock [init] Using Kubernetes version: v1.28.0 [preflight] Running pre-flight checks error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables does not exist [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` To see the stack trace of this error execute with --v=5 or higher
解决办法
vi /etc/sysctl.conf net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1
重新加载
modprobe br_netfilter
sysctl -p
！！！！！！如果初始化失败，需要重新初始化执行如下命令（初始化成功跳过此步骤）
kubeadm reset rm -rf ~/.kube/ /etc/kubernetes/* var/lib/etcd/*
重新初始化成功
[root@master ~]# kubeadm init --apiserver-advertise-address=192.168.43.37 --image-repository=registry.aliyuncs.com/google_containers --kubernetes-version=v1.28.0 --service-cidr=10.96.0.0/12 --pod-network-cidr=10.244.0.0/16 --cri-socket=unix:///var/run/containerd/containerd.sock [init] Using Kubernetes version: v1.28.0 [preflight] Running pre-flight checks [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local master] and IPs [10.96.0.1 192.168.43.37] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [localhost master] and IPs [192.168.43.37 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [localhost master] and IPs [192.168.43.37 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [apiclient] All control plane components are healthy after 7.503764 seconds [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Skipping phase. Please see --upload-certs [mark-control-plane] Marking the node master as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] [mark-control-plane] Marking the node master as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule] [bootstrap-token] Using token: ohvbn8.5ugezvibuyz10dvq [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.43.37:6443 --token ohvbn8.5ugezvibuyz10dvq \ --discovery-token-ca-cert-hash sha256:8cf496d9eda390cf716b67809b025771d1560df9b0c2c46f431f603da5b64b78
按照提示本地执行：
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
3.5、从节点加入集群
执行主节点生产的命令，两个从节点执行
kubeadm join 192.168.43.37:6443 --token ohvbn8.5ugezvibuyz10dvq \ --discovery-token-ca-cert-hash sha256:8cf496d9eda390cf716b67809b025771d1560df9b0c2c46f431f603da5b64b78
在主节点查看
[root@master ~]# kubectl get node NAME STATUS ROLES AGE VERSION master NotReady control-plane 19m v1.28.0 node01 NotReady <none> 2m45s v1.28.0 node02 NotReady <none> 2m27s v1.28.0
3.6、设置kubelet开机自启动
systemctl enable --now kubelet
四、安装Calico插件
4.1、官方网站
https://docs.tigera.io/calico/3.25/getting-started/kubernetes/quickstart
4.2、按照官方网站安装（在master节点）
没有安装前查看节点状态
[root@master ~]# wget https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/tigera-operator.yaml
安装
[root@master ~]# kubectl create -f tigera-operator.yaml
下载
[root@master ~]# wget https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/custom-resources.yaml
修改地址为自己的pod地址
[root@master ~]# vi custom-resources.yaml
安装
[root@master ~]# kubectl create -f custom-resources.yaml
installation.operator.tigera.io/default created
apiserver.operator.tigera.io/default created
没有报错，安装后查看几点状态
[root@master ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
master Ready control-plane 16h v1.28.0
node01 Ready <none> 15h v1.28.0
node02 Ready <none> 15h v1.28.0
https://blog.csdn.net/m0_51510236/article/details/130842122
https://www.cnblogs.com/libruce/p/17687354.html
https://www.yuque.com/echox-dw1a4/tp0mpg/st6ab3nw73kd8rot?singleDoc#
