K8S v1.27.6 安装，使用containerd作为容器运行时

1.主机列表
master-1-230:192.168.3.230
node-1-231:192.168.3.231
node-1-232:192.168.3.232

注意：从2开始到10，三台主机均配置

2.配置hosts文件 
cat >> /etc/hosts <<EOF
192.168.3.230 master-1-230
192.168.3.231 node-1-231
192.168.3.232 node-1-232
EOF

3.关闭防火墙selinux、关闭swap
hostnamectl set-hostname master-1-230
systemctl stop firewalld 
systemctl disable firewalld
sed -i 's/enforcing/disabled/g' /etc/sysconfig/selinux
sed -i 's/enforcing/disabled/g' /etc/selinux/config
setenforce 0
swapoff -a
sed -ri 's/.*swap.*/#&/' /etc/fstab

4.系统优化
cat > /etc/sysctl.d/k8s_better.conf << EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720
EOF

 
modprobe br_netfilter
lsmod |grep conntrack
modprobe ip_conntrack
sysctl -p /etc/sysctl.d/k8s_better.conf

5.时间同步
yum install ntpdate -y
crontab -l
* * * * *    /usr/sbin/ntpdate time1.aliyun.com

6.配置master 节点到node节点的免密钥登录
ssh-keygen  (一直回车)

ssh-copy-id  root@node-1-231
ssh-copy-id  root@node-1-232

7.安装ipvs转发支持
7.1安装系统依赖包
yum install -y conntrack ipvsadm ipset jq iptables curl sysstat libseccomp wget vim net-tools git

7.2开启ipvs转发
modprobe br_netfilter

vim /etc/sysconfig/modules/ipvs.modules
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack

chmod 755 /etc/sysconfig/modules/ipvs.modules
bash /etc/sysconfig/modules/ipvs.modules

lsmod | grep -e ip_vs -e nf_conntrack

8.安装containerd

8.1 创建 /etc/modules-load.d/containerd.conf 配置文件
cat << EOF > /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

modprobe overlay
modprobe br_netfilter

8.2配置阿里云 yum 源
wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

8.3安装containerd
yum install -y containerd.io

生成containerd的配置文件

mkdir /etc/containerd -p
containerd config default > /etc/containerd/config.toml

修改配置文件
cat /etc/containerd/config.toml |egrep "SystemdCgroup|registry.aliyuncs.com/google_containers/pause:3.9"

sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9"
SystemdCgroup = true

启动containerd

systemctl enable containerd
systemctl restart containerd

9.配置K8S v1.27.6 yum源
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

查看所有的可用版本
yum list kubelet --showduplicates | sort -r |grep 1.27

10.安装kubeadm、kubelet 和kubectl
yum install -y kubectl-1.27.6-0 kubelet-1.27.6-0 kubeadm-1.27.6-0

systemctl  enable  kubelet
systemctl  restart  kubelet

11.master主机上操作
查看k8s 1.27.6 所需镜像
kubeadm config images list --kubernetes-version=v1.27.6

设置crictl 连接containerd
crictl config --set runtime-endpoint=unix:///run/containerd/containerd.sock

初始化，注意ip地址
kubeadm init --kubernetes-version=v1.27.6 --pod-network-cidr=10.224.0.0/16 --apiserver-advertise-address=192.168.3.230 --image-repository registry.aliyuncs.com/google_containers

查看镜像
crictl image ls

查看cgroup驱动
cat /var/lib/kubelet/config.yaml

cgroupDriver: systemd

初始化成功后显示的信息
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.3.230:6443 --token d28xa7.s6zgflu9zawqeodg \
        --discovery-token-ca-cert-hash sha256:5c66b4692c82b1149e823ed52574c8391ba58a414b964dcb22aab9243fbb624c 

master主机上操作步骤
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf

12.分别在两台node主机上操作这个：
kubeadm join 192.168.3.230:6443 --token d28xa7.s6zgflu9zawqeodg \
        --discovery-token-ca-cert-hash sha256:5c66b4692c82b1149e823ed52574c8391ba58a414b964dcb22aab9243fbb624c 


13.下载calico(master主机上操作)
wget --no-check-certificate  https://projectcalico.docs.tigera.io/archive/v3.25/manifests/calico.yaml

修改CALICO_IPV4POOL_CIDR 配置
- name: CALICO_IPV4POOL_CIDR
  value: "10.244.0.0/16"

kubectl apply -f calico.yaml

14.验证
kubectl  cluster-info
kubectl  get nodes
kubectl  get pods -A

15.在k8s快速部署一个应用

kubectl create deployment nginxdp --image=nginx:1.25.2 
kubectl get deployment
kubectl get pod -o wide
kubectl describe pod nginxdp-7cf46d7445-bkkvl
kubectl expose deployment nginxdp --port=80 --type=NodePort --target-port=80 --name=nginxsvc
kubectl get svc
nginxsvc     NodePort    10.104.228.111   <none>        80:30401/TCP   5s

curl http://192.168.3.231:30401
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>