kubernetes部署（v1.24.3）不使用docker

一、初始化机器

1、设置机器hostsname

hostnamectl set-hostname k8s-master-01 #master
hostnamectl set-hostname k8s-node-01 #node

2、安装依赖

yum install -y conntrack ipvsadm ipset jq iptables curl sysstat libseccomp wget vim net-tools git


yum install -y chony # 配置时间同步

3、关闭防火墙、SELinux、swap

systemctl stop firewalld && systemctl disable firewalld
swapoff -a && sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
setenforce 0 && sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config

4、添加内核参数

cat > kubernetes.conf <<EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recycle=0
vm.swappiness=0 # 禁止使用 swap 空间，只有当系统 OOM 时才允许使用它
vm.overcommit
_
memory=1 # 不检查物理内存是否够用
vm.panic_on_oom=0 # 开启 OOM
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr
_
open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf
_
conntrack
_
max=2310720
EOF
cp kubernetes.conf /etc/sysctl.d/kubernetes.conf
sysctl -p /etc/sysctl.d/kubernetes.conf

4、调整时区

# 设置系统时区为 中国/上海
timedatectl set-timezone Asia/Shanghai
# 将当前的 UTC 时间写入硬件时钟
timedatectl set-local-rtc 0
# 重启依赖于系统时间的服务
systemctl restart rsyslog
systemctl restart crond
#关闭系统不需要服务
systemctl stop postfix && systemctl disable postfix

5、设置 rsyslogd 和 systemd journald

mkdir /var/log/journal # 持久化保存日志的目录
mkdir /etc/systemd/journald.conf.d
cat > /etc/systemd/journald.conf.d/99-prophet.conf <<EOF
[Journal]
# 持久化保存到磁盘
Storage=persistent
# 压缩历史日志
Compress=yes
SyncIntervalSec=5m
RateLimitInterval=30s
RateLimitBurst=1000
# 最大占用空间 10G
SystemMaxUse=10G
# 单日志文件最大 200M
SystemMaxFileSize=200M
# 日志保存时间 2 周
MaxRetentionSec=2week
# 不将日志转发到 syslog
ForwardToSyslog=no
EOF
systemctl restart systemd-journald

二、安装containerd

1、创建配置文件

cat << EOF > /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF
#命令使其生效
modprobe overlay
modprobe br_netfilter

2、创建k8s内核参数

cat << EOF > /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
user.max_user_namespaces=28633
vm.swappiness=0
EOF
#执行生效
sysctl -p /etc/sysctl.d/99-kubernetes-cri.conf

3、下载containerd二进制包

wget https://github.com/containerd/containerd/releases/download/v1.6.4/cri-containerd-cni-1.6.4-linux-amd64.tar.gz


tar -zxvf cri-containerd-cni-1.6.4-linux-amd64.tar.gz -C /

4、生成containerd配置文件

mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml

5、修改配置文件

grep SystemdCgroup /etc/containerd/config.toml
            SystemdCgroup = true #原本是false
grep sandbox_image /etc/containerd/config.toml
    sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.7" #原本是国外镜像修改成阿里

6、启动containerd

systemctl enable containerd --now
crictl version
#输出结果如下
Version:  0.1.0
RuntimeName:  containerd
RuntimeVersion:  v1.6.4
RuntimeApiVersion:  v1alpha2

7、配置服务器开启支持IPVS

cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOF


chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack

三、安装k8s集群

1、安装kubeadm

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF


yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes


systemctl enable --now kubelet

2、配置kubeadm初始化文件

[root@k8s-master-01 ~]# cat kubeadm-config.yaml 
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 172.16.149.202 #master-01 IP
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///run/containerd/containerd.sock
  taints:
  - effect: PreferNoSchedule
    key: node-role.kubernetes.io/master
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.24.3
imageRepository: registry.aliyuncs.com/google_containers
networking:
  podSubnet: 10.244.0.0/16
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
failSwapOn: false
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs

这里定制了imageRepository为阿里云的registry，避免因gcr被墙，无法直接拉取镜像。criSocket设置了容器运行时为containerd。同时设置kubelet的cgroupDriver为systemd，设置kube-proxy代理模式为ipvs

可以通过 kubeadm config print init-defaults --component-configs KubeletConfiguration 可以打印集群初始化默认的使用的配置.

在开始初始化集群之前可以使用kubeadm config images pull --config kubeadm-config.yaml预先在各个服务器节点上拉取所k8s需要的容器镜像。

3、初始化k8s master-01节点

kubeadm init --config kubeadm-config.yaml |tee kubeinit.log

看到最后有打印：Your Kubernetes control-plane has initialized successfully!

然后还有提示:

o start using your cluster, you need to run the following as a regular user:


  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config


Alternatively, if you are the root user, you can run:


  export KUBECONFIG=/etc/kubernetes/admin.conf


You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/


Then you can join any number of worker nodes by running the following on each as root:


kubeadm join 172.16.129.202:6443 --token e0nvot.l8sgzcgl07d5baq6 \
 --discovery-token-ca-cert-hash sha256:d07d9a5b919c23177881134e3ccf90e26fcb173133b8f6172cbf3d74f3c6a75d

然后在node节点上执行 如下命令来加入到集群中。node节点要执行上面二大步骤就可以了然后再加入到集群

kubeadm join 172.16.129.202:6443 --token e0nvot.l8sgzcgl07d5baq6 \
 --discovery-token-ca-cert-hash sha256:d07d9a5b919c23177881134e3ccf90e26fcb173133b8f6172cbf3d74f3c6a75d

查看一下集群状态，确认各个组件都处于healthy状态，是否有错误:

[root@k8s-master-01 ~]# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS    MESSAGE                         ERROR
scheduler            Healthy   ok                              
controller-manager   Healthy   ok                              
etcd-0               Healthy   {"health":"true","reason":""}   
[root@k8s-master-01 ~]# kubectl get no
NAME            STATUS   ROLES           AGE   VERSION
k8s-master-01   Ready    control-plane   55m   v1.24.3
k8s-node-01     Ready    <none>          42m   v1.24.3

集群初始化如果遇到问题，可以使用kubeadm reset命令进行清理.

这样k8s集群就部署完成了，网络插件可以自行选择开源的flannel、caclio、kube-ovn等