搭建k8s集群 v1.24.4

如果曾经安装过，建议清理旧的配置

rm -rf /root/.kube
rm -rf /etc/cni/net.d

ipvsadm -C
iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X

清理配置
rm -rf /etc/kubernetes/
rm -rf /etc/cni
rm -rf /opt/cni

卸载 kube*
rm -rf /etc/systemd/system/kubelet.service.d
rm -rf /etc/systemd/system/kubelet.service
yum remove kube*

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF

安装docker

docker安装

安装 kubeadm

kubeadm安装文档

kubeadm init \
  --apiserver-advertise-address=172.16.51.9 \
  --image-repository registry.aliyuncs.com/google_containers \
  --kubernetes-version v1.24.4 \
  --service-cidr=10.96.0.0/12 \
  --pod-network-cidr=192.168.0.0/16

如果遇到 container runtime is not running

rm -rf /etc/containerd/config.toml
systemctl restart containerd

如果遇到 invalid capacity 0 on image filesystem

systemctl restart containerd

超时问题

Unfortunately, an error has occurred:
        timed out waiting for the condition

This error is likely caused by:
        - The kubelet is not running
        - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
        - 'systemctl status kubelet'
        - 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
        - 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
        Once you have found the failing container, you can inspect its logs with:
        - 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher

failed to get sandbox image “k8s.gcr.io/pause:3.6”

由于k8s.gcr.io 需要连外网才可以拉取到，导致 k8s 的基础容器 pause 经常无法获取。k8s docker 可使用代理服拉取，再利用 docker tag 解决问题

docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6 k8s.gcr.io/pause:3.6

但是我们k8s集群中使用的CRI是containerd。所以只能通过 docker tag 镜像，再使用 ctr 导入镜像.

docker save k8s.gcr.io/pause -o pause.tar
ctr -n k8s.io images import pause.tar

安装 Pod 网络附加组件（calico或flannel）

calico安装文档

flannel flannel节点之间无法通信解决办法

允许控制平面节点上调度 Pod

kubectl taint nodes --all node-role.kubernetes.io/control-plane- node-role.kubernetes.io/master-

加入节点

遇到问题

error execution phase control-plane-prepare/download-certs: error downloading certs: error downloading the secret: secrets "kubeadm-certs" is forbidden
# certificate-key 默认2小时的有效期，如果过期按照提示重新生成
# 使用命令重新生成 certificate-key 
# 用新证书替换--certificate-key后面的内容
$ kubeadm init phase upload-certs --upload-certs

不关闭swap启动k8s

vi /etc/sysconfig/kubelet

KUBELET_EXTRA_ARGS="--fail-swap-on=false"

dashboard

其他pc访问dashboard，端口转发
安装dashboard

ssh -L localhost:8001:localhost:8001 -NT root@<ip>

创建用户

获取Token

kubectl -n kubernetes-dashboard create token admin-user

安装helm

helm

安装 ingress

选择下面任何一个即可

haproxy

haproxy-ingress

traefik

install-traefik

nginx-ingress

NGINX Ingress Controller

外部负载均衡器 metalLB

metalLB

ingress 配置Https

准备好证书文件，各种证书之间的转换方式参考转换证书格式

创建secret保存证书

kubectl create secret tls your-domian.com-tls -n your-namespace \
  --cert=/root/ca/cert.pem \
  --key=/root/ca/key.pem

ingress使用证书

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: haproxy
  name: hello-https
  namespace: your-namespace
spec:
  rules:
  - host: your-domian.com
    http:
      paths:
      - backend:
          service:
            name: hello-tls
            port:
              number: 8080
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - your-domian.com
    secretName: your-domian.com-tls

常用工具安装

yum install systemd-networkd