Service Mesh服务网格技术探究---VMWare+k8s集群+Istio系列：k8s集群之master安装
一、Master安装
环境准备：
1、卸载podman
[root@k8s-master ~]# yum remove podman Dependencies resolved. ================================================================================ Package Arch Version Repository Size ================================================================================ Removing: podman x86_64 3.2.3-0.10.module_el8.4.0+886+c9a8d9ad @appstream 48 M Removing dependent packages: cockpit-podman noarch 32-2.module_el8.4.0+886+c9a8d9ad @appstream 410 k Removing unused dependencies: conmon x86_64 2:2.0.29-1.module_el8.4.0+886+c9a8d9ad @appstream 164 k podman-catatonit x86_64 3.2.3-0.10.module_el8.4.0+886+c9a8d9ad @appstream 752 k Transaction Summary ================================================================================ Remove 4 Packages Freed space: 50 M Is this ok [y/N]: y Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Running scriptlet: cockpit-podman-32-2.module_el8.4.0+886+c9a8d9ad.noar 1/1 Erasing : cockpit-podman-32-2.module_el8.4.0+886+c9a8d9ad.noar 1/4 Erasing : podman-3.2.3-0.10.module_el8.4.0+886+c9a8d9ad.x86_64 2/4 Running scriptlet: podman-3.2.3-0.10.module_el8.4.0+886+c9a8d9ad.x86_64 2/4 Erasing : podman-catatonit-3.2.3-0.10.module_el8.4.0+886+c9a8d 3/4 Erasing : conmon-2:2.0.29-1.module_el8.4.0+886+c9a8d9ad.x86_64 4/4 Running scriptlet: conmon-2:2.0.29-1.module_el8.4.0+886+c9a8d9ad.x86_64 4/4 Verifying : cockpit-podman-32-2.module_el8.4.0+886+c9a8d9ad.noar 1/4 Verifying : conmon-2:2.0.29-1.module_el8.4.0+886+c9a8d9ad.x86_64 2/4 Verifying : podman-3.2.3-0.10.module_el8.4.0+886+c9a8d9ad.x86_64 3/4 Verifying : podman-catatonit-3.2.3-0.10.module_el8.4.0+886+c9a8d 4/4 Removed: cockpit-podman-32-2.module_el8.4.0+886+c9a8d9ad.noarch conmon-2:2.0.29-1.module_el8.4.0+886+c9a8d9ad.x86_64 podman-3.2.3-0.10.module_el8.4.0+886+c9a8d9ad.x86_64 podman-catatonit-3.2.3-0.10.module_el8.4.0+886+c9a8d9ad.x86_64 Complete!
2、关闭交换区
临时关闭
[root@k8s-master ~]# swapoff -a
永久关闭
[root@k8s-master ~]# sed -i 's/.*swap.*/#&/' /etc/fstab
3、禁用selinux
临时禁用
[root@k8s-master ~]# setenforce 0
永久禁用
[root@k8s-master ~]# sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
4、关闭防火墙
[root@k8s-master ~]# systemctl stop firewalld.service [root@k8s-master ~]# systemctl disable firewalld.service Removed /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
二、k8s安装
1、配置系统基本安装源
[root@k8s-master ~]# curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-8.repo % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 2595 100 2595 0 0 3707 0 --:--:-- --:--:-- --:--:-- 3707
2、添加k8s安装源
编辑/etc/yum.repos.d/kubernetes.repo文件
[root@k8s-master ~]# vim /etc/yum.repos.d/kubernetes.repo
在kubernetes.repo文件中增加如下内容：
[kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
3、安装docker
[root@k8s-master ~]# yum install -y yum-utils device-mapper-persistent-data lvm2 net-tools Repository extras is listed more than once in the configuration CentOS-8 - Base - mirrors.aliyun.com 1.1 MB/s | 7.4 MB 00:06 CentOS-8 - Extras - mirrors.aliyun.com 37 kB/s | 10 kB 00:00 CentOS-8 - AppStream - mirrors.aliyun.com 1.1 MB/s | 8.9 MB 00:08 Kubernetes 3.2 kB/s | 844 B 00:00 Kubernetes 27 kB/s | 3.4 kB 00:00 Importing GPG key 0x307EA071: Userid : "Rapture Automatic Signing Key (cloud-rapture-signing-key-2021-03-01-08_01_09.pub)" Fingerprint: 7F92 E05B 3109 3BEF 5A3C 2D38 FEEA 9169 307E A071 From : https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg Importing GPG key 0x836F4BEB: Userid : "gLinux Rapture Automatic Signing Key (//depot/google3/production/borg/cloud-rapture/keys/cloud-rapture-pubkeys/cloud-rapture-signing-key-2020-12-03-16_08_05.pub) <glinux-team@google.com>" Fingerprint: 59FE 0256 8272 69DC 8157 8F92 8B57 C5C2 836F 4BEB From : https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg Kubernetes 7.3 kB/s | 975 B 00:00 Importing GPG key 0x3E1BA8D5: Userid : "Google Cloud Packages RPM Signing Key <gc-team@google.com>" Fingerprint: 3749 E1BA 95A8 6CE0 5454 6ED2 F09C 394C 3E1B A8D5 From : https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg Kubernetes 193 kB/s | 126 kB 00:00 Package device-mapper-persistent-data-0.8.5-4.el8.x86_64 is already installed. Package lvm2-8:2.03.11-5.el8.x86_64 is already installed. Package net-tools-2.0-0.52.20160912git.el8.x86_64 is already installed. Dependencies resolved. ============================================================================================================================================================ Package Architecture Version Repository Size ============================================================================================================================================================ Installing: yum-utils noarch 4.0.18-4.el8 base 71 k Transaction Summary ============================================================================================================================================================ Install 1 Package Total download size: 71 k Installed size: 22 k Downloading Packages: yum-utils-4.0.18-4.el8.noarch.rpm 185 kB/s | 71 kB 00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 183 kB/s | 71 kB 00:00 warning: /var/cache/dnf/base-43708d1174dbbac2/packages/yum-utils-4.0.18-4.el8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 8483c65d: NOKEY CentOS-8 - Base - mirrors.aliyun.com 2.5 kB/s | 1.6 kB 00:00 Importing GPG key 0x8483C65D: Userid : "CentOS (CentOS Official Signing Key) <security@centos.org>" Fingerprint: 99DB 70FA E1D7 CE22 7FB6 4882 05B5 55B3 8483 C65D From : https://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-Official Key imported successfully Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : yum-utils-4.0.18-4.el8.noarch 1/1 Running scriptlet: yum-utils-4.0.18-4.el8.noarch 1/1 Verifying : yum-utils-4.0.18-4.el8.noarch 1/1 Installed: yum-utils-4.0.18-4.el8.noarch Complete! [root@k8s-master ~]# yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo Repository extras is listed more than once in the configuration Adding repo from: https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo [root@k8s-master ~]# yum -y install docker-ce Repository extras is listed more than once in the configuration Docker CE Stable - x86_64 37 kB/s | 15 kB 00:00 Error: Problem: problem with installed package buildah-1.21.4-1.module_el8.4.0+886+c9a8d9ad.x86_64 - package buildah-1.21.4-1.module_el8.4.0+886+c9a8d9ad.x86_64 requires runc >= 1.0.0-26, but none of the providers can be installed - package buildah-1.19.7-2.module_el8.4.0+830+8027e1c4.x86_64 requires runc >= 1.0.0-26, but none of the providers can be installed - package buildah-1.19.7-1.module_el8.4.0+781+acf4c33b.x86_64 requires runc >= 1.0.0-26, but none of the providers can be installed - package containerd.io-1.4.3-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.3-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.3-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.4.3-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.4.3-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.4.3-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package docker-ce-3:20.10.8-3.el8.x86_64 requires containerd.io >= 1.4.1, but none of the providers can be installed - package containerd.io-1.4.3-3.2.el8.x86_64 conflicts with runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.3-3.2.el8.x86_64 obsoletes runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.3-3.2.el8.x86_64 conflicts with runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.4.3-3.2.el8.x86_64 obsoletes runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.4.3-3.2.el8.x86_64 conflicts with runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.4.3-3.2.el8.x86_64 obsoletes runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.4.4-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.4.4-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.4.6-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.4.6-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.4.8-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.4.8-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.4.9-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.4.9-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - cannot install the best candidate for the job - package runc-1.0.0-56.rc5.dev.git2abd837.module_el8.3.0+569+1bada2e4.x86_64 is filtered out by modular filtering - package runc-1.0.0-64.rc10.module_el8.4.0+522+66908d0c.x86_64 is filtered out by modular filtering - package runc-1.0.0-65.rc10.module_el8.4.0+819+4afbd1d6.x86_64 is filtered out by modular filtering - package runc-1.0.0-70.rc92.module_el8.4.0+786+4668b267.x86_64 is filtered out by modular filtering - package runc-1.0.0-71.rc92.module_el8.4.0+833+9763146c.x86_64 is filtered out by modular filtering - package containerd.io-1.4.4-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.4-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.4-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.4.4-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.4.6-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.6-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.6-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.4.6-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.4.8-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.8-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.8-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.4.8-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.4.9-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.9-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.4.9-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.4.9-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) [root@k8s-master ~]#
问题1：在执行yum -y install docker-ce时出现类似： package containerd.io-1.4.4-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64的错误。
解决方案：
从报错内容来看，应该是containerd.io包出现冲突，到containerd下载页面查看最新版本为containerd.io-1.3.7-3.1.el8.x86_64.rpm，重新下载该包，然后进行安装。
[root@k8s-master ~]# wget https://download.docker.com/linux/centos/8/x86_64/edge/Packages/containerd.io-1.3.7-3.1.el8.x86_64.rpm --2021-09-16 19:33:13-- https://download.docker.com/linux/centos/8/x86_64/edge/Packages/containerd.io-1.3.7-3.1.el8.x86_64.rpm Resolving download.docker.com (download.docker.com)... 18.65.191.124, 18.65.191.34, 18.65.191.111, ... Connecting to download.docker.com (download.docker.com)|18.65.191.124|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 30388860 (29M) [binary/octet-stream] Saving to: ‘containerd.io-1.3.7-3.1.el8.x86_64.rpm’ containerd.io-1.3.7-3.1.el8.x86_64.rpm 100%[=======================================================================================================>] 28.98M 1.20MB/s in 27s 2021-09-16 19:33:40 (1.08 MB/s) - ‘containerd.io-1.3.7-3.1.el8.x86_64.rpm’ saved [30388860/30388860] [root@k8s-master ~]# yum install containerd.io-1.3.7-3.1.el8.x86_64.rpm Repository extras is listed more than once in the configuration Last metadata expiration check: 0:09:59 ago on Thu 16 Sep 2021 07:23:49 PM CST. Error: Problem: problem with installed package buildah-1.21.4-1.module_el8.4.0+886+c9a8d9ad.x86_64 - package buildah-1.21.4-1.module_el8.4.0+886+c9a8d9ad.x86_64 requires runc >= 1.0.0-26, but none of the providers can be installed - package buildah-1.19.7-1.module_el8.4.0+781+acf4c33b.x86_64 requires runc >= 1.0.0-26, but none of the providers can be installed - package buildah-1.19.7-2.module_el8.4.0+830+8027e1c4.x86_64 requires runc >= 1.0.0-26, but none of the providers can be installed - package containerd.io-1.3.7-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.3.7-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 - package containerd.io-1.3.7-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.3.7-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-70.rc92.module_el8.4.0+673+eabfc99d.x86_64 - package containerd.io-1.3.7-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - package containerd.io-1.3.7-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86_64 - conflicting requests - package runc-1.0.0-56.rc5.dev.git2abd837.module_el8.3.0+569+1bada2e4.x86_64 is filtered out by modular filtering - package runc-1.0.0-64.rc10.module_el8.4.0+522+66908d0c.x86_64 is filtered out by modular filtering - package runc-1.0.0-65.rc10.module_el8.4.0+819+4afbd1d6.x86_64 is filtered out by modular filtering - package runc-1.0.0-70.rc92.module_el8.4.0+786+4668b267.x86_64 is filtered out by modular filtering - package runc-1.0.0-71.rc92.module_el8.4.0+833+9763146c.x86_64 is filtered out by modular filtering (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) [root@k8s-master ~]#
问题2：在执行yum install containerd.io-1.3.7-3.1.el8.x86_64.rpm时出现类似：Problem: problem with installed package buildah-1.21.4-1.module_el8.4.0+886+c9a8d9ad.x86_64的错误。
解决方案：执行以下命令
[root@k8s-master ~]# yum erase podman buildah Repository extras is listed more than once in the configuration No match for argument: podman Dependencies resolved. ================================================================================================================================================================================================= Package Architecture Version Repository Size ================================================================================================================================================================================================= Removing: buildah x86_64 1.21.4-1.module_el8.4.0+886+c9a8d9ad @appstream 30 M Removing unused dependencies: container-selinux noarch 2:2.164.1-1.module_el8.4.0+886+c9a8d9ad @appstream 47 k containers-common x86_64 1:1.3.1-5.module_el8.4.0+886+c9a8d9ad @appstream 231 k criu x86_64 3.15-1.module_el8.4.0+641+6116a774 @appstream 1.4 M fuse-overlayfs x86_64 1.6-1.module_el8.4.0+886+c9a8d9ad @appstream 145 k fuse3 x86_64 3.2.1-12.el8 @baseos 90 k fuse3-libs x86_64 3.2.1-12.el8 @baseos 279 k libnet x86_64 1.1.6-15.el8 @appstream 170 k libslirp x86_64 4.3.1-1.module_el8.4.0+575+63b40ad7 @appstream 129 k runc x86_64 1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad @appstream 12 M slirp4netns x86_64 1.1.8-1.module_el8.4.0+641+6116a774 @appstream 98 k Transaction Summary ================================================================================================================================================================================================= Remove 11 Packages Freed space: 44 M Is this ok [y/N]: y Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Erasing : buildah-1.21.4-1.module_el8.4.0+886+c9a8d9ad.x86_64 1/11 Erasing : containers-common-1:1.3.1-5.module_el8.4.0+886+c9a8d9ad.x86_64 2/11 Erasing : container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a8d9ad.noarch 3/11 Running scriptlet: container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a8d9ad.noarch 3/11 Erasing : fuse-overlayfs-1.6-1.module_el8.4.0+886+c9a8d9ad.x86_64 4/11 Erasing : slirp4netns-1.1.8-1.module_el8.4.0+641+6116a774.x86_64 5/11 Erasing : runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 6/11 Erasing : criu-3.15-1.module_el8.4.0+641+6116a774.x86_64 7/11 Erasing : libnet-1.1.6-15.el8.x86_64 8/11 Running scriptlet: libnet-1.1.6-15.el8.x86_64 8/11 Erasing : libslirp-4.3.1-1.module_el8.4.0+575+63b40ad7.x86_64 9/11 Erasing : fuse3-3.2.1-12.el8.x86_64 10/11 Erasing : fuse3-libs-3.2.1-12.el8.x86_64 11/11 Running scriptlet: fuse3-libs-3.2.1-12.el8.x86_64 11/11 Verifying : buildah-1.21.4-1.module_el8.4.0+886+c9a8d9ad.x86_64 1/11 Verifying : container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a8d9ad.noarch 2/11 Verifying : containers-common-1:1.3.1-5.module_el8.4.0+886+c9a8d9ad.x86_64 3/11 Verifying : criu-3.15-1.module_el8.4.0+641+6116a774.x86_64 4/11 Verifying : fuse-overlayfs-1.6-1.module_el8.4.0+886+c9a8d9ad.x86_64 5/11 Verifying : fuse3-3.2.1-12.el8.x86_64 6/11 Verifying : fuse3-libs-3.2.1-12.el8.x86_64 7/11 Verifying : libnet-1.1.6-15.el8.x86_64 8/11 Verifying : libslirp-4.3.1-1.module_el8.4.0+575+63b40ad7.x86_64 9/11 Verifying : runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 10/11 Verifying : slirp4netns-1.1.8-1.module_el8.4.0+641+6116a774.x86_64 11/11 Removed: buildah-1.21.4-1.module_el8.4.0+886+c9a8d9ad.x86_64 container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a8d9ad.noarch containers-common-1:1.3.1-5.module_el8.4.0+886+c9a8d9ad.x86_64 criu-3.15-1.module_el8.4.0+641+6116a774.x86_64 fuse-overlayfs-1.6-1.module_el8.4.0+886+c9a8d9ad.x86_64 fuse3-3.2.1-12.el8.x86_64 fuse3-libs-3.2.1-12.el8.x86_64 libnet-1.1.6-15.el8.x86_64 libslirp-4.3.1-1.module_el8.4.0+575+63b40ad7.x86_64 runc-1.0.0-74.rc95.module_el8.4.0+886+c9a8d9ad.x86_64 slirp4netns-1.1.8-1.module_el8.4.0+641+6116a774.x86_64 Complete!
以上问题都解决后，继续安装docker-ce，经过1-2分钟的等待后顺利完成安装。
[root@k8s-master ~]# yum -y install docker-ce Repository extras is listed more than once in the configuration Last metadata expiration check: 0:15:55 ago on Thu 16 Sep 2021 07:23:49 PM CST. Dependencies resolved. ================================================================================================================================================================================================= Package Architecture Version Repository Size ================================================================================================================================================================================================= Installing: docker-ce x86_64 3:20.10.8-3.el8 docker-ce-stable 22 M Installing dependencies: container-selinux noarch 2:2.164.1-1.module_el8.4.0+886+c9a8d9ad AppStream 52 k containerd.io x86_64 1.4.9-3.1.el8 docker-ce-stable 30 M docker-ce-cli x86_64 1:20.10.8-3.el8 docker-ce-stable 29 M docker-ce-rootless-extras x86_64 20.10.8-3.el8 docker-ce-stable 4.6 M docker-scan-plugin x86_64 0.8.0-3.el8 docker-ce-stable 4.2 M fuse-overlayfs x86_64 1.6-1.module_el8.4.0+886+c9a8d9ad AppStream 73 k fuse3 x86_64 3.2.1-12.el8 base 50 k fuse3-libs x86_64 3.2.1-12.el8 base 94 k libcgroup x86_64 0.41-19.el8 base 70 k libslirp x86_64 4.3.1-1.module_el8.4.0+575+63b40ad7 AppStream 69 k slirp4netns x86_64 1.1.8-1.module_el8.4.0+641+6116a774 AppStream 51 k Transaction Summary ================================================================================================================================================================================================= Install 12 Packages Total download size: 90 M Installed size: 377 M Downloading Packages: (1/12): fuse3-3.2.1-12.el8.x86_64.rpm 217 kB/s | 50 kB 00:00 (2/12): libcgroup-0.41-19.el8.x86_64.rpm 224 kB/s | 70 kB 00:00 (3/12): container-selinux-2.164.1-1.module_el8.4.0+886+c9a8d9ad.noarch.rpm 340 kB/s | 52 kB 00:00 (4/12): fuse3-libs-3.2.1-12.el8.x86_64.rpm 220 kB/s | 94 kB 00:00 (5/12): fuse-overlayfs-1.6-1.module_el8.4.0+886+c9a8d9ad.x86_64.rpm 407 kB/s | 73 kB 00:00 (6/12): libslirp-4.3.1-1.module_el8.4.0+575+63b40ad7.x86_64.rpm 411 kB/s | 69 kB 00:00 (7/12): slirp4netns-1.1.8-1.module_el8.4.0+641+6116a774.x86_64.rpm 296 kB/s | 51 kB 00:00 (8/12): docker-ce-20.10.8-3.el8.x86_64.rpm 409 kB/s | 22 MB 00:55 (9/12): docker-ce-cli-20.10.8-3.el8.x86_64.rpm 460 kB/s | 29 MB 01:05 (10/12): docker-ce-rootless-extras-20.10.8-3.el8.x86_64.rpm 419 kB/s | 4.6 MB 00:11 (11/12): docker-scan-plugin-0.8.0-3.el8.x86_64.rpm 436 kB/s | 4.2 MB 00:09 (12/12): containerd.io-1.4.9-3.1.el8.x86_64.rpm 373 kB/s | 30 MB 01:22 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 1.1 MB/s | 90 MB 01:22 warning: /var/cache/dnf/docker-ce-stable-ab4061364e2cf0db/packages/containerd.io-1.4.9-3.1.el8.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY Docker CE Stable - x86_64 11 kB/s | 1.6 kB 00:00 Importing GPG key 0x621E9F35: Userid : "Docker Release (CE rpm) <docker@docker.com>" Fingerprint: 060A 61C5 1B55 8A7F 742B 77AA C52F EB6B 621E 9F35 From : https://mirrors.aliyun.com/docker-ce/linux/centos/gpg Key imported successfully Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : docker-scan-plugin-0.8.0-3.el8.x86_64 1/12 Running scriptlet: docker-scan-plugin-0.8.0-3.el8.x86_64 1/12 Installing : docker-ce-cli-1:20.10.8-3.el8.x86_64 2/12 Running scriptlet: docker-ce-cli-1:20.10.8-3.el8.x86_64 2/12 Running scriptlet: container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a8d9ad.noarch 3/12 Installing : container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a8d9ad.noarch 3/12 Running scriptlet: container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a8d9ad.noarch 3/12 Installing : containerd.io-1.4.9-3.1.el8.x86_64 4/12 Running scriptlet: containerd.io-1.4.9-3.1.el8.x86_64 4/12 Installing : libslirp-4.3.1-1.module_el8.4.0+575+63b40ad7.x86_64 5/12 Installing : slirp4netns-1.1.8-1.module_el8.4.0+641+6116a774.x86_64 6/12 Running scriptlet: libcgroup-0.41-19.el8.x86_64 7/12 Installing : libcgroup-0.41-19.el8.x86_64 7/12 Running scriptlet: libcgroup-0.41-19.el8.x86_64 7/12 Installing : fuse3-libs-3.2.1-12.el8.x86_64 8/12 Running scriptlet: fuse3-libs-3.2.1-12.el8.x86_64 8/12 Installing : fuse3-3.2.1-12.el8.x86_64 9/12 Installing : fuse-overlayfs-1.6-1.module_el8.4.0+886+c9a8d9ad.x86_64 10/12 Running scriptlet: fuse-overlayfs-1.6-1.module_el8.4.0+886+c9a8d9ad.x86_64 10/12 Installing : docker-ce-3:20.10.8-3.el8.x86_64 11/12 Running scriptlet: docker-ce-3:20.10.8-3.el8.x86_64 11/12 Installing : docker-ce-rootless-extras-20.10.8-3.el8.x86_64 12/12 Running scriptlet: docker-ce-rootless-extras-20.10.8-3.el8.x86_64 12/12 Running scriptlet: container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a8d9ad.noarch 12/12 Running scriptlet: docker-ce-rootless-extras-20.10.8-3.el8.x86_64 12/12 Verifying : fuse3-3.2.1-12.el8.x86_64 1/12 Verifying : fuse3-libs-3.2.1-12.el8.x86_64 2/12 Verifying : libcgroup-0.41-19.el8.x86_64 3/12 Verifying : container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a8d9ad.noarch 4/12 Verifying : fuse-overlayfs-1.6-1.module_el8.4.0+886+c9a8d9ad.x86_64 5/12 Verifying : libslirp-4.3.1-1.module_el8.4.0+575+63b40ad7.x86_64 6/12 Verifying : slirp4netns-1.1.8-1.module_el8.4.0+641+6116a774.x86_64 7/12 Verifying : containerd.io-1.4.9-3.1.el8.x86_64 8/12 Verifying : docker-ce-3:20.10.8-3.el8.x86_64 9/12 Verifying : docker-ce-cli-1:20.10.8-3.el8.x86_64 10/12 Verifying : docker-ce-rootless-extras-20.10.8-3.el8.x86_64 11/12 Verifying : docker-scan-plugin-0.8.0-3.el8.x86_64 12/12 Installed: container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a8d9ad.noarch containerd.io-1.4.9-3.1.el8.x86_64 docker-ce-3:20.10.8-3.el8.x86_64 docker-ce-cli-1:20.10.8-3.el8.x86_64 docker-ce-rootless-extras-20.10.8-3.el8.x86_64 docker-scan-plugin-0.8.0-3.el8.x86_64 fuse-overlayfs-1.6-1.module_el8.4.0+886+c9a8d9ad.x86_64 fuse3-3.2.1-12.el8.x86_64 fuse3-libs-3.2.1-12.el8.x86_64 libcgroup-0.41-19.el8.x86_64 libslirp-4.3.1-1.module_el8.4.0+575+63b40ad7.x86_64 slirp4netns-1.1.8-1.module_el8.4.0+641+6116a774.x86_64 Complete! [root@k8s-master ~]#
4、为docker配置阿里云加速
可以登录阿里云官网地址，按照操作文档获取加速器的地址，如下图：
创建文件并编辑daemon.json
[root@k8s-master ~]# mkdir -p /etc/docker [root@k8s-master ~]# vim /etc/docker/daemon.json
在文件中设置第一步从阿里云获取到的加速地址
{ "registry-mirrors": ["https://uppmlc0d.mirror.aliyuncs.com"] }
以上docker-ce安装成功，接下来开始安装k8s。
5、安装kubectl、kubelet、kubeadm
安装kubectl、kubelet、kubeadm，设置kubelet开机启动，并启动kublet
[root@k8s-master ~]# yum install -y kubectl kubelet kubeadm
Repository extras is listed more than once in the configuration
Last metadata expiration check: 0:03:04 ago on Tue 28 Sep 2021 05:33:31 PM CST.
Dependencies resolved.
==================================================================================================
Package Architecture Version Repository Size
==================================================================================================
Installing:
kubeadm x86_64 1.22.2-0 kubernetes 9.3 M
kubectl x86_64 1.22.2-0 kubernetes 9.6 M
kubelet x86_64 1.22.2-0 kubernetes 23 M
Installing dependencies:
conntrack-tools x86_64 1.4.4-10.el8 base 204 k
cri-tools x86_64 1.13.0-0 kubernetes 5.1 M
kubernetes-cni x86_64 0.8.7-0 kubernetes 19 M
libnetfilter_cthelper x86_64 1.0.0-15.el8 base 24 k
libnetfilter_cttimeout x86_64 1.0.0-11.el8 base 24 k
libnetfilter_queue x86_64 1.0.4-3.el8 base 31 k
socat x86_64 1.7.3.3-2.el8 AppStream 302 k
Transaction Summary
==================================================================================================
Install 10 Packages
Total download size: 67 M
Installed size: 313 M
Downloading Packages:
(1/10): libnetfilter_cthelper-1.0.0-15.el8.x86_64.rpm 164 kB/s | 24 kB 00:00
(2/10): libnetfilter_cttimeout-1.0.0-11.el8.x86_64.rpm 152 kB/s | 24 kB 00:00
(3/10): libnetfilter_queue-1.0.4-3.el8.x86_64.rpm 226 kB/s | 31 kB 00:00
(4/10): socat-1.7.3.3-2.el8.x86_64.rpm 507 kB/s | 302 kB 00:00
(5/10): conntrack-tools-1.4.4-10.el8.x86_64.rpm 179 kB/s | 204 kB 00:01
(6/10): 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabd 369 kB/s | 5.1 MB 00:14
(7/10): 601174c7fbdf37f053d43088913525758704610e8036f0afd422d6e6a 466 kB/s | 9.3 MB 00:20
(8/10): 994be6998becbaa99f3c42cd8f2299364fb6f5c597b5ba1eb5db860d6 309 kB/s | 9.6 MB 00:31
(9/10): 80864433372b7120669c95335d54aedd2cb7e2002b41e5686e71d5605 569 kB/s | 23 MB 00:42
(10/10): db7cb5cb0b3f6875f54d10f02e625573988e3e91fd4fc5eef0b1876b 507 kB/s | 19 MB 00:37
--------------------------------------------------------------------------------------------------
Total 1.1 MB/s | 67 MB 00:58
warning: /var/cache/dnf/kubernetes-d03a9fe438e18cac/packages/14bfe6e75a9efc8eca3f638eb22c7e2ce759c 67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY
Kubernetes 39 kB/s | 3.4 kB 00:00
Importing GPG key 0x307EA071:
Userid : "Rapture Automatic Signing Key (cloud-rapture-signing-key-2021-03-01-08_01_09.pub)"
Fingerprint: 7F92 E05B 3109 3BEF 5A3C 2D38 FEEA 9169 307E A071
From : https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
Key imported successfully
Importing GPG key 0x836F4BEB:
Userid : "gLinux Rapture Automatic Signing Key (//depot/google3/production/borg/cloud-rapture /keys/cloud-rapture-pubkeys/cloud-rapture-signing-key-2020-12-03-16_08_05.pub) <glinux-team@google .com>"
Fingerprint: 59FE 0256 8272 69DC 8157 8F92 8B57 C5C2 836F 4BEB
From : https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
Key imported successfully
Kubernetes 11 kB/s | 975 B 00:00
Importing GPG key 0x3E1BA8D5:
Userid : "Google Cloud Packages RPM Signing Key <gc-team@google.com>"
Fingerprint: 3749 E1BA 95A8 6CE0 5454 6ED2 F09C 394C 3E1B A8D5
From : https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : kubectl-1.22.2-0.x86_64 1/10
Installing : cri-tools-1.13.0-0.x86_64 2/10
Installing : socat-1.7.3.3-2.el8.x86_64 3/10
Installing : libnetfilter_queue-1.0.4-3.el8.x86_64 4/10
Running scriptlet: libnetfilter_queue-1.0.4-3.el8.x86_64 4/10
Installing : libnetfilter_cttimeout-1.0.0-11.el8.x86_64 5/10
Running scriptlet: libnetfilter_cttimeout-1.0.0-11.el8.x86_64 5/10
Installing : libnetfilter_cthelper-1.0.0-15.el8.x86_64 6/10
Running scriptlet: libnetfilter_cthelper-1.0.0-15.el8.x86_64 6/10
Installing : conntrack-tools-1.4.4-10.el8.x86_64 7/10
Running scriptlet: conntrack-tools-1.4.4-10.el8.x86_64 7/10
Installing : kubernetes-cni-0.8.7-0.x86_64 8/10
Installing : kubelet-1.22.2-0.x86_64 9/10
Installing : kubeadm-1.22.2-0.x86_64 10/10
Running scriptlet: kubeadm-1.22.2-0.x86_64 10/10
Verifying : conntrack-tools-1.4.4-10.el8.x86_64 1/10
Verifying : libnetfilter_cthelper-1.0.0-15.el8.x86_64 2/10
Verifying : libnetfilter_cttimeout-1.0.0-11.el8.x86_64 3/10
Verifying : libnetfilter_queue-1.0.4-3.el8.x86_64 4/10
Verifying : socat-1.7.3.3-2.el8.x86_64 5/10
Verifying : cri-tools-1.13.0-0.x86_64 6/10
Verifying : kubeadm-1.22.2-0.x86_64 7/10
Verifying : kubectl-1.22.2-0.x86_64 8/10
Verifying : kubelet-1.22.2-0.x86_64 9/10
Verifying : kubernetes-cni-0.8.7-0.x86_64 10/10
Installed:
conntrack-tools-1.4.4-10.el8.x86_64 cri-tools-1.13.0-0.x86_64
kubeadm-1.22.2-0.x86_64 kubectl-1.22.2-0.x86_64
kubelet-1.22.2-0.x86_64 kubernetes-cni-0.8.7-0.x86_64
libnetfilter_cthelper-1.0.0-15.el8.x86_64 libnetfilter_cttimeout-1.0.0-11.el8.x86_64
libnetfilter_queue-1.0.4-3.el8.x86_64 socat-1.7.3.3-2.el8.x86_64
Complete!
[root@k8s-master ~]# systemctl enable kubelet Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /usr/lib/systemd/system/kubelet.service. [root@k8s-master ~]# systemctl start kubelet [root@k8s-master ~]#
查看k8s版本
[root@k8s-master ~]# kubeadm version kubeadm version: &version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-09-15T21:37:34Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"} [root@k8s-master ~]# kubectl version --client
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-09-15T21:38:50Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}
[root@k8s-master ~]# kubelet --version Kubernetes v1.22.2 [root@k8s-master ~]#
6、初始化k8s集群
[root@k8s-master ~]# kubeadm init --apiserver-advertise-address=0.0.0.0 --apiserver-cert-extra-sans=127.0.0.1 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.22.2 --service-cidr=10.10.0.0/16 --pod- network-cidr=10.18.0.0/16
问题1：初始化失败，通过查看报错信息，发现有如下错误：[ERROR ImagePull]: failed to pull image registry.aliyuncs.com/google_containers/coredns:v1.8.4: output: Error response from daemon: manifest for registry.aliyuncs.com/google_containers/coredns:v1.8.4 not found: manifest unknown: manifest unknown
, error: exit status 1。详细错误信息如下：
[init] Using Kubernetes version: v1.22.2 [preflight] Running pre-flight checks [WARNING FileExisting-tc]: tc not found in system path [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR ImagePull]: failed to pull image registry.aliyuncs.com/google_containers/coredns:v1.8.4: output: Error response from daemon: manifest for registry.aliyuncs.com/google_containers/coredns:v1.8.4 not found: manifest unknown: manifest unknown , error: exit status 1 [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` To see the stack trace of this error execute with --v=5 or higher
解决方案：从错误内容来看，应该是google_containers/coredns:v1.8.4这个镜像没有找到，执行docker images命令，确认该镜像文件确实没有。
[root@k8s-master ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE registry.aliyuncs.com/google_containers/kube-apiserver v1.22.2 e64579b7d886 4 weeks ago 128MB registry.aliyuncs.com/google_containers/kube-controller-manager v1.22.2 5425bcbd23c5 4 weeks ago 122MB registry.aliyuncs.com/google_containers/kube-scheduler v1.22.2 b51ddc1014b0 4 weeks ago 52.7MB registry.aliyuncs.com/google_containers/kube-proxy v1.22.2 873127efbc8a 4 weeks ago 104MB registry.aliyuncs.com/google_containers/etcd 3.5.0-0 004811815584 3 months ago 295MB registry.aliyuncs.com/google_containers/pause 3.5 ed210e3e4a5b 6 months ago 683kB
使用docker命令拉取镜像
[root@k8s-master ~]# docker pull registry.aliyuncs.com/google_containers/coredns:1.8.4 1.8.4: Pulling from google_containers/coredns c6568d217a00: Pull complete bc38a22c706b: Pull complete Digest: sha256:6e5a02c21641597998b4be7cb5eb1e7b02c0d8d23cce4dd09f4682d463798890 Status: Downloaded newer image for registry.aliyuncs.com/google_containers/coredns:1.8.4 registry.aliyuncs.com/google_containers/coredns:1.8.4
虽然此时该镜像文件已经有了，但是这里有个比较坑的地方，从控制台中的错误信息来看kubeadm初始化时需要的版本号是v1.8.4，而刚才拉取的镜像版本号是1.8.4，缺少了字母”v”，所以我们需要对镜像重新命名，然后删除原来的。最后再通过docker images命令发现该镜像文件已经有了。
该问题解决，继续初始化。。。
[root@k8s-master ~]# docker tag registry.aliyuncs.com/google_containers/coredns:1.8.4 registry.aliyuncs.com/google_containers/coredns:v1.8.4 [root@k8s-master ~]# docker rmi registry.aliyuncs.com/google_containers/coredns:1.8.4 Untagged: registry.aliyuncs.com/google_containers/coredns:1.8.4 [root@k8s-master ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE registry.aliyuncs.com/google_containers/kube-apiserver v1.22.2 e64579b7d886 4 weeks ago 128MB registry.aliyuncs.com/google_containers/kube-proxy v1.22.2 873127efbc8a 4 weeks ago 104MB registry.aliyuncs.com/google_containers/kube-controller-manager v1.22.2 5425bcbd23c5 4 weeks ago 122MB registry.aliyuncs.com/google_containers/kube-scheduler v1.22.2 b51ddc1014b0 4 weeks ago 52.7MB registry.aliyuncs.com/google_containers/etcd 3.5.0-0 004811815584 3 months ago 295MB registry.aliyuncs.com/google_containers/coredns v1.8.4 8d147537fb7d 3 months ago 47.6MB registry.aliyuncs.com/google_containers/pause 3.5 ed210e3e4a5b 6 months ago 683kB
问题2：初始化仍然失败，先是根据控制台提示信息，执行systemctl status kubelet发现服务启动报错，再根据控制台提示执行 journalctl -xeu kubelet命令，通过报错信息发现有如下错误：failed to run Kubelet: misconfiguration: kubelet cgroup driver: \"systemd\" is different from docker cgroup driver: >。应该是指cgroup driver版本不一致造成的。详细错误信息如下：
[root@k8s-master ~]# systemctl status kubelet ● kubelet.service - kubelet: The Kubernetes Node Agent Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/kubelet.service.d └─10-kubeadm.conf Active: activating (auto-restart) (Result: exit-code) since Fri 2021-09-17 10:43:46 CST; 5s ago Docs: https://kubernetes.io/docs/ Process: 98822 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=1/FAILURE) Main PID: 98822 (code=exited, status=1/FAILURE) [root@k8s-master ~]# journalctl -xeu kubelet -- Defined-By: systemd -- Support: https://access.redhat.com/support -- -- Unit kubelet.service has finished starting up. -- -- The start-up result is done. Sep 17 10:44:06 k8s-master kubelet[99004]: Flag --network-plugin has been deprecated, will be removed along with dockershim. Sep 17 10:44:06 k8s-master kubelet[99004]: Flag --network-plugin has been deprecated, will be removed along with dockershim. Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.567382 99004 server.go:440] "Kubelet version" kubeletVersion="v1.22.1" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.567676 99004 server.go:868] "Client rotation is on, will bootstrap in background" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.571664 99004 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem". Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.574461 99004 dynamic_cafile_content.go:155] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.640572 99004 server.go:687] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.640930 99004 container_manager_linux.go:280] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.641002 99004 container_manager_linux.go:285] "Creating Container Manager object based on Node Config" nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: > Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.641053 99004 topology_manager.go:133] "Creating topology manager with policy per scope" topologyPolicyName="none" topologyScopeName="container" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.641071 99004 container_manager_linux.go:320] "Creating device plugin manager" devicePluginEnabled=true Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.641126 99004 state_mem.go:36] "Initialized new in-memory state store" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.641197 99004 kubelet.go:314] "Using dockershim is deprecated, please consider using a full-fledged CRI implementation" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.641244 99004 client.go:78] "Connecting to docker on the dockerEndpoint" endpoint="unix:///var/run/docker.sock" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.641268 99004 client.go:97] "Start docker client with request timeout" timeout="2m0s" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.649280 99004 docker_service.go:566] "Hairpin mode is set but kubenet is not enabled, falling back to HairpinVeth" hairpinMode=promiscuous-bridge Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.649313 99004 docker_service.go:242] "Hairpin mode is set" hairpinMode=hairpin-veth Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.649405 99004 cni.go:239] "Unable to update cni config" err="no networks found in /etc/cni/net.d" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.651903 99004 cni.go:239] "Unable to update cni config" err="no networks found in /etc/cni/net.d" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.651960 99004 docker_service.go:257] "Docker cri networking managed by the network plugin" networkPluginName="cni" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.652018 99004 cni.go:239] "Unable to update cni config" err="no networks found in /etc/cni/net.d" Sep 17 10:44:06 k8s-master kubelet[99004]: I0917 10:44:06.659821 99004 docker_service.go:264] "Docker Info" dockerInfo=&{ID:VQUW:ZPAN:JTRU:ILYA:BRMI:KXJB:I4N2:4WV3:IMXL:3EH4:TMWH:FWGR Containers:0 ContainersRunning:0 ContainersPaus> Sep 17 10:44:06 k8s-master kubelet[99004]: E0917 10:44:06.659866 99004 server.go:294] "Failed to run kubelet" err="failed to run Kubelet: misconfiguration: kubelet cgroup driver: \"systemd\" is different from docker cgroup driver: > Sep 17 10:44:06 k8s-master systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE Sep 17 10:44:06 k8s-master systemd[1]: kubelet.service: Failed with result 'exit-code'. -- Subject: Unit failed -- Defined-By: systemd -- Support: https://access.redhat.com/support -- -- The unit kubelet.service has entered the 'failed' state with result 'exit-code'. lines 1753-1788/1788 (END)
既然原因清楚了，那么解决起来就方便了。首先，编辑/usr/lib/systemd/system/docker.service文件，找到ExecStart，在后面增加 --exec-opt native.cgroupdriver=systemd，然后重载配置文件并重启服务，最后通过docker info | grep Cgroup命令发现已经修改为systemd，问题得到解决。
具体如下：
[root@k8s-master system]# vim /usr/lib/systemd/system/docker.service [root@k8s-master system]# systemctl daemon-reload && systemctl restart docker [root@k8s-master system]# docker info | grep Cgroup Cgroup Driver: systemd Cgroup Version: 1 [root@k8s-master system]#
以上2个问题解决后，继续kubeadm init工作，终于出现了Your Kubernetes control-plane has initialized successfully!
说明已经初始化完成，这里控制台最后的kubeadm join 。。。最好是找个地方保存起来，因为后面node加入集群的时候会用到。
[root@k8s-master system]# kubeadm init --apiserver-advertise-address=0.0.0.0 --apiserver-cert-extra-sans=127.0.0.1 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.22.2 --service-cidr=10.10.0.0/16 --pod- network-cidr=10.18.0.0/16 [init] Using Kubernetes version: v1.22.2 [preflight] Running pre-flight checks [WARNING FileExisting-tc]: tc not found in system path [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.186.132 127.0.0.1] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.186.132 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.186.132 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [apiclient] All control plane components are healthy after 7.004367 seconds [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config-1.22" in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Skipping phase. Please see --upload-certs [mark-control-plane] Marking the node k8s-master as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] [mark-control-plane] Marking the node k8s-master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule] [bootstrap-token] Using token: en5aq7.2fnljgjetdr3ou5w [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.186.132:6443 --token en5aq7.2fnljgjetdr3ou5w \ --discovery-token-ca-cert-hash sha256:c4055de4f7fe4bef818e7a8dbede04a84ff75e6126d30d94deea28deee4abd82 [root@k8s-master system]#
按照控制台中提示的信息，执行以下命令：
[root@k8s-master ~]# mkdir -p $HOME/.kube [root@k8s-master ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config [root@k8s-master ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
再执行以下命令查看状态，此时为NotReady，这是因为coredns pod没有启动，缺少网络pod。
[root@k8s-master ~]# kubectl get node NAME STATUS ROLES AGE VERSION k8s-master NotReady control-plane,master 173m v1.22.2 [root@k8s-master ~]# kubectl get pod --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-7f6cbbb7b8-2rrnn 0/1 Pending 0 174m kube-system coredns-7f6cbbb7b8-h96bs 0/1 Pending 0 174m kube-system etcd-k8s-master 1/1 Running 0 175m kube-system kube-apiserver-k8s-master 1/1 Running 0 175m kube-system kube-controller-manager-k8s-master 1/1 Running 0 175m kube-system kube-proxy-tr5vg 1/1 Running 0 174m kube-system kube-scheduler-k8s-master 1/1 Running 0 175m
接下来安装calico网络，安装完成稍等一会再查看信息，节点已经处于Ready状态。
[root@k8s-master ~]# kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml configmap/calico-config created customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created clusterrole.rbac.authorization.k8s.io/calico-node created clusterrolebinding.rbac.authorization.k8s.io/calico-node created daemonset.apps/calico-node created serviceaccount/calico-node created deployment.apps/calico-kube-controllers created serviceaccount/calico-kube-controllers created Warning: policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget poddisruptionbudget.policy/calico-kube-controllers created [root@k8s-master ~]# ^C [root@k8s-master ~]# kubectl get node NAME STATUS ROLES AGE VERSION k8s-master Ready control-plane,master 3h2m v1.22.1 [root@k8s-master ~]# kubectl get pod --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-74b8fbdb46-xjgf4 1/1 Running 0 6m32s kube-system calico-node-v2rnl 1/1 Running 0 6m32s kube-system coredns-7f6cbbb7b8-2rrnn 1/1 Running 0 3h1m kube-system coredns-7f6cbbb7b8-h96bs 1/1 Running 0 3h1m kube-system etcd-k8s-master 1/1 Running 0 3h2m kube-system kube-apiserver-k8s-master 1/1 Running 0 3h2m kube-system kube-controller-manager-k8s-master 1/1 Running 0 3h2m kube-system kube-proxy-tr5vg 1/1 Running 0 3h1m kube-system kube-scheduler-k8s-master 1/1 Running 0 3h2m [root@k8s-master ~]#
k8s集群的master已经安装完成，下一篇介绍kubernetes-dashboard的安装与配置。
~~~未完待续
