Ubuntu22.04 安装 Kubernetes
一、安装docker.io
sudo apt install docker.io
关闭swap:
swapoff -a
在文件/etc/fstab注释 :
#/swapfile none swap sw 0 0
二、安装k8s
1、安装问题1
Err:2 https://packages.cloud.google.com/apt kubernetes-xenial InRelease Could not connect to packages.cloud.google.com:443 (172.217.160.78), connection timed out
解决：
https://packages.cloud.google.com/apt/doc/apt-key.gpg
上面链接手动下载，命令为kubernetes-archive-keyring.gpg
/usr/share/keyrings/kubernetes-archive-keyring.gpg
2、安装源问题
Err:5 https://packages.cloud.google.com/apt kubernetes-xenial InRelease Could not connect to packages.cloud.google.com:443 (172.217.160.110), connection timed out Fetched 99.8 kB in 39s (2,578 B/s) Reading package lists... Done W: Failed to fetch http://apt.kubernetes.io/dists/kubernetes-xenial/InRelease Could not connect to packages.cloud.google.com:443 (172.217.160.110), connection timed out W: Some index files failed to download. They have been ignored, or old ones used instead. root@ubuntu00:/etc/apt# ls
替换安装源
由
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
改为
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] http://mirrors.ustc.edu.cn/kubernetes/apt kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
3、安装
apt-get install kubelet=1.23.6-00
apt-get install kubeadm=1.23.6-00
apt-get install kubectl=1.23.6-00
4、设置节点名称
hostnamectl set-hostname master
在 /etc/hosts 添加主机名映射
192.168.2.108 master
5、使用阿里云下载源
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.23.8 docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.23.8 docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.23.8 docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.23.8 docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6 docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.5.1-0 docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:v1.8.6 docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.23.8 k8s.gcr.io/kube-apiserver:v1.23.8 docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.23.8 docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.23.8 k8s.gcr.io/kube-controller-manager:v1.23.8 docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.23.8 docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.23.8 k8s.gcr.io/kube-scheduler:v1.23.8 docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.23.8 docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.23.8 k8s.gcr.io/kube-proxy:v1.23.8 docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.23.8 docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6 k8s.gcr.io/pause:3.6 docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6 docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.5.1-0 k8s.gcr.io/etcd:3.5.1-0 docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.5.1-0 docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:v1.8.6 k8s.gcr.io/coredns/coredns:v1.8.6 docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:v1.8.6
6.修改配置
kubeadm config print init-defaults > kubeadm.conf
修改 kubeadm.conf
bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token token: abcdef.0123456789abcdef ttl: 24h0m0s usages: - signing - authentication kind: InitConfiguration localAPIEndpoint: advertiseAddress: 192.168.2.108 # 修改 bindPort: 6443 nodeRegistration: criSocket: /var/run/dockershim.sock imagePullPolicy: IfNotPresent name: master # 修改 taints: null --- apiServer: timeoutForControlPlane: 4m0s apiVersion: kubeadm.k8s.io/v1beta3 certificatesDir: /etc/kubernetes/pki clusterName: kubernetes controllerManager: {} dns: {} etcd: local: dataDir: /var/lib/etcd imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers # 修改 kind: ClusterConfiguration kubernetesVersion: 1.23.0 networking: dnsDomain: cluster.local
podSubnet: 10.244.0.0/16 # 新增 serviceSubnet: 10.96.0.0/12 scheduler: {}
7、初始化
kubeadm init --config=kubeadm.conf
产生日志
[init] Using Kubernetes version: v1.23.0 [preflight] Running pre-flight checks [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local master] and IPs [10.96.0.1 192.168.2.108] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [localhost master] and IPs [192.168.2.108 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [localhost master] and IPs [192.168.2.108 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [apiclient] All control plane components are healthy after 4.003269 seconds [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config-1.23" in namespace kube-system with the configuration for the kubelets in the cluster NOTE: The "kubelet-config-1.23" naming of the kubelet ConfigMap is deprecated. Once the UnversionedKubeletConfigMap feature gate graduates to Beta the default name will become just "kubelet-config". Kubeadm upgrade will handle this transition transparently. [upload-certs] Skipping phase. Please see --upload-certs [mark-control-plane] Marking the node master as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] [mark-control-plane] Marking the node master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule] [bootstrap-token] Using token: abcdef.0123456789abcdef [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.2.108:6443 --token abcdef.0123456789abcdef \ --discovery-token-ca-cert-hash sha256:df36cdfb3397daeacdb335d0eca894fa69a93ba1782daf10e814d1b1f76a87c5
token 下面登录有用到 abcdef.0123456789abcdef
8、配置静态ip
修改之前动态：
network: version: 2 renderer: NetworkManager
ubuntu 22.04版本：
➜ ~ cat /etc/netplan/01-network-manager-all.yaml # Let NetworkManager manage all devices on this system network: version: 2 ethernets: enp0s3: dhcp4: no dhcp6: no addresses: - 192.168.2.108/24 routes: - to: default via: 192.168.2.1 nameservers: addresses: - 114.114.114.114 - 8.8.8.8 renderer: networkd
ubuntu 20.04/18.04版本：
# Let NetworkManager manage all devices on this system network: version: 2 ethernets: enp0s3: dhcp4: no addresses: [192.168.2.108/24] optional: true gateway4: 192.168.2.1 nameservers: addresses: [114.114.114.114, 8.8.8.8] renderer: networkd
最后执行：
netplan apply
这样IP就不会随着每次启动服务器变化
参考： https://blog.csdn.net/qq_36393978/article/details/124868232
9. 安装flannel
kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml
kubectl get pods --all-namespaces
出现问题：
1) Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
2) The connection to the server localhost:8080 was refused - did you specify the right host or port?
问题解决：
https://kubernetes.io/zh-cn/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/
-
使用如下方法取消设置
KUBECONFIG环境变量的值：unset KUBECONFIG或者将其设置为默认的
KUBECONFIG位置：export KUBECONFIG=/etc/kubernetes/admin.conf -
另一个方法是覆盖
kubeconfig的现有用户 "管理员"：mv $HOME/.kube $HOME/.kube.bak mkdir $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
正常：
10、安装dashboard
https://github.com/kubernetes/dashboard
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.0/aio/deploy/recommended.yaml
kubectl proxy
访问地址:
http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
token登录：
abcdef.0123456789abcdef
重新获取dashboard token, 获取
deployment-controller-token-m64x2 组件token
root@master:~# kubectl -n kube-system get secret NAME TYPE DATA AGE attachdetach-controller-token-srb2r kubernetes.io/service-account-token 3 27h bootstrap-signer-token-b9krc kubernetes.io/service-account-token 3 27h certificate-controller-token-88htd kubernetes.io/service-account-token 3 27h clusterrole-aggregation-controller-token-dgqtv kubernetes.io/service-account-token 3 27h coredns-token-8lsr4 kubernetes.io/service-account-token 3 27h cronjob-controller-token-4l7wz kubernetes.io/service-account-token 3 27h daemon-set-controller-token-j9m55 kubernetes.io/service-account-token 3 27h default-token-n72vg kubernetes.io/service-account-token 3 27h deployment-controller-token-m64x2 kubernetes.io/service-account-token 3 27h disruption-controller-token-v92lq kubernetes.io/service-account-token 3 27h endpoint-controller-token-4g6sn kubernetes.io/service-account-token 3 27h endpointslice-controller-token-f9q5k kubernetes.io/service-account-token 3 27h endpointslicemirroring-controller-token-wv87c kubernetes.io/service-account-token 3 27h ephemeral-volume-controller-token-xhqpg kubernetes.io/service-account-token 3 27h expand-controller-token-zfk7s kubernetes.io/service-account-token 3 27h generic-garbage-collector-token-q4b82 kubernetes.io/service-account-token 3 27h horizontal-pod-autoscaler-token-s75jq kubernetes.io/service-account-token 3 27h job-controller-token-5vpgv kubernetes.io/service-account-token 3 27h kube-proxy-token-m9sll kubernetes.io/service-account-token 3 27h namespace-controller-token-wv5n7 kubernetes.io/service-account-token 3 27h node-controller-token-6x8mx kubernetes.io/service-account-token 3 27h persistent-volume-binder-token-sbrp8 kubernetes.io/service-account-token 3 27h pod-garbage-collector-token-kkccv kubernetes.io/service-account-token 3 27h pv-protection-controller-token-bfpwz kubernetes.io/service-account-token 3 27h pvc-protection-controller-token-hz6sq kubernetes.io/service-account-token 3 27h replicaset-controller-token-p8mv8 kubernetes.io/service-account-token 3 27h replication-controller-token-dmxm7 kubernetes.io/service-account-token 3 27h resourcequota-controller-token-lff72 kubernetes.io/service-account-token 3 27h root-ca-cert-publisher-token-nfkv5 kubernetes.io/service-account-token 3 27h service-account-controller-token-9zsvf kubernetes.io/service-account-token 3 27h service-controller-token-zmg8n kubernetes.io/service-account-token 3 27h statefulset-controller-token-jwzw6 kubernetes.io/service-account-token 3 27h token-cleaner-token-xh98r kubernetes.io/service-account-token 3 27h ttl-after-finished-controller-token-ndvw4 kubernetes.io/service-account-token 3 27h ttl-controller-token-ljm5h kubernetes.io/service-account-token 3 27h root@master:~# kubectl -n kube-system get secret | grep controller attachdetach-controller-token-srb2r kubernetes.io/service-account-token 3 27h certificate-controller-token-88htd kubernetes.io/service-account-token 3 27h clusterrole-aggregation-controller-token-dgqtv kubernetes.io/service-account-token 3 27h cronjob-controller-token-4l7wz kubernetes.io/service-account-token 3 27h daemon-set-controller-token-j9m55 kubernetes.io/service-account-token 3 27h deployment-controller-token-m64x2 kubernetes.io/service-account-token 3 27h disruption-controller-token-v92lq kubernetes.io/service-account-token 3 27h endpoint-controller-token-4g6sn kubernetes.io/service-account-token 3 27h endpointslice-controller-token-f9q5k kubernetes.io/service-account-token 3 27h endpointslicemirroring-controller-token-wv87c kubernetes.io/service-account-token 3 27h ephemeral-volume-controller-token-xhqpg kubernetes.io/service-account-token 3 27h expand-controller-token-zfk7s kubernetes.io/service-account-token 3 27h job-controller-token-5vpgv kubernetes.io/service-account-token 3 27h namespace-controller-token-wv5n7 kubernetes.io/service-account-token 3 27h node-controller-token-6x8mx kubernetes.io/service-account-token 3 27h pv-protection-controller-token-bfpwz kubernetes.io/service-account-token 3 27h pvc-protection-controller-token-hz6sq kubernetes.io/service-account-token 3 27h replicaset-controller-token-p8mv8 kubernetes.io/service-account-token 3 27h replication-controller-token-dmxm7 kubernetes.io/service-account-token 3 27h resourcequota-controller-token-lff72 kubernetes.io/service-account-token 3 27h service-account-controller-token-9zsvf kubernetes.io/service-account-token 3 27h service-controller-token-zmg8n kubernetes.io/service-account-token 3 27h statefulset-controller-token-jwzw6 kubernetes.io/service-account-token 3 27h ttl-after-finished-controller-token-ndvw4 kubernetes.io/service-account-token 3 27h ttl-controller-token-ljm5h kubernetes.io/service-account-token 3 27h root@master:~# kubectl -n kube-system get secret | grep deployement-controller root@master:~# kubectl -n kube-system get secret | grep deployement root@master:~# kubectl -n kube-system get secret deployment-controller-token-m64x2 NAME TYPE DATA AGE deployment-controller-token-m64x2 kubernetes.io/service-account-token 3 27h root@master:~# kubectl -n kube-system describe secret deployment-controller-token-m64x2 Name: deployment-controller-token-m64x2 Namespace: kube-system Labels: <none> Annotations: kubernetes.io/service-account.name: deployment-controller kubernetes.io/service-account.uid: 33e9b6e5-127f-4431-8250-b4430c5a28cb Type: kubernetes.io/service-account-token Data ==== token: eyJhbGciOiJSUzI1NiIsImtpZCI6IjdrUTNsVFpBc21WamdmeWtBSk4wcDJRQWlYTVFZd1kzRTlfc0Y1WjFDQ0EifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZXBsb3ltZW50LWNvbnRyb2xsZXItdG9rZW4tbTY0eDIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGVwbG95bWVudC1jb250cm9sbGVyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMzNlOWI2ZTUtMTI3Zi00NDMxLTgyNTAtYjQ0MzBjNWEyOGNiIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRlcGxveW1lbnQtY29udHJvbGxlciJ9.X5YA68bwELziyQSRcUeJ1SdXsHFfkJbR0O5nuiLfjM7MGYfrwDGJ47uYgKqUB-REN-eY7Dad1_LgvZH5MbQuhGvE6MLtM6b65BZ_ap9IZtt60u9f0IcPFVGHwpS_ERkalRus-f3XK8vL42C6YLAK_k3lcfiHFjeKr8pgGChN-V7tOVq8zh6y1iN7u_K0ZjTbT1JAwp8DZ9LxRvVe4bZ727X0KhPD2kfORrcgqdQwglj9j7fLrPWaQsdPEDZ-zmOyLucmFQbynE8UnArZMVn4y1KJH1aNU1ACMve7S8NeGfqKa6wdewgA5x9lLynbxDY3nNxPD4_ivTms5jE5LsYjew ca.crt: 1099 bytes namespace: 11 bytes root@master:~#
11, 添加节点
另外启动一个ubuntu 节点，命名为 node01
按1-5步骤安装 kubectl kubeadm kubelet
修改为静态ip
12、在master节点创建token
kubeadm token create
root@master:~# kubeadm token list TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS amek9z.4jznc1u5hkgb5n3r 23h 2023-03-15T15:48:43Z authentication,signing <none> system:bootstrappers:kubeadm:default-node-token
可以看到token 为 amek9z.4jznc1u5hkgb5n3r ， 时效为24h
13、创建discovery-token-ca-cert-hash
在master上创建
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
8f0bc0744acdbee8d548f1f34fe120bf630def1c36382a232f02f8ba36d610c5
14、节点node01 join 集群
在节点node01执行
root@node01:~# kubeadm join 192.168.1.108:6443 --token amek9z.4jznc1u5hkgb5n3r --discovery-token-ca-cert-hash sha256:8f0bc0744acdbee8d548f1f34fe120bf630def1c36382a232f02f8ba36d610c5
如无异常，需要等待一会，节点状态才能从notReady转化为Ready状态
15、master节点查看节点
root@master:~# kubectl get nodes NAME STATUS ROLES AGE VERSION master Ready control-plane,master 177d v1.23.6 node01 Ready <none> 22m v1.23.6
至此，集群搭建完毕
参考：
https://skyao.io/learning-kubernetes/docs/installation/kubeadm/join.html
https://segmentfault.com/a/1190000012755243
