kubernetes1.20版本启用ipvs模式
在1.19版本之前,kubeadm部署方式启用ipvs模式时,初始化配置文件需要添加以下内容:
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration featureGates: SupportIPVSProxyMode: true mode: ipvs
本次在1.20.2版本中,使用kubeadm进行集群初始化时,虽然可以正常部署,但是查看pod情况的时候可以看到kube-proxy无法运行成功,报错部分内容如下:
]# kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
....#省略其他输出内容
kube-system kube-proxy-7vrbv 0/1 CrashLoopBackOff 9 43m
kube-system kube-proxy-ghs7h 0/1 CrashLoopBackOff 9 43m
kube-system kube-proxy-l9twb 0/1 CrashLoopBackOff 1 7s
kube-system kube-proxy-mzfrf 0/1 CrashLoopBackOff 9 42m
kube-system kube-proxy-nxpls 0/1 CrashLoopBackOff 9 3h4m
kube-system kube-proxy-pmmtq 0/1 CrashLoopBackOff 8 42m
#查看日志信息
]# kubectl logs kube-proxy-l9twb -n kube-system F0114 12:58:34.042769 1 server.go:488] failed complete: unrecognized feature gate: SupportIPVSProxyMode goroutine 1 [running]: k8s.io/kubernetes/vendor/k8s.io/klog/v2.stacks(0xc00000e001, 0xc0004b6000, 0x6e, 0xc0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:1026 +0xb9 k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).output(0x29b65c0, 0xc000000003, 0x0, 0x0, 0xc0003d8230, 0x28edbc9, 0x9, 0x1e8, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:975 +0x19b k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).printf(0x29b65c0, 0xc000000003, 0x0, 0x0, 0x0, 0x0, 0x1b3a573, 0x13, 0xc000431310, 0x1, ...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:750 +0x191 k8s.io/kubernetes/vendor/k8s.io/klog/v2.Fatalf(...)
通过报错可以看到kube-proxy无法识别SupportIPVSProxyMode这个字段,于是访问官方查看最新版本ipvs开启的正确配置,通过https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/README.md可以看到官方说明:
Cluster Created by Kubeadm
If you are using kubeadm with a configuration file, you have to add mode: ipvs below the kubeProxy field as part of the kubeadm configuration.
...
kubeProxy:
config:
mode: ipvs
...
由于集群已经初始化成功了,所以现在改kubeadm初始化配置文件没有意义,因为我们需要直接修改kube-proxy的启动配置
通过查看kube-pxory的资源清单可以知道, kube-proxy的配置文件是通过configmap方式挂载到容器中的,因此我们只需要对应修改configmap中的配置内容,就可以将无效字段删除
]# kubectl -n kube-system get pod kube-proxy-24tkb -o yaml apiVersion: v1 kind: Pod metadata: ..... #其他内容省略 containers: - command: - /usr/local/bin/kube-proxy - --config=/var/lib/kube-proxy/config.conf - --hostname-override=$(NODE_NAME) ..... #其他内容省略 volumeMounts: - mountPath: /var/lib/kube-proxy name: kube-proxy ..... #其他内容省略 volumes: - configMap: defaultMode: 420 name: kube-proxy name: kube-proxy
]# kubectl get cm -n kube-system NAME DATA AGE coredns 1 5h18m extension-apiserver-authentication 6 5h18m kube-proxy 2 5h18m kube-root-ca.crt 1 5h18m kubeadm-config 2 5h18m kubelet-config-1.20 1 5h18m
]# kubectl edit cm kube-proxy -n kube-system
#在编辑模式中找到以下字段,删除后保存退出
featureGates:
SupportIPVSProxyMode: true
然后将删除所有kube-proxy进行重启,查看pod运行情况
]# kubectl get pod -n kube-system NAME READY STATUS RESTARTS AGE ... #其他内容省略 kube-proxy-24tkb 1/1 Running 0 122m kube-proxy-9cl7j 1/1 Running 0 123m kube-proxy-cxbg5 1/1 Running 0 123m kube-proxy-cxgqk 1/1 Running 0 123m kube-proxy-hkq54 1/1 Running 0 123m kube-proxy-ttdqb 1/1 Running 0 122m
在服务器上安装ipvsadm,查看ipvs模式是否启用成功
]# yum install ipvsadm -y ]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.96.0.1:443 rr -> 和谐ip:6443 Masq 1 0 0 -> 和谐ip:6443 Masq 1 0 0 -> 和谐ip:6443 Masq 1 0 0 TCP 10.96.0.10:53 rr TCP 10.96.0.10:9153 rr UDP 10.96.0.10:53 rr
