kubelet failed with kubelet cgroup driver: "cgroupfs" is different from docker cgroup driver: "systemd"

问题：

$ kubectl get nodes

The connection to the server host:6443 was refused - did you specify the right host or port

查看日志

$ sudo journalctl -xu kubelet.service

Mar 01 03:23:29 k8s-master kubelet[691]: E0301 03:23:29.855015     691 server.go:302] "Failed to run kubelet" err="failed to run Kubelet: misconfiguration: kubelet cgroup driver: \"systemd\" is different from docker cgroup driver: \"cgroupfs\""
Mar 01 03:23:29 k8s-master systemd[1]: kubelet.service: main process exited, code=exited, status=1/FAILURE
Mar 01 03:23:29 k8s-master systemd[1]: Unit kubelet.service entered failed state.
Mar 01 03:23:29 k8s-master systemd[1]: kubelet.service failed.
Mar 01 03:23:39 k8s-master systemd[1]: kubelet.service holdoff time over, scheduling restart.
Mar 01 03:23:39 k8s-master systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
-- Subject: Unit kubelet.service has finished shutting down

解决方案

Step 1

It may be better to do the reverse and make kubelet to use systemd

In Kubernetes site, they recommend using systemd https://kubernetes.io/docs/setup/production-environment/container-runtimes/ More details here (https://stackoverflow.com/a/60630294/429476)

And you can change kubelet to use systemd by following https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/ adding this to /etc/sysconfig/kubelet

cat /etc/sysconfig/kubelet 
KUBELET_EXTRA_ARGS=--cgroup-driver=systemd

Step 2

https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/

Using the cgroupfs driver As this guide explains using the cgroupfs driver with kubeadm is not recommended.

To continue using cgroupfs and to prevent kubeadm upgrade from modifying the KubeletConfiguration cgroup driver on existing setups, you must be explicit about its value. This applies to a case where you do not wish future versions of kubeadm to apply the systemd driver by default.

See the below section on "Modify the kubelet ConfigMap" for details on how to be explicit about the value.

If you wish to configure a container runtime to use the cgroupfs driver, you must refer to the documentation of the container runtime of your choice

Go to change config docker cgroupfs to systemd

edit: /etc/docker/daemon.json （if not exit, create this file)

{
  "exec-opts": ["native.cgroupdriver=systemd"]
}

restart service docker

$ systemctl daemon-reload && systemctl restart docker && systemctl restart kubelet

show the config in

$ docker info | grep Cgroup

 Cgroup Driver: systemd
 Cgroup Version: 1