k8s使用cifs创建pv,某些节点不能挂载

1.测试环境有一个节点不能挂载

从报错可以看到pod中的挂载点已经存在了，不能创建

RunContainerError (failed to start container "0ea31f12b87474dbd125a3999114afae53fb92b2c1106d5e65110170e3f3ac80": Error response from daemon: error while creating mount source path '/var/lib/kubelet/pods/a0098332-f18f-4566-804d-9cd801439376/volumes/kubernetes.io~csi/pvc-smb-jjfa/mount': mkdir /var/lib/kubelet/pods/a0098332-f18f-4566-804d-9cd801439376/volumes/kubernetes.io~csi/pvc-smb-jjfa/mount: file exists) | Last state: Terminated with 128: ContainerCannotRun (error while creating mount source path '/var/lib/kubelet/pods/a0098332-f18f-4566-804d-9cd801439376/volumes/kubernetes.io~csi/pvc-smb-jjfa/mount': mkdir /var/lib/kubelet/pods/a0098332-f18f-4566-804d-9cd801439376/volumes/kubernetes.io~csi/pvc-smb-jjfa/mount: file exists), started: Tue, Jan 23 2024 9:32:58 am, finished: Tue, Jan 23 2024 9:32:58 am


CrashLoopBackOff (back-off 10s restarting failed container=nginx1 pod=nginx1-65b8d4cf48-vl6q7_wfs-test(a0098332-f18f-4566-804d-9cd801439376)) | Last state: Terminated with 128: ContainerCannotRun (error while creating mount source path '/var/lib/kubelet/pods/a0098332-f18f-4566-804d-9cd801439376/volumes/kubernetes.io~csi/pvc-smb-jjfa/mount': mkdir /var/lib/kubelet/pods/a0098332-f18f-4566-804d-9cd801439376/volumes/kubernetes.io~csi/pvc-smb-jjfa/mount: file exists), started: Tue, Jan 23 2024 9:32:47 am, finished: Tue, Jan 23 2024 9:32:47 am

查看插件中的cifs可以用看到

/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-smb-jjfa/globalmount

查看其详细信息，显示如下，且root账户无法更改属主或者权限，报错无法访问"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-smb-jjfa/globalmount": 权限不够

d????????? ? ?    ?     ?            ? globalmount 

处理方法，重新创建一个新的pv及pvc,新的pvc可以在任何挂载点进行挂载

2.生产环境，第一次pvc挂载在哪个节点后面就只能挂载在哪个节点，其他节点报错

MountVolume.MountDevice failed for volume "pv-smb-jjfa" : rpc error: code = Internal desc = volume(smb-server.default.svc.cluster.local/sharejjfa##) mount "//10.1.253.15/jjfa" on "/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pv-smb-jjfa/globalmount" failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t cifs -o dir_mode=0777,file_mode=0777,<masked> //10.1.253.15/jjfa /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pv-smb-jjfa/globalmount Output: mount error(5): Input/output error Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel log messages (dmesg)

查看各个节点cifs的pvc目录,发现只有第一次挂载的节点里面有pvc-smb-jjfa，其他的都没有cifs的挂载

/var/lib/kubelet/plugins/kubernetes.io/csi/pv

由于测试环境的cifs是都可以创建的，所以首先将集群中的csi-smb-controller和csi-smb-node的镜像都升级，然后对比两个集群中的csi-smb-controller，发现两个cis控制插件的网络模式不一致，生产环境为Default,测试环境为ClusterFirstWithHostNet，修改网络模式，然后创建新pv及pvc,所有节点可挂载，解决了这个问题。