Podman 故障处理记录

1- 1.podman 遇到there might not be enough IDs available in the namespace

1.1- 发现错误

在jenkins pipeline中配置docker,拉去镜像时发现以下错误

+ docker inspect -f . registry.cn-shenzhen.aliyuncs.com/ccapp/maven:3.6.1-jdk-8
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
time="2020-12-20T11:16:21+08:00" level=error msg="cannot find UID/GID for user jenkins: No subuid ranges found for user \"jenkins\" in /etc/subuid - check rootless mode in man pages."
Error: error inspecting object: no such object: "registry.cn-shenzhen.aliyuncs.com/ccapp/maven:3.6.1-jdk-8"
[Pipeline] isUnix
[Pipeline] sh
+ docker pull registry.cn-shenzhen.aliyuncs.com/ccapp/maven:3.6.1-jdk-8
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
Trying to pull registry.cn-shenzhen.aliyuncs.com/ccapp/maven:3.6.1-jdk-8...
Getting image source signatures
Copying blob sha256:22b3d1b850808b26226842403e3f57439c41bc577f06e4d3597a8ded282a6734
Copying blob sha256:f145544f1adc64dc129d42a19fa91e78d38d4c6103c4611aa15efee87d226a3d
Copying blob sha256:f0543937d580f9242fec6b84f70a9c0e962e37db6f56c6386725820b83b90ff6
Copying blob sha256:ae8bc0cc0ce1d7af41b15ff5d4570f4287ba5d6ccf8532d3e6652e8596b7243a
Copying blob sha256:169185f82c45a6eb72e0ca4ee66152626e7ace92a0cbc53624fb46d0a553f0bd
Copying blob sha256:a3ce0abc68f2b281aac8b348b9d8e26c57825ca5be3de9551314cd77f3e9b654
Copying config sha256:c4c2999f247ba204c056733b881fc49db3d927ccceb845a3b4c51c2c48d0c154
Writing manifest to image destination
Storing signatures
  Error processing tar file(exit status 1): there might not be enough IDs available in the namespace (requested 0:42 for /etc/shadow): lchown /etc/shadow: invalid argument
Error: unable to pull registry.cn-shenzhen.aliyuncs.com/ccapp/maven:3.6.1-jdk-8: Error committing the finished image: error adding layer with blob "sha256:169185f82c45a6eb72e0ca4ee66152626e7ace92a0cbc53624fb46d0a553f0bd": Error processing tar file(exit status 1): there might not be enough IDs available in the namespace (requested 0:42 for /etc/shadow): lchown /etc/shadow: invalid argument

1.2- 解决问题

[root@localhost ~]# cat /etc/passwd|grep jenkins
jenkins:x:988:984:Jenkins Automation Server:/var/lib/jenkins:/bin/bash


[root@localhost ~]# echo jenkins:988:984 > /etc/subuid
[root@localhost ~]# echo jenkins:988:984 > /etc/subgid
[root@localhost ~]# podman system migrate

1.3- 总结

经过重复测试后，发现解决这种问题还是要先解决namespace分配的问题，正确的步骤应该这样.

• 检查现有用户的UID和GID，并且找出最大的ID。

[root@localhost ~]# cat /etc/passwd|awk -F ':' '{print $3,$4}'|sort
0 0
1 1
11 0
12 100
14 50
193 193
2 2
3 4
4 7
5 0
59 59
6 0
65534 65534
7 0
72 72
74 74
8 12
81 81
983 979
984 980
985 981
986 982
987 983
988 984
989 985
990 986
991 987
992 988
993 989
994 990
995 991
996 993
997 995
998 996
999 997
[root@localhost ~]#
[root@localhost ~]# cat /etc/group|awk -F ':' '{print $3}'|sort
0
1
10
100
11
12
15
18
19
190
193
2
20
21
22
3
33
35
36
39
4
5
50
54
59
6
63
65534
7
72
74
8
81
9
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999

• 检查现有的/etc/subuid和/etc/subgid

[root@localhost ~]# cat /etc/subgid /etc/subgid

• 为jenkins用户配置 subuid 和subgid

echo jenkins:988:984 > /etc/subgid
echo jenkins:988:984 > /etc/subuid
podman system migrate
podman info

• 容器启动后验证uidmap

podman unshare cat /proc/self/uid_map
         0  300005526          1
         1  410000000  500000000

2- podman pull或者push http镜像遇到http: server gave HTTP response to HTTPS client

2.1- 发现错误

[javadm@instance-2 ~]$  podman pull 47.52.22.186:5000/ubuntu-robin2
Trying to pull 47.52.22.186:5000/ubuntu-robin2...
  Get https://47.52.22.186:5000/v2/: http: server gave HTTP response to HTTPS client
Error: error pulling image "47.52.22.186:5000/ubuntu-robin2": unable to pull 47.52.22.186:5000/ubuntu-robin2: unable to pull image: Error initializing source docker://47.52.22.186:5000/ubuntu-robin2:latest: error pinging docker registry 47.52.22.186:5000: Get https://47.52.22.186:5000/v2/: http: server gave HTTP response to HTTPS client
[javadm@instance-2 ~]$

2.2解决错误

[robin@instance-2 ~]$ cat /etc/containers/registries.conf
[registries.insecure]
registries = ['myregistry.local','47.52.22.186:5000']

默认情况下，这个registries=[]

2.3总结错误

默认情况下，podman客户端pull或者push镜像使用的是https协议，因此如果仓库为http就会遇到这种错误。

3.podman 容器挂载本地磁盘，访问时遇到Permission denied的错误

3.1 发现问题

[javadm@gbl20154099 ~]$ docker container ls
CONTAINER ID  IMAGE    COMMAND               CREATED         STATUS            PORTS                     NAMES
43b0099a694e  nexus1     /bin/bash             33 minutes ago  Up 2 seconds ago                            test_user1
[javadm@gbl20154099 ~]$ docker container exec test_user1 ls /xxx/yyyyyy/
ls: cannot open directory '/xxx/yyyyyy/': Permission denied
Error: non zero exit code: 2: OCI runtime error

检查本地目录权限没有问题。

3.2 解决问题

[javadm@gbl20154099 ~]$ chcon -Rt svirt_sandbox_file_t /xxx/yyyyyy/
[javadm@gbl20154099 ~]$ docker container exec test_user1 ls /xxx/yyyyyy/
funding
landing
rejected
success
[javadm@gbl20154099 ~]$

猜测到可能是selinux拦截了对目录的访问，于是修改了selinux设置。

3.3 总结问题

一旦宿主机和容器内部有目录的挂载，除了保证宿主机上这个目录权限外，还要保证OS层面允许容器访问这个目录。