Containerd高阶命令行工具 - nerdctl 【2】
1.前言
对于用惯了docker cli的用户来说，containerd的命令行工具ctr使用起来不是很顺手，此时别慌，还有另外一个命令行工具项目nerdctl可供我们选择。 nerdctl是一个与docker cli风格兼容的containerd的cli工具。 nerdctl已经作为子项目加入了containerd项目，它的github地址是https://github.com/containerd/nerdctl，而且从最近的nerdctl 0.8开始，nerdctl直接兼容了docker compose的语法(不包含swarm)， 这很大提高了直接将containerd作为本地开发、测试和单机容器部署使用的体验。本来k8s后续将不再支持dockershim，docker在k8s社区的地位急剧下降，现在单机直接使用containerd易用性也不断被完善，也许docker的辉煌已经远去了。
实际上nerdctl compose实现的是Compose Specification规范， 这个规范是从自Docker Compose file version 3 specification规范发展而来的。
2.安装nerdctl
nerdctl 官方发布包含两个安装版本：
- Minimal：仅包含 nerdctl 二进制文件及 rootless 模式下的辅助安装脚本；
- Full：全量包，其中包含了 Containerd、CNI、runc、BuildKit 等完整组件。
2.1 下载nerdctl
注意：安装 nerdctl-full 版本集成了 containerd 。如主机已安装 containerd 请选择 nerdctl简易版（nerdctl-1.2.1-linux-amd64.tar.gz）
wget https://github.com/containerd/nerdctl/releases/download/v1.2.1/nerdctl-full-1.2.1-linux-amd64.tar.gz
2.2 解压安装
本次版本如下：
| 软件包 | 版本号 |
|---|---|
| nerdctl-full | 1.2.1 |
>tar xf nerdctl-full-1.2.1-linux-amd64.tar.gz -C /usr/local/
2.3 启动服务
>systemctl enable containerd; systemctl start containerd
2.4 验证查看
>ctr version
Client:
Version: v1.6.19
Revision: 1e1ea6e986c6c86565bc33d52e34b81b3e2bc71f
Go version: go1.20.1
Server:
Version: v1.6.19
Revision: 1e1ea6e986c6c86565bc33d52e34b81b3e2bc71f
UUID: 50e615cc-7427-4959-ad40-641cec86bfd8
>runc -v
runc version 1.1.4
commit: v1.1.4-0-g5fd4c4d1
spec: 1.0.2-dev
go: go1.20.1
libseccomp: 2.5.1
>nerdctl version
Client:
Version: v1.2.1
OS/Arch: linux/amd64
Git commit: a0bbfd75ba92bcb11ac6059bf4f6f4e50c6da0b8
buildctl:
Version: v0.11.3
GitCommit: 4ddee42a32aac4cd33bf9c2be4c87c2ffd34747b
Server:
containerd:
Version: v1.6.19
GitCommit: 1e1ea6e986c6c86565bc33d52e34b81b3e2bc71f
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d1
3.nerdctl使用
nerdctl 是 containerd 的命令行界面的工具。nerdctl 兼容 docker ，如果会使用 docker-cli 就等于掌握了 nerdctl 80% 的使用方法。nerdctl 不但兼容docker-cli 甚至还兼容了 docker-compose的功能点。
3.1 更名docker
甚至可以直接将nerdctl更名为 docker
cat << 'EOF' > /usr/local/bin/docker
#!/bin/bash
/usr/local/bin/nerdctl $@
EOF
chmod +x /usr/local/bin/docker
3.2 nerdctl bash自动补全
>yum install bash-completion -y
>nerdctl completion bash > /etc/bash_completion.d/nerdctl
>source /etc/bash_completion.d/nerdctl
上面补全的是 nerdctl 的命令，而当 nerdctl 重命名 docker 后，没有 docker 的自动补全。
添加 docker 别名的自动补全：
生成自动补全文件
>nerdctl completion bash > /etc/bash_completion.d/nerdctl
>nerdctl completion bash > /etc/bash_completion.d/docker
生效
>source /etc/bash_completion.d/nerdctl
>source /etc/bash_completion.d/docker
测试：
输入 docker image + 两下 tab
>docker image
image (Manage images) images (List images)
>nerdctl image
image (Manage images) images (List images)
3.3 安装常用插件
安装docker常用扩展插件
docker run --rm -v /usr/local/bin:/sysdir registry.cn-beijing.aliyuncs.com/k7scn/tools tar zxf /pkg.tgz -C /sysdir
安装扩展命令如下：
>docker run -it registry.cn-beijing.aliyuncs.com/k7scn/tools bash
bash-5.2# mkdir /sysdir
bash-5.2# tar xf pkg.tgz -C /sysdir/
bash-5.2# cd /sysdir/
bash-5.2# ls
cclear ctop docker-compose ergoget iclear kdtoken upgrade-tools
crictl din dps helminit kbtoken scope
执行完成后，就已经拷贝到 /usr/local/bin 目录下。
4 镜像管理
4.1 查看镜像
>nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
registry.cn-beijing.aliyuncs.com/k7scn/tools latest 71442d19f1f3 About an hour ago linux/amd64 54.5 MiB 45.7 MiB
或者
>nerdctl image ls
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
registry.cn-beijing.aliyuncs.com/k7scn/tools latest 71442d19f1f3 About an hour ago linux/amd64 54.5 MiB 45.7 MiB
4.2 下载镜像
>nerdctl pull nginx:alpine
>nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
nginx alpine c94a22b036af 19 seconds ago linux/amd64 42.7 MiB 16.0 MiB
4.3 查看镜像详细信息
>nerdctl inspect nginx:alpine
4.4 构建镜像
>mkdir dockerfile
~/dockerfile>vim Dockerfile
FROM nginx:alpine
RUN echo "hello world." > /usr/share/nginx/html/index.html
~/dockerfile>nerdctl build -t mynginx:alpine ./
ERRO[0000] `buildctl` needs to be installed and `buildkitd` needs to be running, see https://github.com/moby/buildkit error="2 errors occurred:\n\t* failed to ping to host unix:///run/buildkit-default/buildkitd.sock: exit status 1\n\t* failed to ping to host unix:///run/buildkit/buildkitd.sock: exit status 1\n\n"
FATA[0000] no buildkit host is available, tried 2 candidates: 2 errors occurred:
* failed to ping to host unix:///run/buildkit-default/buildkitd.sock: exit status 1
* failed to ping to host unix:///run/buildkit/buildkitd.sock: exit status 1
当构建镜像时，出现如上报错信息，是因为 buildkit.service 服务没有启动，启动服务：
>systemctl enable buildkit.service ; systemctl start buildkit.service
BuildKit 是由 docker 公司开发的下一代 docker build 工具，具有更高效、更安全、 易于扩展等特点。BuildKit 是由 buildkitd 守护程序 和 buildctl 客户端组成。
- buildkitd 作为服务端，连接容器运行时，目前支持 runc 和 containerd 作为镜像构建环境，默认是 runc
- buildctl 作为客户端，负责解析 Dockerfile 文件，并向 buildkitd 发出构建请求。由于命令复杂，使用 nerdctl 替代
再次进行构建
>nerdctl build -t myapp:v1 ./
[+] Building 3.5s (5/6)
[+] Building 3.5s (6/6) FINISHED
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 114B 0.0s
=> [internal] load metadata for docker.io/library/nginx:alpine 2.5s
=> [1/2] FROM docker.io/library/nginx:alpine@sha256:c94a22b036afa972426b82d5b0a49c959786005b4f6f81ac7467ca5538d0158f 0.0s
=> => resolve docker.io/library/nginx:alpine@sha256:c94a22b036afa972426b82d5b0a49c959786005b4f6f81ac7467ca5538d0158f 0.0s
=> CACHED [2/2] RUN echo "hello world." > /usr/share/nginx/html/index.html 0.0s
=> exporting to docker image format 0.9s
=> => exporting layers 0.0s
=> => exporting manifest sha256:4c33b3b11666960d0ac4216730378f70a897623e9888b77066189a1dea54aeb4 0.0s
=> => exporting config sha256:a2470d7093fd645ea5adf7055d38110b4d1e4f1221cb2d5feb48c58f47c94b07 0.0s
=> => sending tarball 0.9s
Loaded image: docker.io/library/myapp:v1
查看镜像
>nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
myapp v1 4c33b3b11666 25 seconds ago linux/amd64 42.7 MiB 16.0 MiB
nginx alpine c94a22b036af 49 seconds ago linux/amd64 42.7 MiB 16.0 MiB
注意：
nerdctl 构建的机制和 docker 是完全不同的。
- docker 首先会检查本地是否有 Dockerfile 中 FROM 的镜像。如果有，直接使用。没有则通过网络下载镜像;
- nerdctl 会根据 Dockerfile FROM参数指定镜像的域名去网上找这个镜像，找到后确认和本地同名镜像校验无误之后，才会使用本地的镜像构建新镜像。
举例：
通过tag 打标一个不存在域名的镜像
>ctr i tag docker.io/library/nginx:alpine abc.com/library/nginx:alpine
查看镜像
>nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
abc.com/library/nginx alpine c94a22b036af 1 minutes ago linux/amd64 42.7 MiB 16.0 MiB
nginx alpine c94a22b036af 17 minutes ago linux/amd64 42.7 MiB 16.0 MiB
通过abc.com/library/nginx:alpine构建新镜像
>vim Dockerfile
FROM abc.com/library/nginx:alpine
RUN echo "hello world." > /usr/share/nginx/html/index.html
root@containerd(192.168.199.101)~/dockerfile>nerdctl build -t myapp:v1 ./
[+] Building 0.8s (3/3) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 130B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> ERROR [internal] load metadata for abc.com/library/nginx:alpine 0.7s
------
> [internal] load metadata for abc.com/library/nginx:alpine:
------
Dockerfile:1
--------------------
1 | >>> FROM abc.com/library/nginx:alpine
2 | RUN echo "hello world." > /usr/share/nginx/html/index.html
3 |
--------------------
error: failed to solve: abc.com/library/nginx:alpine: abc.com/library/nginx:alpine: not found
FATA[0001] no image was built
构建时，直接就抛出了错误信息，这里要 非常注意！
4.5 镜像标签TAG
>nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
nginx alpine c94a22b036af About a minute ago linux/amd64 42.7 MiB 16.0 MiB
>nerdctl tag nginx:alpine myapp:v1
>nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
myapp v1 c94a22b036af 1 second ago linux/amd64 42.7 MiB 16.0 MiB
nginx alpine c94a22b036af About a minute ago linux/amd64 42.7 MiB 16.0 MiB
4.6 删除镜像
>nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
myapp v1 c94a22b036af 1 second ago linux/amd64 42.7 MiB 16.0 MiB
nginx alpine c94a22b036af About a minute ago linux/amd64 42.7 MiB 16.0 MiB
>nerdctl rmi myapp:v1
Untagged: docker.io/library/myapp:v1@sha256:c94a22b036afa972426b82d5b0a49c959786005b4f6f81ac7467ca5538d0158f
Deleted: sha256:f1417ff83b319fbdae6dd9cd6d8c9c88002dcd75ecf6ec201c8c6894681cf2b5
Deleted: sha256:1003ff723696bfd596cd65592fa26554840e90780f6937e6ddccc909b8ed1443
Deleted: sha256:1d54586a1706c0af48668c10cbd8246626acb4fec01287be54cd9b26d72df15d
Deleted: sha256:c1cd5c8c68ef2336b2504336206d58931e9215a863a35a741f66aa3f4970b0f5
Deleted: sha256:f0fb842dea4179a94f1b8c2ac178e72690fa2b30e25e03a7a7893794fe9520a5
Deleted: sha256:f9cb3f1f1d3d7c591c4ab02118816fe6761a8f2f7b2500a5ec7421a42b8a5ea2
Deleted: sha256:31531248c7cbf5b31a8d9695c20041b9b3749b8c04b9831331ad93333fcf1474
>nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
nginx alpine c94a22b036af 2 minutes ago linux/amd64 42.7 MiB 16.0 MiB
4.7 导出镜像
导出，不压缩
>nerdctl save nginx:alpine -o nginx.tar
导出且压缩
>nerdctl save nginx:alpine | gzip > nginx-alpine-image.tar.gz
比较两者大小，越大的镜像越压缩后越明显
>du -sh *
16M nginx-alpine-image.tar.gz
17M nginx.tar
4.8 导入镜像
>nerdctl load < nginx-alpine-image.tar.gz
或
>nerdctl load -i nginx-alpine-image.tar.gz
通过上面的展示，基本和docker无差别，其他镜像管理的功能不再赘述。
5 网络
在安装 cri-containerd-cni 时，网络插件也安装了。
>tar tvf cri-containerd-cni-1.7.0-linux-amd64.tar.gz
...
-rw-r--r-- root/root 57 2023-03-11 02:14 etc/crictl.yaml
drwxr-xr-x root/root 0 2023-03-11 02:13 etc/cni/
drwxr-xr-x root/root 0 2023-03-11 02:13 etc/cni/net.d/
-rw-r--r-- root/root 604 2023-03-11 02:13 etc/cni/net.d/10-containerd-net.conflist
...
5.1 查看网络
>nerdctl network ls
NETWORK ID NAME FILE
containerd-net /etc/cni/net.d/10-containerd-net.conflist
17f29b073143 bridge /etc/cni/net.d/nerdctl-bridge.conflist
host
5.2 创建桥接网络
>nerdctl network create -d bridge --subnet 10.244.0.0/16 mynet
>nerdctl network ls
NETWORK ID NAME FILE
containerd-net /etc/cni/net.d/10-containerd-net.conflist
17f29b073143 bridge /etc/cni/net.d/nerdctl-bridge.conflist
11c844f95e28 mynet /etc/cni/net.d/nerdctl-mynet.conflist
host
none
查看创建的网络的配置文件
>cat /etc/cni/net.d/nerdctl-mynet.conflist
{
"cniVersion": "1.0.0",
"name": "mynet",
"nerdctlID": "11c844f95e2862126712e209cd3acbc68c137931c639633da9dfc17b3a464bde",
"nerdctlLabels": {},
"plugins": [
{
"type": "bridge",
"bridge": "br-11c844f95e28",
"isGateway": true,
"ipMasq": true,
"hairpinMode": true,
"ipam": {
"ranges": [
[
{
"gateway": "10.244.0.1",
"subnet": "10.244.0.0/16"
}
]
],
"routes": [
{
"dst": "0.0.0.0/0"
}
],
"type": "host-local"
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
},
{
"type": "firewall",
"ingressPolicy": "same-bridge"
},
{
"type": "tuning"
}
]
}
nerdctl 所使用的网络及模式和 docker 完全一致，这里不再赘述。
6 容器管理
nerdctl 和 dockerc-cli 如出一辙，nerdctl 出现的原因之一就是为了 兼容 docker-cli，所以用法一致，这里只列举几个，其他使用请直接参考 docker-cli
6.1 启动容器
>nerdctl run --name ngx -d -p 80:80 nginx:alpine
20629b70f5c444dd6c379012a6519e21d16b38cc6affa0457e844c6bb3aaedc8
启动容器并指定特定网络(使用宿主机网络直接启动容器)
>nerdctl run --name ngx --net host -d nginx:alpine
49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e
6.2 查看容器
>nerdctl ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0c671d5ac3f1 docker.io/library/nginx:alpine "/docker-entrypoint.…" About a minute ago Up 0.0.0.0:80->80/tcp ngx
查看所有容器
>nerdctl ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0c671d5ac3f1 docker.io/library/nginx:alpine "/docker-entrypoint.…" About a minute ago Up 0.0.0.0:80->80/tcp ngx
5ff17a6ba473 docker.io/library/nginx:alpine "/docker-entrypoint.…" 19 seconds ago Exited (137) 14 seconds ago 0.0.0.0:80->80/tcp ngx-1
查看容器详细信息
>nerdctl inspect ngx
[
{
"Id": "49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e",
"Created": "2023-04-06T01:45:57.035984068Z",
"Path": "/docker-entrypoint.sh",
"Args": [
"nginx",
"-g",
"daemon off;"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"Pid": 17233,
"ExitCode": 0,
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "docker.io/library/nginx:alpine",
"ResolvConfPath": "",
"HostnamePath": "/var/lib/nerdctl/1935db59/containers/default/49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e/hostname",
"LogPath": "/var/lib/nerdctl/1935db59/containers/default/49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e/49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e-json.log",
"Name": "ngx",
"RestartCount": 0,
"Driver": "overlayfs",
"Platform": "linux",
"AppArmorProfile": "",
"Mounts": null,
"Config": {
"Hostname": "49d08525ad53",
"AttachStdin": false,
"Labels": {
"io.containerd.image.config.stop-signal": "SIGQUIT",
"nerdctl/extraHosts": "null",
"nerdctl/hostname": "49d08525ad53",
"nerdctl/log-uri": "binary:///usr/local/bin/nerdctl?_NERDCTL_INTERNAL_LOGGING=%2Fvar%2Flib%2Fnerdctl%2F1935db59",
"nerdctl/name": "ngx",
"nerdctl/namespace": "default",
"nerdctl/networks": "[\"host\"]",
"nerdctl/platform": "linux/amd64",
"nerdctl/state-dir": "/var/lib/nerdctl/1935db59/containers/default/49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e"
}
},
"NetworkSettings": {
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "192.168.199.101",
"IPPrefixLen": 24,
"MacAddress": "52:54:00:e8:88:2b",
"Networks": {
"unknown-br-11c844f95e28": {
"IPAddress": "10.244.0.1",
"IPPrefixLen": 16,
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "56:2d:35:b5:47:9c"
},
"unknown-eth0": {
"IPAddress": "192.168.199.101",
"IPPrefixLen": 24,
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "52:54:00:e8:88:2b"
},
"unknown-nerdctl0": {
"IPAddress": "10.4.0.1",
"IPPrefixLen": 24,
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "86:26:27:ea:8a:a2"
}
}
}
}
]
6.3 删除容器
1.stop 容器
2.删除 容器
或者
强制删除容器
>nerdctl stop ngx
>nerdctl rm ngx
or
>nerdctl rm -f ngx
7.运行docker-compose
nerdctl 直接兼容了 docker-compose 。
docker-compose 配置清单文件
>cat docker-compose.yml
version: "3.7"
services:
ngx:
container_name: "ngx"
image: nginx:alpine
restart: always
networks:
- test_net
ports:
- 80:80
networks:
test_net:
name: test_net
driver: bridge
ipam:
config:
- subnet: "172.100.0.0/16"
通过docker-compose 启动
>nerdctl compose up -d
INFO[0000] Creating network test_net
INFO[0000] Creating network dockerfile_default
INFO[0000] Ensuring image nginx:alpine
INFO[0000] Creating container ngx
查看启动
>nerdctl compose ps
NAME COMMAND SERVICE STATUS PORTS
ngx "/docker-entrypoint.…" ngx running 0.0.0.0:80->80/tcp
停止，删除
>nerdctl compose down
INFO[0000] Removing container ngx
INFO[0000] Removing network test_net
INFO[0000] Removing network dockerfile_default
>nerdctl compose ps
NAME COMMAND SERVICE STATUS PORTS
