14、基于nerdctl + buildkitd+containerd构建容器镜像

基于nerdctl + buildkitd+containerd构建容器镜像

介绍：

容器技术除了的docker之外，还有coreOS的rkt、google的gvisor、以及docker开源的containerd、redhat的podman、阿⾥的pouch等，为了保证容器⽣态的标准性和健康可持续发展，包括Linux 基⾦会、Docker、微软、红帽、⾕歌和IBM等公司在2015年6⽉共同成⽴了⼀个叫open container（OCI）的组织，其⽬的就是制定开放的标准的容器规范，⽬前OCI⼀共发布了两个规范，分别是runtime spec和image format spec，有了这两个规范，不同的容器公司开发的容器只要兼容这两个规范，就可以保证容器的可移植性和相互可操作性。

https://github.com/alibaba/pouch #阿⾥开源

https://github.com/rkt/rkt #已停⽌维护

https://containerd.io/

https://gvisor.dev/

https://podman.io/

buildkitd

buildkit项目也是Docker公司的人开源出来的一个构建工具包，支持OCI标准的镜像构建。它主要包含以下部分:

• buildkitd(服务端)，⽬前⽀持runc和containerd作为镜像构建环境，默认是runc，可以更换为containerd。

• buildctl(客户端)，负责解析Dockerfile⽂件，并向服务端buildkitd发出构建请求。

部署buildkitd

github地址:https://github.com/moby/buildkit

#下载
root@k8s-containerd:/opt/soft/buildkid# wget https://github.com/moby/buildkit/releases/download/v0.10.3/buildkit-v0.10.3.linux-amd64.tar.gz
#解压
root@k8s-containerd:/opt/soft/buildkid# tar zxvf buildkit-v0.10.3.linux-amd64.tar.gz 
bin/
bin/buildctl
bin/buildkit-qemu-aarch64
bin/buildkit-qemu-arm
bin/buildkit-qemu-i386
bin/buildkit-qemu-mips64
bin/buildkit-qemu-mips64el
bin/buildkit-qemu-ppc64le
bin/buildkit-qemu-riscv64
bin/buildkit-qemu-s390x
bin/buildkit-runc
bin/buildkitd
#
root@k8s-containerd:/opt/soft/buildkid# cp bin/buildctl bin/buildkitd /usr/local/bin/
#systemd服务相关文件
root@k8s-containerd:/opt/soft/buildkid# vim /lib/systemd/system/buildkit.socket
[Unit]
Description=BuildKit
Documentation=https://github.com/moby/buildkit
[Socket]
ListenStream=%t/buildkit/buildkitd.sock
[Install]
WantedBy=sockets.target

root@k8s-containerd:/opt/soft/buildkid# vim /lib/systemd/system/buildkitd.service
[Unit]
Description=BuildKit
Requires=buildkit.socket
After=buildkit.socketDocumentation=https://github.com/moby/buildkit
[Service]
ExecStart=/usr/local/bin/buildkitd --oci-worker=false --containerd-worker=true
[Install]
WantedBy=multi-user.target
#设置开机自启
root@k8s-containerd:~# systemctl daemon-reload
root@k8s-containerd:~# systemctl enable buildkitd
root@k8s-containerd:~# systemctl start buildkitd
root@k8s-containerd:~# systemctl status buildkitd
#buildkitd配置
root@k8s-containerd:~# cat /etc/buildkit/buildkitd.toml
[registry."xmtx.harbor.com"]
http = true
insecure = true

harbor证书分发

root@k8s-containerd:~# mkdir /etc/containerd/certs.d/xmtx.harbor.com/ -p
root@k8s-harbor-01:/opt/soft/certs# scp xmtx.harbor.com.cert 172.31.3.102:/etc/docker/certs.d/xmtx.harbor.com/ 
root@k8s-harbor-01:/opt/soft/certs# scp xmtx.harbor.com.key  172.31.3.102:/etc/docker/certs.d/xmtx.harbor.com/   
root@k8s-harbor-01:/opt/soft/certs# scp ca.key  172.31.3.102:/etc/docker/certs.d/xmtx.harbor.com/ 
#测试登录
root@k8s-containerd:~# nerdctl login xmtx.harbor.com
WARNING: Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

镜像构建测试

root@k8s-containerd:/opt/dockerfile/base/centos# ll
total 31852
drwxr-xr-x 2 root root     4096 Aug  9 09:42 ./
drwxr-xr-x 3 root root     4096 Aug  9 09:40 ../
-rw-r--r-- 1 root root      421 Aug  9 09:41 dockerfile
-rw-r--r-- 1 root root 32600353 Aug  9 09:42 filebeat-7.12.1-x86_64.rpm
root@k8s-containerd:/opt/dockerfile/base/centos# cat Dockerfile
#自定义Centos 基础镜像
FROM centos:7.9.2009 
MAINTAINER xmtx

ADD filebeat-7.12.1-x86_64.rpm /tmp
RUN yum install -y /tmp/filebeat-7.12.1-x86_64.rpm vim wget tree  lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop &&  rm -rf /etc/localtime /tmp/filebeat-7.12.1-x86_64.rpm && ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime  && useradd nginx -u 2088

#nerdctl命令构建
root@k8s-containerd:/opt/dockerfile/base/centos# nerdctl build -t xmtx.harbor.com/baseimages/centos-base:7.9.2009 .

构建过程

上传镜像

root@k8s-containerd:/opt/dockerfile/base/centos# nerdctl push xmtx.harbor.com/baseimages/centos-base:7.9.2009

问题：

如果harbor使用的是自签发的https认证，则构建镜像时会出现证书认证错误：

解决方法：

1、harbor使用http

2、使用机构签发的证书

3、基于nginx代理harbor并实现https

基于nginx代理harbor并实现https

修改harbor为http

root@k8s-harbor-01:/opt/soft/harbor/harbor# docker-compose stop
root@k8s-harbor-01:/opt/soft/harbor/harbor# vim harbor.yml
root@k8s-harbor-01:/opt/soft/harbor/harbor# ./prepare
root@k8s-harbor-01:/opt/soft/harbor/harbor# docker-compose up -d

配置nginx

root@k8s-nginx-01:~# apt install nginx -y
#将harbor证书拷贝到nginx
root@k8s-harbor-01:/opt/soft/certs# scp xmtx.harbor.com.crt xmtx.harbor.com.key 10.0.0.103:/apps/nginx/certs/
#修改nginx配置文件

        client_max_body_size 2000m;
        server {
        listen 80;
        listen 443 ssl;
        server_name xmtx.harbor.com;
        ssl_certificate /apps/nginx/certs/xmtx.harbor.com.crt;
        ssl_certificate_key /apps/nginx/certs/xmtx.harbor.com.key;
        ssl_session_cache shared:sslcache:20m;
        ssl_session_timeout 10m;

        location / {
          proxy_pass http://10.0.0.140;
        
        }
        
#重启nginx
root@k8s-nginx-01:~# systemctl restart nginx.service

构建镜像

#修改hosts文件
root@k8s-containerd:~# vim /etc/hosts
10.0.0.103 xmtx.harbor.com
#构建
root@k8s-containerd:/opt/dockerfile/base/nginx-base# nerdctl build -t  xmtx.harbor.com/baseimages/nginx-base:v1.19.3  .