PC机上使用qemu-user-static + Docker实现arm/aarch64本地编译

参考的链接：
https://www.cnblogs.com/chen2ha/p/17180287.html
文章很长，一些关键点在：

1. 取得docker#

Docker可以通过Linux发布版中取得，也可以从docker网站上下载编译好的程序。

下载链接：#

https://download.docker.com/linux/static/stable/x86_64/
这里下载的是PC上的Docker，如果需要arm或者aarch64机器的，将链接的最后一级目录修改一下即可。下载下来的文件解压到一个目录，其实就可以执行了。为了让docker能自己找到containerd等程序，修改环境变量PATH，将解压路径加入到PATH中

配置一下docker的配置文件：#

• 配置/etc/docker/daemon.json，文件格式是json格式，最关键是需要增加如下一行：

Copy Highlighter-hljs
  "experimental": true

• 配置/usr/lib/systemd/system/docker.service服务（如果是手工下载的，示例文件将docker解压在/data/docker目录下）

Copy Highlighter-hljs
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
 
[Service]
Type=notify
EnvironmentFile=-/data/docker/etc/sysconfig/docker
EnvironmentFile=-/data/docker/etc/sysconfig/docker-storage
EnvironmentFile=-/data/docker/etc/sysconfig/docker-network
Environment=GOTRACEBACK=crash
Environment=PATH=/usr/sbin:/usr/bin:/sbin:/bin:/data/docker
 
#ExecStart=/data/docker/dockerd -H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock
ExecStart=/data/docker/dockerd $OPTIONS \
                               $DOCKER_STORAGE_OPTIONS \
                               $DOCKER_NETWORK_OPTIONS \
                               $INSECURE_REGISTRY
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
 
[Install]
WantedBy=multi-user.target

• 启动docker后台服务sudo systemctl start docker.service（如果是下载至本地的，则在编辑完docker.service后，先执行一下命令sudo systemctl daemon-reload，每修改编译一次服务文件，均需要执行一下这条命令）

2. 下载qemu-user-static#

这个程序可以在PC机上解析arm或aarch64的ELF文件并执行之（用户侧的“虚拟机”）

下载链接：#

https://github.com/multiarch/qemu-user-static/releases
下载诸如： qemu-aarch64-static.tar.gz/qemu-arm-static.tar.gz的文件，下载下来后，解压。建议解压到docker包的目录下

3. 通过docker下载并注册一个可支持的架构解释器#

这一步让后续的arm或aarch64的docker容器，可以根据arm或aarch64的ELF文件格式调用注册的解析器来解析ELF并执行。

相应的命令如下：#

Copy Highlighter-hljs
docker run --rm --privileged multiarch/qemu-user-static:register --reset

执行成功后，会返回类似如下的结果来表明支持的架构解析器

Copy Highlighter-hljs
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
Setting /usr/bin/qemu-hexagon-static as binfmt interpreter for hexagon

4. 下载特定架构的docker容器#

下载ubuntu 20.04/22.04的arm或aarch64容器的命令如下：#

Copy Highlighter-hljs
docker pull --platform arm ubuntu:20.04
docker pull --platform arm64 ubuntu:20.04
docker pull --platform arm ubuntu:22.04
docker pull --platform arm64 ubuntu:22.04

5. 启动相应的docker容器#

相应的命令如下：#

Copy Highlighter-hljs
docker run -t --rm --platform arm64 \
		-v $(pwd)/qemu-aarch64-static:/usr/bin/qemu-aarch64-static \
		ubuntu:20.04 uname -m
				
docker run -t --rm --platform arm \
		-v $(pwd)/qemu-arm-static:/usr/bin/qemu-arm-static \
		ubuntu:20.04 uname -m

命令有两个关键点：#

• 指定CPU架构为arm或aarch64
• 将外部的qemu-arm-static挂载为容器中的/usr/bin/qemu-arm-static，或
• 将外部的qemu-aarch64-static挂载为容器中的/usr/bin/qemu-aarch64-static。

外部qemu-arm-static或qemu-aarch64-static所在路径是当前路径（所以，使用$(pwd)）
带是--rm参数是因为我们仅仅验证一下相应的容器能否启动正常。

正常情况下，aarch64的容器会输出：#

Copy Highlighter-hljs
aarch64

arm的容器会输出：#

Copy Highlighter-hljs
armv7l

6. 验证通过#

我们就可以基于以上工作在容器中下载相应arm/aarch64编译器，做“本地”编译。