QEMU EDU设备模拟PCI设备驱动编写

环境安装

buildroot编译

  • buildroot下载，编译：

    • 下载地址：Index of /downloads (buildroot.org)

    • 下载版本：https://www.buildroot.org/downloads/buildroot-2022.02.2.tar.gz

    • 下载完成后，解压：

      $ tar -vxf buildroot-2022.02.2.tar.gz
$ cd buildroot-2022.02.2/
$ make qemu_aarch64_virt_defconfig
which: no g++ in (/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/arm-linaro-4-9-4/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin:/root/bin)
## 可以看出，我的环境下没有g++，需要安装
$ yum install -y gcc-c++
# g++安装完成后, 重新make config
$ make qemu_aarch64_virt_defconfig
#
# configuration written to /home/grace/QEMU/buildroot-2022.02.2/.config
#
# 然后执行make
$ make
/bin/make -j1 O=/home/grace/QEMU/buildroot-2022.02.2/output HOSTCC="/bin/gcc" HOSTCXX="/bin/g++" syncconfig
make[1]: Entering directory `/home/grace/QEMU/buildroot-2022.02.2'
make[1]: Leaving directory `/home/grace/QEMU/buildroot-2022.02.2'
Your Perl installation is not complete enough; at least the following
modules are missing:

​         Data::Dumper
​         ExtUtils::MakeMaker
​         Thread::Queue
# 遇到没有依赖的问题
make: *** [dependencies] Error 1
# 解决方式
$ yum install perl*
$ make
checking whether mknod can create fifo without root privileges... configure: error: in `/home/grace/QEMU/buildroot-2022.02.2/output/build/host-tar-1.34':
configure: error: you should not run configure as root (set FORCE_UNSAFE_CONFIGURE=1 in environment to bypass this check)
See `config.log' for more details
make: *** [/home/grace/QEMU/buildroot-2022.02.2/output/build/host-tar-1.34/.stamp_configured] Error 1
# 遇到问题
# 解决方式
$ export FORCE_UNSAFE_CONFIGURE=1
# 遇到问题
You have PERL_MM_OPT defined because Perl local::lib
is installed on your system. Please unset this variable
before starting Buildroot, otherwise the compilation of
Perl related packages will fail
# 解决方式
$ unset PERL_MM_OPT
# 编译完成
Filesystem UUID: 0504b844-dbbf-4a0e-8799-ff3114bfc2aa
Superblock backups stored on blocks:
        8193, 24577, 40961, 57345

Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Copying files into the device: done
Writing superblocks and filesystem accounting information: done

ln -sf rootfs.ext2 /home/grace/QEMU/buildroot-2022.02.2/output/images/rootfs.ext4
>>>   Executing post-image script board/qemu/post-image.sh

QEMU编译和安装

编译

$ ./configure --prefix=$PWD --target-list=aarch64-softmmu --enable-debug --enable-sdl --enable-kvm --enable-tools --disable-curl
ERROR: User requested feature sdl
       configure was not able to find it.
       Install SDL2-devel

$ ./configure --prefix=$PWD --target-list=aarch64-softmmu --enable-debug --enable-vnc --enable-kvm --enable-tools --disable-curl
ERROR: glib-2.40 gthread-2.0 is required to compile QEMU
$ yum install glib2-devel.x86_64

$ ./configure --prefix=$PWD --target-list=aarch64-softmmu --enable-debug --enable-vnc --enable-kvm --enable-tools --disable-curl
ERROR: zlib check failed
       Make sure to have the zlib libs and headers installed.
[root@localhost] /home/grace/QEMU/qemu-4.1.1
$ yum install zlib-devel.x86_64

$ ./configure --prefix=$PWD --target-list=aarch64-softmmu --enable-debug --enable-vnc --enable-kvm --enable-tools --disable-curl
ERROR: pixman >= 0.21.8 not present.
       Please install the pixman devel package.
$ yum install pixman-devel.x86_64

$ ./configure --prefix=$PWD --target-list=aarch64-softmmu --enable-debug --enable-vnc --enable-kvm --enable-tools --disable-curl
Install prefix    /home/grace/QEMU/qemu-4.1.1
BIOS directory    /home/grace/QEMU/qemu-4.1.1/share/qemu
firmware path     /home/grace/QEMU/qemu-4.1.1/share/qemu-firmware
binary directory  /home/grace/QEMU/qemu-4.1.1/bin
library directory /home/grace/QEMU/qemu-4.1.1/lib
module directory  /home/grace/QEMU/qemu-4.1.1/lib/qemu
libexec directory /home/grace/QEMU/qemu-4.1.1/libexec
include directory /home/grace/QEMU/qemu-4.1.1/include
config directory  /home/grace/QEMU/qemu-4.1.1/etc
local state directory   /home/grace/QEMU/qemu-4.1.1/var
Manual directory  /home/grace/QEMU/qemu-4.1.1/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path       /home/grace/QEMU/qemu-4.1.1
GIT binary        git
GIT submodules
C compiler        cc
Host C compiler   cc
C++ compiler      c++
Objective-C compiler cc
ARFLAGS           rv
CFLAGS            -g
QEMU_CFLAGS       -I/usr/include/pixman-1   -I$(SRC_PATH)/dtc/libfdt   -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include   -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99  -Wendif-labels -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -Wno-missing-braces -I$(SRC_PATH)/capstone/include
LDFLAGS           -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g
QEMU_LDFLAGS      -L$(BUILD_DIR)/dtc/libfdt
make              make
install           install
python            python -B (2.7.5)
slirp support     internal
smbd              /usr/sbin/smbd
module support    no
host CPU          x86_64
host big endian   no
target list       aarch64-softmmu
gprof enabled     no
sparse enabled    no
strip binaries    no
profiler          no
static build      no
SDL support       no
SDL image support no
GTK support       no
GTK GL support    no
VTE support       no
TLS priority      NORMAL
GNUTLS support    no
libgcrypt         no
nettle            no
libtasn1          no
PAM               no
iconv support     yes
curses support    yes
virgl support     no
curl support      no
mingw32 support   no
Audio drivers      oss
Block whitelist (rw)
Block whitelist (ro)
VirtFS support    no
Multipath support no
VNC support       yes
VNC SASL support  no
VNC JPEG support  no
VNC PNG support   no
xen support       no
brlapi support    no
bluez  support    no
Documentation     no
PIE               yes
vde support       no
netmap support    no
Linux AIO support no
ATTR/XATTR support yes
Install blobs     yes
KVM support       yes
HAX support       no
HVF support       no
WHPX support      no
TCG support       yes
TCG debug enabled yes
TCG interpreter   no
malloc trim support yes
RDMA support      no
PVRDMA support    no
fdt support       git
membarrier        no
preadv support    yes
fdatasync         yes
madvise           yes
posix_madvise     yes
posix_memalign    yes
libcap-ng support no
vhost-net support yes
vhost-crypto support yes
vhost-scsi support yes
vhost-vsock support yes
vhost-user support yes
Trace backends    log
spice support     no
rbd support       no
xfsctl support    no
smartcard support no
libusb            no
usb net redir     no
OpenGL support    no
OpenGL dmabufs    no
libiscsi support  no
libnfs support    no
build guest agent yes
QGA VSS support   no
QGA w32 disk info no
QGA MSI support   no
seccomp support   no
coroutine backend ucontext
coroutine pool    yes
debug stack usage no
mutex debugging   yes
crypto afalg      no
GlusterFS support no
gcov              gcov
gcov enabled      no
TPM support       yes
libssh support    no
QOM debugging     yes
Live block migration yes
lzo support       no
snappy support    no
bzip2 support     no
lzfse support     no
NUMA host support no
libxml2           no
tcmalloc support  no
jemalloc support  no
avx2 optimization yes
replication support yes
VxHS block device no
bochs support     yes
cloop support     yes
dmg support       yes
qcow v1 support   yes
vdi support       yes
vvfat support     yes
qed support       yes
parallels support yes
sheepdog support  yes
capstone          internal
docker            no
libpmem support   no
libudev           no
default devices   yes

warning: Python 2 support is deprecated
warning: Python 3 will be required for building future versions of QEMU

当您在编译 QEMU 源代码时，使用 ./configure 命令配置选项可以定制您的构建。让我为您解释一下这些选项的含义：

  1. --prefix=$PWD：这个选项指定了安装目录的前缀。$PWD 表示当前工作目录，因此编译后的二进制文件将被安装到当前目录下。如果您希望将其安装到其他目录，可以更改这个选项的值。
  2. --target-list=aarch64-softmmu：这个选项指定了要构建的目标架构。在这里，我们选择了 aarch64 架构，以便生成支持 64 位 ARM 的二进制文件。-softmmu 表示我们正在构建模拟器，而不是用户态工具。
  3. --enable-debug：启用调试支持。这将在生成的二进制文件中包含调试信息，以便您可以使用调试器进行故障排除。
  4. --enable-vnc：启用 VNC 支持。这允许您通过 VNC 连接到 QEMU 模拟的虚拟机图形界面。
  5. --enable-kvm：启用 KVM 支持。KVM 是 Linux 内核的一部分，它允许 QEMU 利用硬件虚拟化扩展来提高性能。
  6. --enable-tools：启用其他工具的构建。这将生成一些额外的实用程序，例如 qemu-img 和 qemu-nbd。
  7. --disable-curl：禁用 libcurl 支持。libcurl 是一个用于处理 URL 的库，但在某些情况下您可能不需要它。通过禁用它，您可以减小生成的二进制文件的大小。

安装

$ make install
install -d -m 0755 "/home/grace/QEMU/qemu-4.1.1/share/qemu/keymaps"
set -e; for x in da     en-gb  et  fr     fr-ch  is  lt  no  pt-br  sv ar      de     en-us  fi  fr-be  hr     it  lv  nl         pl  ru     th de-ch  es     fo  fr-ca  hu     ja  mk  pt  sl     tr bepo    cz; do \
        install -c -m 0644 /home/grace/QEMU/qemu-4.1.1/pc-bios/keymaps/$x "/home/grace/QEMU/qemu-4.1.1/share/qemu/keymaps"; \
done
install -c -m 0644 /home/grace/QEMU/qemu-4.1.1/trace-events-all "/home/grace/QEMU/qemu-4.1.1/share/qemu/trace-events-all"

​

然后新建个目录，将qemu编译出的文件和buildroot编译出的文件拷贝到新建目录中：

$ mkdir release
$ cd release/
$ mkdir qemu-arm64

# 拷贝QEUM编译出的文件到新建目录
$ cp qemu-4.1.1/libexec/ release/qemu-arm64/ -rf
$ cp qemu-4.1.1/share/ release/qemu-arm64/ -rf

# 拷贝buildroot编译的文件到新建目录
[root@localhost] /home/grace/QEMU/buildroot-2022.02.2
$ cd output/images/
$ cp * ../../../release/qemu-arm64/bin/ -ra
$ cd  ../../../release/qemu-arm64/bin/

# 修改启动脚本, 然后运行
[grace@localhost] ~/QEMU/release/qemu-arm64/bin
$ more start-qemu-bak.sh
./qemu-system-aarch64 -M virt -cpu cortex-a53 -nographic -smp 1 -kernel Image -dtb qemu-my.dtb -append "rootwait root=/dev/vda console=ttyAMA0" -netdev user,id=eth0 -device virtio-net-de
vice,netdev=eth0 -drive file=rootfs.ext4,if=none,format=raw,id=hd0 -device virtio-blk-device,drive=hd0 -nographic -device edu
# 我没找到qemu-my.dtb，就没使用
$ more start-qemu.sh
./qemu-system-aarch64 -M virt -cpu cortex-a53 -nographic -smp 1 -kernel Image -append "rootwait root=/dev/vda console=ttyAMA0" -netdev user,id=eth0 -device virtio-net-device,netdev=eth0
-drive file=rootfs.ext4,if=none,format=raw,id=hd0 -device virtio-blk-device,drive=hd0 -nographic -device edu

QEMU网络配置

方式一

参考官网：文档/网络/NAT - QEMU --- Documentation/Networking/NAT - QEMU

操作步骤：

  1. 首先，Linux主机的IP要配置为192.168.53.0这个网段，为了不修改官方脚本，比如我配置为了：192.168.53.128

  2. 安装

    yum install bridge-utils iptables dnsmasq

  3. 编写qemu-ifup 脚本，将其保存到 /etc/qemu-ifup ，并确保该文件具有执行权限

    chmod 755 /etc/qemu-ifup

    qemu-ifup如下：

    #!/bin/sh
#
# Copyright IBM, Corp. 2010  
#
# Authors:
#  Anthony Liguori <aliguori@us.ibm.com>
#
# This work is licensed under the terms of the GNU GPL, version 2.  See
# the COPYING file in the top-level directory.

# Set to the name of your bridge
BRIDGE=br0

# Network information
NETWORK=192.168.53.0
NETMASK=255.255.255.0
GATEWAY=192.168.53.1
DHCPRANGE=192.168.53.2,192.168.53.254

# Optionally parameters to enable PXE support
TFTPROOT=
BOOTP=

do_brctl() {
    brctl "$@"
}

do_ifconfig() {
    ifconfig "$@"
}

do_dd() {
    dd "$@"
}

do_iptables_restore() {
    iptables-restore "$@"
}

do_dnsmasq() {
    dnsmasq "$@"
}

check_bridge() {
    if do_brctl show | grep "^$1" > /dev/null 2> /dev/null; then
	return 1
    else
	return 0
    fi
}

create_bridge() {
    do_brctl addbr "$1"
    do_brctl stp "$1" off
    do_brctl setfd "$1" 0
    do_ifconfig "$1" "$GATEWAY" netmask "$NETMASK" up
}

enable_ip_forward() {
    echo 1 | do_dd of=/proc/sys/net/ipv4/ip_forward > /dev/null
}

add_filter_rules() {
do_iptables_restore <<EOF
# Generated by iptables-save v1.3.6 on Fri Aug 24 15:20:25 2007
*nat
:PREROUTING ACCEPT [61:9671]
:POSTROUTING ACCEPT [121:7499]
:OUTPUT ACCEPT [132:8691]
-A POSTROUTING -s $NETWORK/$NETMASK -j MASQUERADE 
COMMIT
# Completed on Fri Aug 24 15:20:25 2007
# Generated by iptables-save v1.3.6 on Fri Aug 24 15:20:25 2007
*filter
:INPUT ACCEPT [1453:976046]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1605:194911]
-A INPUT -i $BRIDGE -p tcp -m tcp --dport 67 -j ACCEPT 
-A INPUT -i $BRIDGE -p udp -m udp --dport 67 -j ACCEPT 
-A INPUT -i $BRIDGE -p tcp -m tcp --dport 53 -j ACCEPT 
-A INPUT -i $BRIDGE -p udp -m udp --dport 53 -j ACCEPT 
-A FORWARD -i $1 -o $1 -j ACCEPT 
-A FORWARD -s $NETWORK/$NETMASK -i $BRIDGE -j ACCEPT 
-A FORWARD -d $NETWORK/$NETMASK -o $BRIDGE -m state --state RELATED,ESTABLISHED -j ACCEPT 
-A FORWARD -o $BRIDGE -j REJECT --reject-with icmp-port-unreachable 
-A FORWARD -i $BRIDGE -j REJECT --reject-with icmp-port-unreachable 
COMMIT
# Completed on Fri Aug 24 15:20:25 2007
EOF
}

start_dnsmasq() {
    do_dnsmasq \
	--strict-order \
	--except-interface=lo \
	--interface=$BRIDGE \
	--listen-address=$GATEWAY \
	--bind-interfaces \
	--dhcp-range=$DHCPRANGE \
	--conf-file="" \
	--pid-file=/var/run/qemu-dnsmasq-$BRIDGE.pid \
	--dhcp-leasefile=/var/run/qemu-dnsmasq-$BRIDGE.leases \
	--dhcp-no-override \
	${TFTPROOT:+"--enable-tftp"} \
	${TFTPROOT:+"--tftp-root=$TFTPROOT"} \
	${BOOTP:+"--dhcp-boot=$BOOTP"}
}

setup_bridge_nat() {
    if check_bridge "$1" ; then
	create_bridge "$1"
	enable_ip_forward
	add_filter_rules "$1"
	start_dnsmasq "$1"
    fi
}

setup_bridge_vlan() {
    if check_bridge "$1" ; then
	create_bridge "$1"
	start_dnsmasq "$1"
    fi
}

setup_bridge_nat "$BRIDGE"

if test "$1" ; then
    do_ifconfig "$1" 0.0.0.0 up
    do_brctl addif "$BRIDGE" "$1"
fi

  4. 现在启动带有 tap networking 的 qemu，将您的访客配置为使用 DHCP。

$ more start-qemu.sh 
./qemu-system-aarch64 \
-M virt \
-cpu cortex-a53 \
-nographic \
-smp 1 \
-kernel Image \
-append "rootwait root=/dev/vda console=ttyAMA0" \
-netdev user,id=eth0 \
-drive file=rootfs.ext4,if=none,format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0 -nographic \
-device edu \
-net tap \
-net nic

遗留问题：

  • QEMU无法ping通外网，比如百度？？
  • Linux主机无法使用SSH和Samba？？

方式二

参考：QEMU 网络配置一把梭 | CataLpa's Site (wzt.ac.cn)

博客中的配置步骤：

  1. 首先安装如下软件

    $ yum install bridge-utils iptables dnsmasq

  2. 添加网桥，大部分操作都需要 root 权限：

    ifconfig <你的网卡名称(能上网的那张)> down    # 首先关闭宿主机网卡接口
brctl addbr br0                     # 添加名为 br0 的网桥
brctl addif br0 <你的网卡名称>        # 在 br0 中添加一个接口
brctl stp br0 off                   # 如果只有一个网桥，则关闭生成树协议
brctl setfd br0 1                   # 设置 br0 的转发延迟
brctl sethello br0 1                # 设置 br0 的 hello 时间
ifconfig br0 0.0.0.0 promisc up     # 启用 br0 接口
ifconfig <你的网卡名称> 0.0.0.0 promisc up    # 启用网卡接口
dhclient br0                        # 从 dhcp 服务器获得 br0 的 IP 地址
brctl show br0                      # 查看虚拟网桥列表
brctl showstp br0                   # 查看 br0 的各接口信息

  3. 当配置完成之后执行 ifconfig 结果应该如下：

    $ ifconfig
br0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        inet 192.168.53.128  netmask 255.255.255.0  broadcast 192.168.53.255
        inet6 fe80::20c:29ff:fe2b:ec35  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:2b:ec:35  txqueuelen 1000  (Ethernet)
        RX packets 657  bytes 52987 (51.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 122  bytes 13309 (12.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens33: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        inet 192.168.53.128  netmask 255.255.255.0  broadcast 192.168.53.255
        ether 00:0c:29:2b:ec:35  txqueuelen 1000  (Ethernet)
        RX packets 477  bytes 49868 (48.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 518  bytes 55515 (54.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 12  bytes 1184 (1.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 12  bytes 1184 (1.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  4. 此时网桥已经得到了 IP，并且能够连接网络的网卡 enp0s5 也加入了网桥，此时我们的网桥状态大致是这种情况：

    img

  5. 桥的一端连接到 enp0s5，我们只需要再把另一端接到 QEMU 虚拟机(准确的说是 VLAN )上面就可以了。

    创建一个 TAP 设备，作为 QEMU 一端的接口：

tunctl -t tap0 -u root              # 创建一个 tap0 接口，只允许 root 用户访问
brctl addif br0 tap0                # 在虚拟网桥中增加一个 tap0 接口
ifconfig tap0 0.0.0.0 promisc up    # 启用 tap0 接口
brctl showstp br0                   # 显示 br0 的各个接口

此时网桥的信息应该是：

image-20240530220911527

这样就相当于把两张网卡通过网桥连起来了：

img

现在只需要启动镜像，指定网络连接模式是 TAP 即可。

总的来说，就是需要执行下面的命令：

$ yum install bridge-utils iptables dnsmasq

# 配置命令, ens33是Linux主机的网卡名
ifconfig ens33 down
brctl addbr br0
brctl addif br0 ens33
brctl stp br0 off
brctl setfd br0 1
brctl sethello br0 1
ifconfig br0 0.0.0.0 promisc up
ifconfig ens33 0.0.0.0 promisc up
dhclient br0
brctl show br0
brctl showstp br0

tunctl -t tap0 -u root
brctl addif br0 tap0
ifconfig tap0 0.0.0.0 promisc up
brctl showstp br0

遇到的问题，安装tunctl失败，解决方式：https://www.jianshu.com/p/0c13a00104ac

  1. 创建配置文件：vim /etc/yum.repos.d/nux-misc.repo，配置文件中的信息如下：

    [nux-misc]
name=Nux Misc
baseurl=http://li.nux.ro/download/nux/misc/el7/x86_64/
enabled=0
gpgcheck=1
gpgkey=http://li.nux.ro/download/nux/RPM-GPG-KEY-nux.ro

  2. 安装tunctl软件包：

    yum --enablerepo=nux-misc install tunctl

启动QEMU时，增加下面的参数：

-net nic -net tap,ifname=tap0,script=no,downscript=no
  • -net nic 表示希望 QEMU 在虚拟机中创建一张虚拟网卡
  • -net tap 表示连接类型为 TAP，并且指定了网卡接口名称(就是刚才创建的 tap0，相当于把虚拟机接入网桥)。
  • script 和 downscript 两个选项的作用是告诉 QEMU 在启动系统的时候是否调用脚本自动配置网络环境，如果这两个选项为空，那么 QEMU 启动和退出时会自动选择第一个不存在的 tap 接口(通常是 tap0)为参数，调用脚本 /etc/qemu-ifup 和 /etc/qemu-ifdown。由于我们已经配置完毕，所以这两个参数设置为 no 即可。

疑问：

  1. 按照上述的方式配置之后，QEMU可以ping通百度和笔记本，但是不能ping通虚拟机（虚拟机中安装的Linux，再在Linux中运行的QEMU）。
  2. Linux主机无法使用ssh和samba？
  • 解决方式：？？？

测试

pcie edu介绍

官网介绍：EDU device — QEMU documentation

简介：EDU Device是用于编写（内核）驱动程序的教育设备。它的初衷是支持在马萨里克大学教授的 Linux 内核讲座。学生将获得此虚拟设备，并应编写具有 I/O、IRQ、DMA 等的驱动程序。

PCI ID: 1234:11e8

PCI Region 0: I/O 内存，大小为 1 MB。用户应该通过此内存与卡进行通信。

相关寄存器：

  • 0x00 (RO)：identification，返回值格式：0xRRrr00ed， - RR – 主要版本 - rr – 次要版本
  • 0x04 (RW)：卡活体检查，对写入的值取反，并返回
  • 0x08 (RW)：因子计算，写入一个数值，返回该数值的阶乘
  • 0x20 (RW)：状态寄存器，其低1位为只读，记录设备是否在进行阶乘操作，完成则为0，否则为1；其从低向高第8位为 读写，记录设备是否在完成阶乘操作后发起中断，发起则为 1，否则为 0。
  • 0x24 (RO)：中断状态寄存器，记录中断状态，0x00000001为阶乘中断，0x00000100为 DMA 中断。
  • 0x60 (WO)：触发中断寄存器，引发中断，中断状态将放入中断状态寄存器（使用按位 OR）。
  • 0x64（WO）：用于清除中断，将 0x24 寄存器清零并阻止设备继续发出中断。

pcie测试

测试程序分成两部分，内核态和用户态：

  • 内核态：完成两部分事情，1）创建一个字符设备和用户态进行通讯，使用了ioctl和mmap操作函数；2）注册pci设备驱动，当探测到edu设备时，获取Bar空间信息，并注册中断处理函数。
  • 用户态：完成两部分事情，1）用户态程序在初始化的时候使用ioctl获取到EDU Bar空间信息，然后使用mmap进行映射，这样就可以在用户空间访问EDU Bar空间寄存器；2）创建一个线程，用户接收内核态edu的中断，并注册一个中断回调函数。

内核态代码

#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/fs.h>
#include <linux/cdev.h>
#include <linux/device.h>
#include <linux/slab.h>
#include <linux/uaccess.h>
#include "edu.h"

#define EDU_PRINTK(level, format, arg...) printk(level "[Kernel: %s - %d] " format, __FUNCTION__, __LINE__, ##arg)
#define EDU_ERR(format, arg...)           EDU_PRINTK(KERN_ERR, format, ##arg)
#define EDU_INFO(format, arg...)          EDU_PRINTK(KERN_INFO, format, ##arg)

/* 定义 PCI 设备 ID */
#define EDU_DEVICE_VENDOR_ID 0x1234 /* Vendor ID */
#define EDU_DEVICE_DEVICE_ID 0x11e8 /* Device ID */

/* 定义 character device 的名称以及类名 */
#define EDU_DEVICE_NAME "edu"
#define EDU_CLASS_NAME  "edu"

/* EDU 寄存器读写操作 */
#define EDU_READ_REG(addr)        readl(g_edu_dev->ioaddr + (addr))
#define EDU_WRITE_REG(addr, data) writel((data), g_edu_dev->ioaddr + (addr))

struct edu_ioctl {
    uint64_t start; /* pci bar0起始地址 */
    uint64_t end;   /* pci bar0结束地址 */
    uint64_t len;   /* pci bar0空间大小 */
};

/* EDU设备管理结构体 */
struct edu_pci_dev {
    struct pci_dev *dev;
    void __iomem   *ioaddr; /* 映射后的地址 */
    uint64_t        start;  /* pci bar0起始地址 */
    uint64_t        end;    /* pci bar0结束地址 */
    uint64_t        len;    /* pci bar0空间大小 */
    int             irq;    /* edu设备中断号 */

    /* character device 所需的内容 */
    int            chr_major;
    struct class  *chr_class;
    struct device *chr_device;

    wait_queue_head_t irq_wq;      /* 等待队列，用于阻塞EDU_WAIT_IRQ请求 */
    atomic_t          irq_handled; /* 用于阻塞用户态中断请求，作为条件变量存在 */
};

/* 创建一个字符设备, 和用户空间进行通讯 */
static int  edu_mmap(struct file *filp, struct vm_area_struct *vma);
static long edu_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
static long edu_compat_ioctl(struct file *f, unsigned int cmd, unsigned long arg);

static struct file_operations g_edu_fops = {
    .owner = THIS_MODULE,
    .mmap = edu_mmap,
    .unlocked_ioctl = edu_ioctl,
    .compat_ioctl = edu_compat_ioctl,
};

static struct edu_pci_dev *g_edu_dev;

// 定义 PCI 设备表
static const struct pci_device_id edu_table[] = {
    {PCI_DEVICE(EDU_DEVICE_VENDOR_ID, EDU_DEVICE_DEVICE_ID)},
    {0},
};

// 定义 PCI 驱动
static int edu_probe(struct pci_dev *dev, const struct pci_device_id *id);
static void edu_remove(struct pci_dev *dev);

static struct pci_driver g_edu_driver = {
    .name = "edu",
    .id_table = edu_table,
    .probe = edu_probe,
    .remove = edu_remove,
};

static int edu_mmap(struct file *filp, struct vm_area_struct *vma)
{
    size_t      size = vma->vm_end - vma->vm_start;
    phys_addr_t offset = (phys_addr_t)vma->vm_pgoff << PAGE_SHIFT;

    if ((offset >> PAGE_SHIFT) != vma->vm_pgoff) {
        return -EINVAL;
    }

    if ((offset + (phys_addr_t)size - 1) < offset) {
        return -EINVAL;
    }

    if (!pfn_valid(vma->vm_pgoff)) {
        vma->vm_page_prot = phys_mem_access_prot(filp, vma->vm_pgoff, size, vma->vm_page_prot);
    }

    if (remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff, size, vma->vm_page_prot) != 0) {
        return -EAGAIN;
    }

    return 0;
}

static long edu_compat_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
{
    return edu_ioctl(f, cmd, (unsigned long)((void __user *)(unsigned long)(arg)));
}

static long edu_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
    struct edu_ioctl ioctl;

    switch (cmd) {
        case EDU_WAIT_IRQ:
            EDU_INFO("Edu live reg 0x%x\n", EDU_READ_REG(IO_DEV_CARD_LIVENESS));
            EDU_INFO("Edu ioctl wait irq!\n");
            wait_event_interruptible(g_edu_dev->irq_wq, atomic_read(&g_edu_dev->irq_handled) != 0);
            atomic_set(&g_edu_dev->irq_handled, 0);
            break;
        case EDU_ENABLE_IRQ:
            EDU_INFO("Edu ioctl enable irq!\n");
            EDU_WRITE_REG(IO_DEV_STATUS, 0x80);
            EDU_INFO("Edu ioctl: IO_DEV_STATUS 0x%x\n", EDU_READ_REG(IO_DEV_STATUS));
            break;
        case EDU_GET_BAR_INFO:
            EDU_INFO("Edu get bar information!\n");
            ioctl.start = g_edu_dev->start;
            ioctl.end = g_edu_dev->end;
            ioctl.len = g_edu_dev->len;
            break;
        default:
            return -EINVAL;
    }

    if (copy_to_user((void *)arg, &ioctl, sizeof(struct edu_ioctl))) {
        return -1;
    }

    return 0;
}

// 定义中断处理函数
static irqreturn_t edu_irq_handler(int irq, void *dev)
{
    uint32_t value = 0;
    uint32_t irq_status = 0;

    /* 关闭中断 */
    EDU_WRITE_REG(IO_DEV_STATUS, 0x0);
    EDU_INFO("edu: IO_DEV_STATUS 0x%x\n", EDU_READ_REG(IO_DEV_STATUS));

    /* 读取中断状态 */
    irq_status = EDU_READ_REG(IO_DEV_IRQ_STATUS);
    EDU_INFO("edu: IRQ %d triggered, %d\n", irq, irq_status);

    /* 将中断状态写入中断确认寄存器，清除中断
     * TODO? 不清楚为什么不能放到用户态执行 (现象: 放到用户态会一直上报中断)
    */
    EDU_WRITE_REG(IO_DEV_IRQ_ACK, irq_status);

    /* 读取阶乘结果 */
    value = EDU_READ_REG(IO_DEV_VALUE);
    EDU_INFO("edu: value read from device: 0x%x\n", value);

    /* 唤醒用户态中断处理线程 */
    atomic_set(&g_edu_dev->irq_handled, 1);
    wake_up_interruptible(&g_edu_dev->irq_wq);

    return IRQ_HANDLED;
}

// 定义 PCI 设备的 probe 函数
static int edu_probe(struct pci_dev *dev, const struct pci_device_id *id)
{
    int retval = 0;

    EDU_INFO("Irq num: %d\n", dev->irq);
    EDU_INFO("Vendor id: 0x%x\n", dev->vendor);
    EDU_INFO("Device id: 0x%x\n", dev->device);

    // 首先打开设备
    if (pci_enable_device(dev)) {
        EDU_ERR("edu: Cannot enable PCI device\n");
        retval = -EIO;
        goto out_edu_all;
    }

    // 复制设备的中断号到结构体并检查
    g_edu_dev->irq = dev->irq;
    if (g_edu_dev->irq < 0) {
        EDU_ERR("edu: Invalid IRQ number\n");
        goto out_edu_all;
    }

    // 请求设备的内存区域
    retval = pci_request_regions(dev, "edu");
    if (retval) {
        EDU_ERR("edu: Cannot request regions\n");
        goto out_edu_all;
    }

    g_edu_dev->start = pci_resource_start(dev, 0);
    g_edu_dev->end = pci_resource_end(dev, 0);
    g_edu_dev->len = pci_resource_len(dev, 0);
    EDU_INFO("Bar0 address start: 0x%llx\n", g_edu_dev->start);
    EDU_INFO("Bar0 address   end: 0x%llx\n", g_edu_dev->end);
    EDU_INFO("Bar0 address  size: 0x%llx\n", g_edu_dev->len);

    // 映射设备的内存区域
    g_edu_dev->ioaddr = pci_ioremap_bar(dev, 0);
    if (!g_edu_dev->ioaddr) {
        EDU_ERR("edu: Cannot map device memory\n");
        retval = -ENOMEM;
        goto out_regions;
    }
    EDU_INFO("Bar0 ioaddr: 0x%px\n", g_edu_dev->ioaddr);

    // 设置中断处理函数
    retval = request_irq(dev->irq, edu_irq_handler, IRQF_SHARED, "edu", g_edu_dev);
    if (retval) {
        EDU_ERR("edu: Cannot set up IRQ handler\n");
        goto out_ioremap;
    }
    // 启用设备的中断发起
    EDU_WRITE_REG(IO_DEV_STATUS, 0x80);

    g_edu_dev->dev = dev;

    return 0;

out_ioremap:
    pci_iounmap(dev, g_edu_dev->ioaddr);
out_regions:
    pci_release_regions(dev);
out_edu_all:
    return retval;
}

// 定义 PCI 设备的 remove 函数
static void edu_remove(struct pci_dev *dev)
{
    // 释放中断
    free_irq(g_edu_dev->irq, g_edu_dev);

    // 释放内存区域
    pci_iounmap(dev, g_edu_dev->ioaddr);
    pci_release_regions(dev);

    // 停用设备
    pci_disable_device(dev);
    
    EDU_INFO("edu remove done!\n");
}

// 驱动的初始化函数与卸载函数
static int __init edu_init(void)
{
    int32_t retval = 0;

    /* 为EDU设备结构体分配内存 */
    g_edu_dev = kmalloc(sizeof(struct edu_pci_dev), GFP_KERNEL);
    if (!g_edu_dev) {
        EDU_ERR("edu: Cannot allocate memory for the device\n");
        return -ENOMEM;
    }

    /* 先注册一个字符设备 */
    g_edu_dev->chr_major = register_chrdev(0, EDU_DEVICE_NAME, &g_edu_fops);
    if (g_edu_dev->chr_major < 0) {
        EDU_ERR("edu: Cannot register char device\n");
        goto out_edu_all;
    }
    g_edu_dev->chr_class = class_create(THIS_MODULE, EDU_CLASS_NAME);
    if (IS_ERR(g_edu_dev->chr_class)) {
        EDU_ERR("edu: Cannot create class\n");
        goto out_edu_chr_device;
    }
    g_edu_dev->chr_device = device_create(g_edu_dev->chr_class, NULL, MKDEV(g_edu_dev->chr_major, 0), NULL, EDU_DEVICE_NAME);
    if (IS_ERR(g_edu_dev->chr_device)) {
        EDU_ERR("edu: Cannot create device\n");
        goto out_edu_class;
    }

    /* pci device register */
    retval = pci_register_driver(&g_edu_driver);
    if (retval) {
        EDU_ERR("pci_register_driver fail!\n");
        goto out_pci_register;
    }

    /* 初始化等待队列 */
    init_waitqueue_head(&g_edu_dev->irq_wq);
    atomic_set(&g_edu_dev->irq_handled, 0);

    return retval;

out_pci_register:
    device_destroy(g_edu_dev->chr_class, MKDEV(g_edu_dev->chr_major, 0));
out_edu_class:
    class_destroy(g_edu_dev->chr_class);
out_edu_chr_device:
    unregister_chrdev(g_edu_dev->chr_major, EDU_DEVICE_NAME);
out_edu_all:
    kfree(g_edu_dev);
    return retval;
}

static void __exit edu_exit(void)
{
    /* 字符设备注销 */
    device_destroy(g_edu_dev->chr_class, MKDEV(g_edu_dev->chr_major, 0));
    class_destroy(g_edu_dev->chr_class);
    unregister_chrdev(g_edu_dev->chr_major, EDU_DEVICE_NAME);

    /* pci注销 */
    pci_unregister_driver(&g_edu_driver);

    kfree(g_edu_dev);
    EDU_INFO("edu ko exit done!\n");
}

// 注册驱动的初始化函数与卸载函数
module_init(edu_init);
module_exit(edu_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("MrLayfolk");
MODULE_DESCRIPTION("edu driver");
MODULE_VERSION("1.0.0");

用户态代码

#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <errno.h>
#include <fcntl.h>
#include <unistd.h>
#include <pthread.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include "edu.h"

static int   g_edu_filep = -1;
static void *g_edu_bar_vaddr = NULL; /* Bar空间虚拟地址 */

struct edu_ioctl {
    uint64_t start; /* pci bar0起始地址 */
    uint64_t end;   /* pci bar0结束地址 */
    uint64_t len;   /* pci bar0空间大小 */
};

int edu_read_reg(uint32_t addr)
{
    volatile uint32_t *vaddr32 = NULL;
    vaddr32 = (uint32_t *)((uint8_t *)g_edu_bar_vaddr + addr);
    return *vaddr32;
}

void edu_write_reg(uint32_t addr, uint32_t data)
{
    volatile uint32_t *vaddr32 = NULL;
    vaddr32 = (uint32_t *)((uint8_t *)g_edu_bar_vaddr + addr);
    *vaddr32 = data;
}

void print_edu_regs(void)
{
    printf("IO_DEV_CARD_ID       (0x00) = 0x%x\n", edu_read_reg(IO_DEV_CARD_ID));
    printf("IO_DEV_CARD_LIVENESS (0x04) = 0x%x\n", edu_read_reg(IO_DEV_CARD_LIVENESS));
    printf("IO_DEV_VALUE         (0x08) = 0x%x\n", edu_read_reg(IO_DEV_VALUE));
    printf("IO_DEV_STATUS        (0x20) = 0x%x\n", edu_read_reg(IO_DEV_STATUS));
    printf("IO_DEV_IRQ_STATUS    (0x24) = 0x%x\n", edu_read_reg(IO_DEV_IRQ_STATUS));
    printf("IO_DEV_IRQ_ACK       (0x64) = 0x%x\n", edu_read_reg(IO_DEV_IRQ_ACK));
}

void *edu_mmap(uint32_t size, off_t target, uint64_t *v_addr)
{
    void    *map_base = NULL;
    void    *virt_addr = NULL;
    unsigned page_size = 0;
    unsigned mapped_size = 0;
    unsigned offset_in_page = 0;

    mapped_size = page_size = getpagesize();
    offset_in_page = (unsigned)target & (page_size - 1);
    if (offset_in_page + size > page_size) {
        mapped_size = ((offset_in_page + size) / page_size) * page_size;
        if ((offset_in_page + size) % page_size) {
            mapped_size += page_size;
        }
    }
    map_base = mmap(NULL, mapped_size, (PROT_READ | PROT_WRITE), MAP_SHARED,
                    g_edu_filep, target & ~(off_t)(page_size - 1));
    if (map_base == MAP_FAILED) {
        printf("mmap target addr 0x%x failed.\n", target);
        return MAP_FAILED;
    }
    virt_addr = (char *)map_base + offset_in_page;
    return virt_addr;
}

void edu_irq_handler(void)
{
    printf("irq handler in userspace start ... \n");
    print_edu_regs();
    printf("irq handler in userspace done ... \n");
}

void *edu_irq_thread_fn(void *arg)
{
    printf("Thread Running\n");
    /* 内核态:
     * 1. 使能中断, 等待中断产生;
     * 2. 中断产生后, 关闭中断, 清除中断, 唤醒用户态中断线程;
     * 用户态:
     * 1. 初始化请求中断, 等待中断产生;
     * 2. 中断产生后, 进行相关处理, 然后使能中断
     */
    while (1) {
        /* 等待中断 */
        ioctl(g_edu_filep, EDU_WAIT_IRQ);

        /* 中断处理函数 */
        edu_irq_handler();

        /* 使能中断 */
        ioctl(g_edu_filep, EDU_ENABLE_IRQ);
    }
}

int main(int argc, char **argv)
{
    int              ret = 0;
    struct edu_ioctl edu_ioctl = {0};
    pthread_t        thread_id;

    g_edu_filep = open("/dev/edu", O_RDWR);
    if (g_edu_filep < 0) {
        printf("open %s failed!\n", "/dev/edu");
    }

    /* 获取pci bar信息 */
    ioctl(g_edu_filep, EDU_GET_BAR_INFO, &edu_ioctl);
    printf("Bar0 address start: 0x%llx\n", edu_ioctl.start);
    printf("Bar0 address   end: 0x%llx\n", edu_ioctl.end);
    printf("Bar0 address  size: 0x%llx\n", edu_ioctl.len);
    g_edu_bar_vaddr = edu_mmap(edu_ioctl.len, (off_t)edu_ioctl.start, NULL);
    printf("Bar0 vaddr: 0x%p\n", g_edu_bar_vaddr);

    /* 寄存器读写测试 */
    print_edu_regs();
    edu_write_reg(IO_DEV_CARD_LIVENESS, 0x2);
    print_edu_regs();

    /* 创建中断处理线程 */
    if (pthread_create(&thread_id, NULL, edu_irq_thread_fn, NULL) != 0) {
        printf("Failed to create thread\n");
        return 1;
    }
    /* 等待线程结束 */
    pthread_join(thread_id, NULL);

    return 0;
}

内核态用态态公共头文件

#ifndef EDU_H
#define EDU_H

#include <linux/ioctl.h>

/* 定义要用到的寄存器偏移量:
- 0x04 (RW)：卡活体检查，对写入的值取反，并返回
- 0x08 (RW)：因子计算，写入一个数值，返回该数值的阶乘
- 0x20 (RW)：状态寄存器，其低1位为只读，记录设备是否在进行阶乘操作，完成则为0，否则为1；
             其从低向高第8位为读写，记录设备是否在完成阶乘操作后发起中断，发起则为1，否则为0。
- 0x24 (RO)：中断状态寄存器，记录中断状态，0x00000001为阶乘中断，0x00000100为 DMA 中断。
- 0x60 (WO)：触发中断寄存器，引发中断，中断状态将放入中断状态寄存器（使用按位 OR）。
- 0x64 (WO)：用于清除中断，将0x24寄存器清零并阻止设备继续发出中断。
*/
#define IO_DEV_CARD_ID       0x00
#define IO_DEV_CARD_LIVENESS 0x04
#define IO_DEV_VALUE         0x08
#define IO_DEV_STATUS        0x20
#define IO_DEV_IRQ_STATUS    0x24
#define IO_DEV_IRQ_ACK       0x64

/* 定义 ioctl 的操作号 */
#define EDU_IOCTL_CMD_MAGIC 'e'
#define EDU_WAIT_IRQ        _IO(EDU_IOCTL_CMD_MAGIC, 1) /* 等待EDU中断 */
#define EDU_ENABLE_IRQ      _IO(EDU_IOCTL_CMD_MAGIC, 2) /* 使能EDU中断 */
#define EDU_GET_BAR_INFO    _IO(EDU_IOCTL_CMD_MAGIC, 3) /* 获取EDU Bar信息 */

#endif

Makefile

# 指定交叉编译工具链的路径
CROSS_COMPILE := /home/grace/output/host/bin/aarch64-buildroot-linux-uclibc-

# 指定内核源码路径
KERNEL_DIR := /home/grace/output/build/linux-5.15.18/

# 指定要编译的模块文件名（例如：my_module.ko）
obj-m := edu.o

# 编译规则
all:
	$(MAKE) -C $(KERNEL_DIR) M=$(PWD) ARCH=arm64 CROSS_COMPILE=$(CROSS_COMPILE) modules
	$(CROSS_COMPILE)gcc -o edu edu_user.c

# 清理规则
clean:
	$(MAKE) -C $(KERNEL_DIR) M=$(PWD) clean
	rm -rf edu

测试代码：

#include <linux/module.h>

static int __init edu_init(void)
{
    printk(KERN_INFO "edu: Hello, World!\n");
    return 0;
}

static void __exit edu_exit(void)
{
    printk(KERN_INFO "edu: Goodbye, World!\n");
}

module_init(edu_init);
module_exit(edu_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("SpartaEN <i@evo.moe>");
MODULE_DESCRIPTION("edu driver");
MODULE_VERSION("0.0.1");

测试

QEMU上电启动后，先使用lspci命令查看是否有edu设备：

# lspci -vvv
00:01.0 Class 0200: 1af4:1000
00:00.0 Class 0600: 1b36:0008
00:02.0 Class 00ff: 1234:11e8

可以看到，有edu设备，但是-vvv参数并没有打印出详细信息，这是由于我使用buildroot编译的工具裁剪了，可以自己编译一个lspci工具，具体方式参考下一个章节。

使用自己编译的工具查看，可以看到详细的信息：

# ./lspci -s 00:02.0 -vv
00:02.0 Class 00ff: Device 1234:11e8 (rev 10)
	Subsystem: Device 1af4:1100
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 0
	Region 0: Memory at 10000000 (32-bit, non-prefetchable) [disabled] [size=1M]
	Capabilities: [40] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000

 ./lspci -s 00:02.0 -mm
00:02.0 "Class 00ff" "Vendor 1234" "Device 11e8" -r10 -p00 "Unknown vendor 1af4" "Device 1100"

测试步骤：

  1. 插入KO，可看出中断号为53，Bar0起始地址为0x10000000，Bar0空间大小为0x100000B(1MB)

    # insmod edu.ko 
edu: loading out-of-tree module taints kernel.
[Kernel: edu_probe - 181] Irq num: 53
[Kernel: edu_probe - 182] Vendor id: 0x1234
[Kernel: edu_probe - 183] Device id: 0x11e8
edu 0000:00:02.0: enabling device (0000 -> 0002)
[Kernel: edu_probe - 209] Bar0 address start: 0x10000000
[Kernel: edu_probe - 210] Bar0 address   end: 0x100fffff
[Kernel: edu_probe - 211] Bar0 address  size: 0x100000
[Kernel: edu_probe - 221] Bar0 ioaddr: 0xffffffc008e00000

  2. 执行用户态程序，以背景线程方式运行：

    # ./edu &
# [Kernel: edu_ioctl - 130] Edu get bar information!
Bar0 address start: 0x10000000
Bar0 address   end: 0x100fffff
Bar0 address  size: 0x100000
Bar0 vaddr: 0x0x7fa3307000   # mmap映射后的虚拟地址
IO_DEV_CARD_ID       (0x00) = 0x10000ed
IO_DEV_CARD_LIVENESS (0x04) = 0xffffffff
IO_DEV_VALUE         (0x08) = 0x0
IO_DEV_STATUS        (0x20) = 0x80
IO_DEV_IRQ_STATUS    (0x24) = 0x0
IO_DEV_IRQ_ACK       (0x64) = 0xffffffff
# 更改IO_DEV_CARD_LIVENESS后的值
IO_DEV_CARD_ID       (0x00) = 0x10000ed
IO_DEV_CARD_LIVENESS (0x04) = 0xfffffffd
IO_DEV_VALUE         (0x08) = 0x0
IO_DEV_STATUS        (0x20) = 0x80
IO_DEV_IRQ_STATUS    (0x24) = 0x0
IO_DEV_IRQ_ACK       (0x64) = 0xffffffff
Thread Running
[Kernel: edu_ioctl - 119] Edu live reg 0xfffffffd
[Kernel: edu_ioctl - 120] Edu ioctl wait irq!

  3. 使用devmem命令模拟中断发生，可以看出内核态接收到了中断，并上报到了内核态，然后用户态中断线程继续等待中断到来。

    # devmem 0x10000008 32 2
# [Kernel: edu_irq_handler - 154] edu: IO_DEV_STATUS 0x0
[Kernel: edu_irq_handler - 158] edu: IRQ 53 triggered, 1
[Kernel: edu_irq_handler - 167] edu: value read from device: 0x2
irq handler in userspace start ... 
IO_DEV_CARD_ID       (0x00) = 0x10000ed
IO_DEV_CARD_LIVENESS (0x04) = 0xfffffffd
IO_DEV_VALUE         (0x08) = 0x2
IO_DEV_STATUS        (0x20) = 0x0
IO_DEV_IRQ_STATUS    (0x24) = 0x0
IO_DEV_IRQ_ACK       (0x64) = 0xffffffff
irq handler in userspace done ... 
[Kernel: edu_ioctl - 125] Edu ioctl enable irq!
[Kernel: edu_ioctl - 127] Edu ioctl: IO_DEV_STATUS 0x80
[Kernel: edu_ioctl - 119] Edu live reg 0xfffffffd
[Kernel: edu_ioctl - 120] Edu ioctl wait irq!

    devmem使用说明：

    Usage: devmem ADDRESS [WIDTH [VALUE]]

Read/write from physical address

	ADDRESS	Address to act upon
	WIDTH	Width (8/16/...)
	VALUE	Data to be written

lspci工具编译

编译参考：pciutils交叉编译 - 只道寻常 | Blog (copyright1999.github.io)

lspci使用的详细命令可参考：lspci Command: What Is It and How to Use It {7 Examples} (phoenixnap.com)

步骤：

  1. 下载源码，我下载的是：pciutils-3.12.0 (linuxfromscratch.org)

  2. 修改顶层Makefile，需要修改两个变量：

    # Host OS and release (override if you are cross-compiling)
HOST=arm-linux
CROSS_COMPILE=/home/grace/output/host/bin/aarch64-buildroot-linux-uclibc-

  3. 编译，直接执行make

  4. 编译完成后，生成的lspci工具就在顶层目录

    $ file lspci
lspci: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked (uses shared libs), not stripped

  5. 使用lspci查看edu设备详细信息，可看出插入KO前后的变化：

    • Mem-变成了Mem+：表示可以进行Bar空间访问了
    • 中断号变成了32
    • Region 0状态由[disabled]变成了(32-bit, non-prefetchable)
    # ./lspci -s 00:02.0  -vvvvvv
00:02.0 Class 00ff: Device 1234:11e8 (rev 10)
	Subsystem: Device 1af4:1100
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 0
	Region 0: Memory at 10000000 (32-bit, non-prefetchable) [disabled] [size=1M]
	Capabilities: [40] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
# 
# insmod edu.ko 
edu: loading out-of-tree module taints kernel.
[Kernel: edu_probe - 181] Irq num: 53
[Kernel: edu_probe - 182] Vendor id: 0x1234
[Kernel: edu_probe - 183] Device id: 0x11e8
edu 0000:00:02.0: enabling device (0000 -> 0002)
[Kernel: edu_probe - 209] Bar0 address start: 0x10000000
[Kernel: edu_probe - 210] Bar0 address   end: 0x100fffff
[Kernel: edu_probe - 211] Bar0 address  size: 0x100000
[Kernel: edu_probe - 221] Bar0 ioaddr: 0xffffffc008e00000
# 
# ./lspci -s 00:02.0  -vvvvvv
00:02.0 Class 00ff: Device 1234:11e8 (rev 10)
	Subsystem: Device 1af4:1100
	Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 53
	Region 0: Memory at 10000000 (32-bit, non-prefetchable) [size=1M]
	Capabilities: [40] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Kernel driver in use: edu
posted @ 2024-06-04 23:17  zhengcixi  阅读(235)  评论(0)  编辑  收藏  举报
回到顶部