TrustZone-OPTEE 编译调试笔记
OP-TEE
日期: 2023-11-14, 基于最新 OP-TEE 版本
源: https://github.com/OP-TEE/manifest
导读文档: http://optee.readthedocs.io
国内一些文章都太旧了, 无法完成验证
OP-TEE 组件
OP-TEE is divided in various components:
- A secure privileged layer, executing at Arm secure PL-1 (v7-A) or EL-1 (v8-A) level.
- A set of secure user space libraries designed for Trusted Applications needs.
- A Linux kernel TEE framework and driver (merged to the official tree in v4.12).
- A Linux user space library designed upon the GlobalPlatform TEE Client API specifications.
- A Linux user space supplicant daemon (tee-supplicant) responsible for remote services expected by the TEE OS.
- A test suite (xtest), for doing regression testing and testing the consistency of the API implementations.
- An example git containing a couple of simple host- and TA-examples.
- And some build scripts, debugging tools to ease its integration and the development of Trusted Applications and secure services.
The main ones are build, optee_os, optee_client, optee_test, optee_examples and the Linux kernel TEE framework.
OP-TEE 可信存储设计
摘自: https://optee.readthedocs.io/en/latest/architecture/secure_storage.html#ree-fs-secure-storage
how to build and run OP-TEE
see this: https://optee.readthedocs.io/en/latest/building/index.html
step1: 基础环境安装和编译
系统版本: ubuntu:22.04, 磁盘空间: 大于30GB, 我编译完成之后, 磁盘空间总共占用26GB
FROM ubuntu:22.04
ARG DEBIAN_FRONTEND=noninteractive
RUN apt update && apt upgrade -y
RUN apt install -y \
adb \
acpica-tools \
autoconf \
automake \
bc \
bison \
build-essential \
ccache \
cpio \
cscope \
curl \
device-tree-compiler \
e2tools \
expect \
fastboot \
flex \
ftp-upload \
gdisk \
git \
libattr1-dev \
libcap-ng-dev \
libfdt-dev \
libftdi-dev \
libglib2.0-dev \
libgmp3-dev \
libhidapi-dev \
libmpc-dev \
libncurses5-dev \
libpixman-1-dev \
libslirp-dev \
libssl-dev \
libtool \
libusb-1.0-0-dev \
make \
mtools \
netcat \
ninja-build \
python3-cryptography \
python3-pip \
python3-pyelftools \
python3-serial \
python-is-python3 \
rsync \
swig \
unzip \
uuid-dev \
wget \
xdg-utils \
xterm \
xz-utils \
zlib1g-dev
## 以下步骤需FQ, 我设置的全局代理, 需要github账号验证
RUN curl https://storage.googleapis.com/git-repo-downloads/repo > /bin/repo && chmod a+x /bin/repo
## 如报错, 分步执行 curl xxx/repo > repo, 然后 设置权限, 在 mv 到 /bin/repo 下
RUN mkdir /optee
WORKDIR /optee
RUN repo init -u https://github.com/OP-TEE/manifest.git -m qemu_v8.xml
## 如果拉取有问题, 配置 "--repo-url=https://mirrors.tuna.tsinghua.edu.cn/git/git-repo"
RUN repo sync -j2
## 拉取之后目录 如下
## hgx@ubuntu:~/TrustZone/optee$ ls
## build buildroot hafnium linux mbedtls optee_benchmark optee_client optee_examples optee_os optee_rust optee_test out out-br qemu toolchains trusted-firmware-a u-boot
## 然后执行编译
WORKDIR /optee/build
RUN make -j2 toolchains
RUN make -j$(nproc) check
step2: 测似用启动, qemu 版本确认
cd build
make run
运行之后界面如下, 不要用ssh 终端, ssh 终端看不了效果
会产生一个 Normal 和 Secure World, 在 Normal World 输出 c 回车即可启动 linux 镜像和 TEE-OS
版本确认:
step3: 测试用例测试
-
optee-examples 测试用例
qemu 输入 "optee" + Tab, 可以看到相关测试用例, 对应目录optee_examples, 在这里面添加模块, 根据makefile 查看, 在build目录中, 使用make run编译构建测试用例
-
xtest 测试程序
用来测试 optee 所有相关接口, 直接输入xtest测试所有接口, 输入xtest --help查看帮助文档
step4: 启用自测测试用例
在 测试用例的目录当中, 添加目录 han_test
目录结构如下, 添加 han_test 测试程序需要修改涉及到的配置文件如下 grep 搜索显示
hgx@ubuntu:~/TrustZone/optee/optee_examples/han_test$ pwd
/home/hgx/TrustZone/optee/optee_examples/han_test
hgx@ubuntu:~/TrustZone/optee/optee_examples/han_test$ ls ../
acipher aes Android.mk CMakeLists.txt CMakeToolchain.txt han_test hello_world hotp LICENSE Makefile plugins random README.md secure_storage
各个对应目录说明
build : 构建环境的目录
buildroot : 各个平台镜像配置构建
hafnium : Hafnium is a hypervisor, initially supporting aarch64 (64-bit Armv8 CPUs).
linux : linux 源码
mbedtls : 加密库, X.509证书操作以及SSL/TLS和DTLS协议
optee_benchmark : 为OP-TEE每一层提供详细和精确的分析信息,
optee_client : OP-TEE 客户端调用接口封装
optee_examples : OP-TEE 测试用例编写模块示例
optee_os : OP-TEE OS 相关模块, 在 TA 中相关接口可以在这里查看
optee_rust : RUST 语言接口开发示例
optee_test : xtest 测试工具, 测试 OP-TEE 的所有接口
out : 存放了启动时的各个系统镜像, 包含了 Secure Boot
out-br : 构建文件系统的目录, 我们编译完成后的测试程序会存放到 /out-br/build/ 目录下,
qemu : qemu 虚拟机
toolchains : 涉及的编译工具链
trusted-firmware-a : ATF(Arm Trust Firmware), 包含监控程序, 固件启动等模块
u-boot : uboot 模块
代码导读1: REE->TEE 调用, RA->TA 调用
问题:
- TEE-OS 如何加载 TA 程序
- REE 和 TEE 如何通讯
- RA 如何调用 TA 应用程序相关接口
- TA 如何 调用 TEE-OS 提供的响应的接口
异常指令
对于ARM而言，system call被运行在non-privilege的软件用来向运行在privilege mode（例如linux kernel）的软件申请服务。来到ARM64，privilege包括了 EL1/EL2/EL3，随之而来的是system call这个概念的扩展。从low privilege level都可以通过系统调用来申请higer privilege level 的服务，因此，在ARM64中，能正常产生异常（不是abort）并申请拥有更高权利的软件服务的指令有三条：
- SVC(Supervisor Call): 当用户空间通过系统调用陷入到内核空间的时候，则最终会通过SVC指令进入到内核空间
- HVC(Hypervisor Call): 当在ARMv8-A架构下，normal world, EL1尝试去访问 EL2的时候，则会陷入到虚拟化层的，其中是通过HVC指令
- SMC(Secure Moniter Call): 用于切换noramal world 和 secure world使用。
TEE-OS 通过 SVC 陷入系统调用
/**
* MRS: 状态寄存器到通用寄存器的传送指令。
* MSR: 通用寄存器到状态寄存器的传送指令
* vbar_el1: ESR 寄存器, el(1 ~ 3): http://www.wowotech.net/?post=238
* ESR寄存器（ Exception Syndrome Register）保存了更详细的异常信息。ESR寄存器有三个，分别是ESR_EL1，ESR_EL2，ESR_EL3。
* b: 跳转不返回地址
* bl: 跳转时携带返回地址, 然后子程序可以将返回数据保存在返回地址中
*/
// 主核初始化之前 -> 初始化主核 -> 初始化 cpu 之前 -> 初始化 excp_vect -> 将 thread_excp_vect 基地址写入 el1(64bit)
boot_init_primary_early: -> init_primary: -> thread_init_per_cpu:
thread_init_per_cpu: thread_init_vbar(get_excp_vect());
get_excp_vect: return thread_excp_vect
// thread_init_vbar:
FUNC thread_init_vbar , :
msr vbar_el1, x0
ret
END_FUNC thread_init_vbar
// thread_excp_vect: SVC(el1) 中断触发 之后会执行
FUNC thread_excp_vect , :, align=32
UNWIND( .cantunwind) __thread_svc_handler // UNWIND 作用 栈回溯, 可以理解压的函数栈
b . /* Reset */
b __thread_und_handler /* Undefined instruction */
b __thread_svc_handler /* System call */
b __thread_pabort_handler /* Prefetch abort */
b __thread_dabort_handler /* Data abort */
b . /* Reserved */
b __thread_irq_handler /* IRQ */
b __thread_fiq_handler /* FIQ */
// __thread_svc_handler
__thread_svc_handler: -> thread_scall_handler(core/arch/arm/kernel/thread.c)
// thread_scall_handler -> handle_scall: 这个函数主要回去当前 会话, handle_scall 就是调用 OP-TEE os 接口的函数
void __weak thread_scall_handler(struct thread_scall_regs *regs)
{
struct ts_session *sess = NULL;
uint32_t state = 0;
/* Enable native interrupts */
state = thread_get_exceptions();
thread_unmask_exceptions(state & ~THREAD_EXCP_NATIVE_INTR);
thread_user_save_vfp();
sess = ts_get_current_session();
/*
* User mode service has just entered kernel mode, suspend gprof
* collection until we're about to switch back again.
*/
gprof_set_status(sess, TS_GPROF_SUSPEND);
/* Restore foreign interrupts which are disabled on exception entry */
thread_restore_foreign_intr();
assert(sess && sess->handle_scall);
if (sess->handle_scall(regs)) {
/* We're about to switch back to user mode */
gprof_set_status(sess, TS_GPROF_RESUME);
} else {
/* We're returning from __thread_enter_user_mode() */
setup_unwind_user_mode(regs);
}
}
// handle_scall -> scall_handle_user_ta -> get_tee_syscall_func
#ifdef TRACE_SYSCALLS
#define SYSCALL_ENTRY(_fn) { .fn = (syscall_t)_fn, .name = #_fn }
#else
#define SYSCALL_ENTRY(_fn) { .fn = (syscall_t)_fn }
#endif
/*
* This array is ordered according to the SYSCALL ids TEE_SCN_xxx
*/
static const struct syscall_entry tee_syscall_table[] = {
SYSCALL_ENTRY(syscall_sys_return),
...
REE 侧调用服务过程
// 调用唤起 安全世界的事件代码, 可以看出linux 采用队列管理了要唤醒的线程:
// 队列相关文件更改查看版本: https://lkml.org/lkml/2023/10/30/208
/**
* optee_smc_do_call_with_arg() - Do an SMC to OP-TEE in secure world
* @ctx: calling context
* @shm: shared memory holding the message to pass to secure world
* @offs: offset of the message in @shm
*
* Does and SMC to OP-TEE in secure world and handles eventual resulting
* Remote Procedure Calls (RPC) from OP-TEE.
*
* Returns return code from secure world, 0 is OK
*/
static int optee_smc_do_call_with_arg(struct tee_context *ctx,
struct tee_shm *shm, u_int offs,
bool system_thread)
{
//...
/* Initialize waiter */
optee_cq_wait_init(&optee->call_queue, &w, system_thread);
while (true) {
struct arm_smccc_res res;
trace_optee_invoke_fn_begin(¶m);
optee->smc.invoke_fn(param.a0, param.a1, param.a2, param.a3,
param.a4, param.a5, param.a6, param.a7,
&res);
trace_optee_invoke_fn_end(¶m, &res);
if (res.a0 == OPTEE_SMC_RETURN_ETHREAD_LIMIT) {
/*
* Out of threads in secure world, wait for a thread
* become available.
*/
optee_cq_wait_for_completion(&optee->call_queue, &w);
} else if (OPTEE_SMC_RETURN_IS_RPC(res.a0)) {
cond_resched();
param.a0 = res.a0;
param.a1 = res.a1;
param.a2 = res.a2;
param.a3 = res.a3;
optee_handle_rpc(ctx, rpc_arg, ¶m, &call_ctx);
} else {
rc = res.a0;
break;
}
}
optee_rpc_finalize_call(&call_ctx);
/*
* We're done with our thread in secure world, if there's any
* thread waiters wake up one.
*/
optee_cq_wait_final(&optee->call_queue, &w);
return rc;
}
// 以下为 kernel 对 OPTEE OS 快速调用
optee/smc_abi.c:1006: invoke_fn(OPTEE_SMC_GET_ASYNC_NOTIF_VALUE, 0, 0, 0, 0, 0, 0, 0, &res);
optee/smc_abi.c:1254: invoke_fn(OPTEE_SMC_ENABLE_ASYNC_NOTIF, 0, 0, 0, 0, 0, 0, 0, &res);
optee/smc_abi.c:1265: invoke_fn(OPTEE_SMC_CALLS_UID, 0, 0, 0, 0, 0, 0, 0, &res);
optee/smc_abi.c:1278: invoke_fn(OPTEE_SMC_CALLS_UID, 0, 0, 0, 0, 0, 0, 0, &res);
optee/smc_abi.c:1300: invoke_fn(OPTEE_SMC_CALL_GET_OS_REVISION, 0, 0, 0, 0, 0, 0, 0,
optee/smc_abi.c:1317: invoke_fn(OPTEE_SMC_CALLS_REVISION, 0, 0, 0, 0, 0, 0, 0, &res.smccc);
optee/smc_abi.c:1343: invoke_fn(OPTEE_SMC_EXCHANGE_CAPABILITIES, a1, 0, 0, 0, 0, 0, 0,
optee/smc_abi.c:1366: invoke_fn(OPTEE_SMC_GET_THREAD_COUNT, 0, 0, 0, 0, 0, 0, 0, &res);
optee/smc_abi.c:1387: invoke_fn(OPTEE_SMC_GET_SHM_CONFIG, 0, 0, 0, 0, 0, 0, 0, &res.smccc);
optee/smc_abi.c:1580: invoke_fn(OPTEE_SMC_CALL_LOAD_IMAGE, data_size_high, data_size_low,
TEE侧 中断向量表
FUNC thread_vector_table , : , .identity_map, , nobti
b vector_std_smc_entry: TEESMC_OPTEED_FUNCID_RETURN_CALL_DONE 5 : thread_handle_std_smc -> 有实现
b vector_fast_smc_entry: TEESMC_OPTEED_FUNCID_RETURN_ENTRY_DONE 0 : thread_handle_fast_smc -> 有实现
b vector_cpu_on_entry : TEESMC_OPTEED_FUNCID_RETURN_ON_DONE 1 : cpu_on_handler -> reset_vect_table -> 复位向量表
b vector_cpu_off_entry : TEESMC_OPTEED_FUNCID_RETURN_OFF_DONE 2 : thread_cpu_off_handler
b vector_cpu_resume_entry : TEESMC_OPTEED_FUNCID_RETURN_RESUME_DONE 4 : thread_cpu_resume_handler
b vector_cpu_suspend_entry : TEESMC_OPTEED_FUNCID_RETURN_SUSPEND_DONE 3 : thread_cpu_suspend_handler
b vector_fiq_entry : TEESMC_OPTEED_FUNCID_RETURN_FIQ_DONE 6 : interrupt_main_handler -> 有实现
b vector_system_off_entry : TEESMC_OPTEED_FUNCID_RETURN_SYSTEM_OFF_DONE 7 : thread_system_off_handler
b vector_system_reset_entry : TEESMC_OPTEED_FUNCID_RETURN_SYSTEM_RESET_DONE 8 : thread_system_reset_handler
END_FUNC thread_vector_table
/*
其中 vector_std_smc_entry 实现了 app 调用内核 -> 调用 TEE OS 相关接口, 比如初始化, OPENSessione 等,
调用链: thread_handle_std_smc
-> thread_alloc_and_run
-> __thread_alloc_and_run
-> thread_std_smc_entry
-> __thread_std_smc_entry
-> std_smc_entry 代码实现如下
vector_fast_smc_entry 实现了对 smc 快速返回的操作, 这里提供的接口基本上都是安全系统初始化的时候调用的, a0 类型判断
*/
static uint32_t std_smc_entry(uint32_t a0, uint32_t a1, uint32_t a2,
uint32_t a3 __unused)
{
const bool with_rpc_arg = true;
switch (a0) {
case OPTEE_SMC_CALL_WITH_ARG: //TODO: break;
case OPTEE_SMC_CALL_WITH_RPC_ARG: //TODO: break;
case OPTEE_SMC_CALL_WITH_REGD_ARG:
std_entry_with_regd_arg: // 对 TA 程序的控制操作调用链如下:
-> call_entry_std
-> call_entry_std
-> tee_entry_std
-> __tee_entry_std
default:
EMSG("Unknown SMC 0x%"PRIx32, a0);
return OPTEE_SMC_RETURN_EBADCMD;
}
}
TEE_Result __tee_entry_std(struct optee_msg_arg *arg, uint32_t num_params)
{
TEE_Result res = TEE_SUCCESS;
/* Enable foreign interrupts for STD calls */
thread_set_foreign_intr(true);
switch (arg->cmd) {
case OPTEE_MSG_CMD_OPEN_SESSION:
entry_open_session: // open 一个 TA 程序的调用链
-> tee_ta_open_session
-> tee_ta_init_session
-> tee_ta_init_user_ta_session
-> ldelf_load_ldelf // 这里就是 对 TA 程序动态装载的过程
case OPTEE_MSG_CMD_CLOSE_SESSION: //TODO: break;
case OPTEE_MSG_CMD_INVOKE_COMMAND: //TODO: break;
case OPTEE_MSG_CMD_CANCEL: //TODO: break;
case OPTEE_MSG_CMD_REGISTER_SHM: //TODO: break;
case OPTEE_MSG_CMD_UNREGISTER_SHM: //TODO: break;
case OPTEE_MSG_CMD_DO_BOTTOM_HALF: //TODO: break;
case OPTEE_MSG_CMD_STOP_ASYNC_NOTIF: //TODO: break;
default:
err:
EMSG("Unknown cmd 0x%x", arg->cmd);
res = TEE_ERROR_NOT_IMPLEMENTED;
}
return res;
}
void __tee_entry_fast(struct thread_smc_args *args)
{
switch (args->a0) {
case OPTEE_SMC_CALLS_COUNT: //TODO: break;
case OPTEE_SMC_CALLS_UID: //TODO: break;
case OPTEE_SMC_CALLS_REVISION: //TODO: break;
case OPTEE_SMC_CALL_GET_OS_UUID: //TODO: break;
case OPTEE_SMC_CALL_GET_OS_REVISION: //TODO: break;
case OPTEE_SMC_GET_SHM_CONFIG: //TODO: break;
case OPTEE_SMC_L2CC_MUTEX: //TODO: break;
case OPTEE_SMC_EXCHANGE_CAPABILITIES: //TODO: break;
case OPTEE_SMC_DISABLE_SHM_CACHE: //TODO: break;
case OPTEE_SMC_ENABLE_SHM_CACHE: //TODO: break;
case OPTEE_SMC_BOOT_SECONDARY: //TODO: break;
case OPTEE_SMC_GET_THREAD_COUNT: //TODO: break;
case OPTEE_SMC_VM_CREATED: //TODO: break;
case OPTEE_SMC_VM_DESTROYED: //TODO: break;
case OPTEE_SMC_ENABLE_ASYNC_NOTIF: //TODO: break;
case OPTEE_SMC_GET_ASYNC_NOTIF_VALUE: //TODO: break;
case CFG_WDT_SM_HANDLER_ID: //TODO: break;
default:
args->a0 = OPTEE_SMC_RETURN_UNKNOWN_FUNCTION;
break;
}
}
REE 到 TEE 中断处理细节
所有的 SMC 中断都会被 ATF ARM Trust firmware 接管, 细节交互图如下:
详细参考: trusted-firmware-a/docs/design/interrupt-framework-design.rst
启动流程
如下图: 知道 BL33 阶段, 才真正启动到 U-BOOT 这一环节, 接着再启动的 linux 镜像
bl1: MASKROM
bl2: Loader
bl31: ARM TRUST Firmware
bl32: OP-TEE OS
bl33: U-boot
bl1 加载 b2 并运行bl2, 然后 bl2 会将对应的 bl31, bl32, bl33 加载到对应权限的 RAM 当中, bl2 通过 entry_point_info 调用 SMC 指令运行 bl31程序, 然后bl31 执行 BL32 的镜像, BL32 启动之后, 再去启动 BL33 的镜像。
本文来自博客园踩坑狭，作者：韩若明瞳，转载请注明原文链接：https://www.cnblogs.com/han-guang-xue/p/17825453.html
