Intel TDT检测 & PMU数据采集分析——todo，待使用实际恶意样本跑数据分析效果
背景：
Intel TDT检测：英特尔®威胁检测技术（TDT），使用 GPU 扫描内存中的恶意软件，GPU加速，CPU 利用率从 20％降至 2％。
挖矿检测：PMU监视的重复数学计算，当恶意软件的算力达到某个阈值时，PMU就会生成预警信号，由机器学习引擎进行分析，确定该活动是否与加密货币矿工相关。
优势1：可以检测伪造CPU利用率高的挖矿木马
优势2：不受恶意软件常用的反分析技术(例如代码混淆或内存解密等手段)的影响
英特尔处理器的片上（on-chip）性能监视单元（PMU，Performance Monitoring Unit）通过采样硬件事件实现以极低的开销提高数据收集能力。因此，高级热点分析可以用PMU来识别很小的性能瓶颈并发现快速函数的性能瓶颈。
数据采集使用的PMU，然后我看到Stack Overflow上提了下pmu使用：
在运行英特尔内存延迟测试器时计算总周期、未分派 Uop 的周期以及已分派至少一个 Uop 的周期的示例：
perf stat -e r0043003c -e r01c301b1 -e r014301b1 ./mlc --idle_latency
Intel(R) Memory Latency Checker - v3.7
Command line parameters: --idle_latency
Using buffer size of 2000.000MiB
*** Unable to modify prefetchers (try executing 'modprobe msr')
*** So, enabling random access for latency measurements
Each iteration took 182.4 core clocks ( 87.1 ns)
Performance counter stats for './mlc --idle_latency':
91,815,806,587 r0043003c
64,132,006,584 r01c301b1
27,683,941,060 r014301b1
14.587156882 seconds time elapsed
TDT给的一个数据采集示例：
https://github.com/JUSDJTIN/lib-tdt/blob/master/Application/library/configuration/linux/telemetry_collect_tdtlib.profile
; controls driver PMU configuration and receives hardware telemetry data to be used by the normalizer
pmu_publisher
{
; array of event configuration strings to control PMU settings
event_strings
{
; For general programmable counters, you can use the pcm format, one per line.
; The counters will be filled in order, 0-3, so be sure to keep that in mind
; for events which must be on counters 0-1 such as offcore events.
;
; Example: cpu/umask=0x06,event=0xA3,name=CYCLE_ACTIVITY.STALLS_L3_MISS,cmask=6,int=1000000/
;
; Extra support has been added to add interrupting on a counter using the int=# flag.
;
; Example: cpu/umask=0x00,event=0xC0,name=INST_RETIRED.ANY_P/
;
; The fixed counters are always recording, but you can choose to interrupt on them
; by putting cpu/fixed=<0-3 counter index>,int=<number of counted events to trigger an interrupt>
;
; Example: cpu/fixed=0,int=100000/
; The above would generate an interrupt for every 100,000 instructions executed.
;
"0" "cpu/umask=0x06,event=0xA3,name=CYCLE_ACTIVITY.STALLS_L3_MISS,cmask=6,int=1000000/" ; event0
"1" "cpu/umask=0x01,event=0x3C,name=CPU_CLK_THREAD_UNHALTED.REF_XCLK/" ; event1
"2" "cpu/umask=0x01,event=0xA6,name=EXE_ACTIVITY.EXE_BOUND_0_PORTS/" ; event2
"3" "cpu/umask=0x00,event=0xC0,name=INST_RETIRED.ANY_P/" ; event3
}
}
https://perfmon-events.intel.com/
介绍
该站点为英特尔性能监控单元 (PMU) 支持的性能监控事件提供参考。PMU 是内置于处理器内部的硬件，用于测量其性能参数，例如指令周期、高速缓存命中、高速缓存未命中、分支未命中等。性能监控事件提供了表征编程指令序列和微体系结构子系统之间交互的工具.
性能分析工具（例如英特尔® VTune™ Profiler）积极使用性能监控事件，这些工具提供基于事件的采样微架构分析类型，以了解代码如何有效地使用硬件资源并推荐相关的优化技术。
列出的事件是可以使用 Intel® 64 或 IA-32 处理器监控的性能监控事件。监控性能事件的能力和这些处理器中可以监控的事件大多是特定于模型的，除了单独列出的架构性能事件。
这些性能监控事件旨在用作性能调整的指南。性能监控事件报告的计数器值是近似值，被认为可用作调整软件的相关指南。在适用的情况下记录已知的差异。
给定处理器未记录的所有性能事件编码都被视为保留，并且它们的使用将导致未定义的计数器更新以及相关的溢出操作。
有关使用哪些事件的建议，请参阅英特尔® 64 和 IA-32 架构优化参考手册。有关使用性能监控事件的编程工具的更多详细信息，请参阅英特尔® 64 位和 IA-32 架构软件开发人员手册第 3B 卷中的第 18 章性能监控以 了解内核或查看相应的非核心性能监控参考手册以了解您的平台非核心。工具开发人员可以使用https://download.01.org/perfmon/作为获取工具可以使用格式的最新事件列表的地方。有关开发使用这些事件的工具的更多详细信息，请参阅英特尔® SDM 规范的各种文章。 有关性能监控的更多详细信息，请参阅Intel® 64 和 IA-32 架构软件开发人员手册第 3B 卷中的第 18 章“性能监控”
我们随便看一个CPU系列的数据吧，更全的可以看官网：
Intel(R) Xeon Phi Coprocessor based on the Intel(R) Many Integrated Core Architecture
This section provides reference for hardware events that can be monitored for the CPU(s):
- Intel® Xeon Phi™ processor Knights Landing Events
- Intel® Xeon Phi™ processor Knights Mill Events
| Event Name | Description | Additional Info |
|---|---|---|
| CORE | ||
| INST_RETIRED.ANY | This event counts the number of instructions that retire. For instructions that consist of multiple micro-ops, this event counts exactly once, as the last micro-op of the instruction retires. The event continues counting while instructions retire, including during interrupt service routines caused by hardware interrupts, faults or traps. | IA32_FIXED_CTR0 Architectural, Fixed |
| CPU_CLK_UNHALTED.THREAD | This event counts the number of core cycles while the thread is not in a halt state. The thread enters the halt state when it is running the HLT instruction. This event is a component in many key event ratios. The core frequency may change from time to time due to transitions associated with Enhanced Intel SpeedStep Technology or TM2. For this reason this event may have a changing ratio with regards to time. When the core frequency is constant, this event can approximate elapsed time while the core was not in the halt state. It is counted on a dedicated fixed counter | IA32_FIXED_CTR1 Architectural, Fixed |
| CPU_CLK_UNHALTED.REF_TSC | Fixed Counter: Counts the number of unhalted reference clock cycles | IA32_FIXED_CTR2 Architectural, Fixed |
| BR_INST_RETIRED.ALL_BRANCHES | Counts the number of branch instructions retired | EventSel=C4H UMask=00H Counter=0,1 PEBS:[PreciseEventingIP] Architectural |
| BR_MISP_RETIRED.ALL_BRANCHES | Counts the number of mispredicted branch instructions retired | EventSel=C5H UMask=00H Counter=0,1 PEBS:[PreciseEventingIP] Architectural |
| CPU_CLK_UNHALTED.REF | Counts the number of unhalted reference clock cycles | EventSel=3CH UMask=01H Counter=0,1 Architectural |
| CPU_CLK_UNHALTED.THREAD_P | Counts the number of unhalted core clock cycles | EventSel=3CH UMask=00H Counter=0,1 Architectural |
| INST_RETIRED.ANY_P | Counts the total number of instructions retired | EventSel=C0H UMask=00H Counter=0,1 Architectural |
| L2_REQUESTS.MISS | Counts the number of L2 cache misses | EventSel=2EH UMask=41H Counter=0,1 Architectural |
| L2_REQUESTS.REFERENCE | Counts the total number of L2 cache references. | EventSel=2EH UMask=4FH Counter=0,1 Architectural |
| LONGEST_LAT_CACHE.MISS | Counts the number of L2 cache misses | EventSel=2EH UMask=41H Counter=0,1 Architectural |
| LONGEST_LAT_CACHE.REFERENCE | Counts the total number of L2 cache references. | EventSel=2EH UMask=4FH Counter=0,1 Architectural |
| BACLEARS.ALL | Counts the number of times the front end resteers for any branch as a result of another branch handling mechanism in the front end. | EventSel=E6H UMask=01H Counter=0,1 |
| BACLEARS.COND | Counts the number of times the front end resteers for conditional branches as a result of another branch handling mechanism in the front end. | EventSel=E6H UMask=10H Counter=0,1 |
| BACLEARS.RETURN | Counts the number of times the front end resteers for RET branches as a result of another branch handling mechanism in the front end. | EventSel=E6H UMask=08H Counter=0,1 |
| BR_INST_RETIRED.ALL_BRANCHES_PS | Counts the number of branch instructions retired (Precise Event) | EventSel=C4H UMask=00H Counter=0 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.CALL | Counts the number of near CALL branch instructions retired. | EventSel=C4H UMask=F9H Counter=0,1 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.CALL_PS | Counts the number of near CALL branch instructions retired. (Precise Event) | EventSel=C4H UMask=F9H Counter=0 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.FAR_BRANCH | Counts the number of far branch instructions retired. | EventSel=C4H UMask=BFH Counter=0,1 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.FAR_BRANCH_PS | Counts the number of far branch instructions retired. (Precise Event) | EventSel=C4H UMask=BFH Counter=0 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.IND_CALL | Counts the number of near indirect CALL branch instructions retired. | EventSel=C4H UMask=FBH Counter=0,1 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.IND_CALL_PS | Counts the number of near indirect CALL branch instructions retired. (Precise Event) | EventSel=C4H UMask=FBH Counter=0 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.JCC | Counts the number of branch instructions retired that were conditional jumps. | EventSel=C4H UMask=7EH Counter=0,1 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.JCC_PS | Counts the number of branch instructions retired that were conditional jumps. (Precise Event) | EventSel=C4H UMask=7EH Counter=0 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.NON_RETURN_IND | Counts the number of branch instructions retired that were near indirect CALL or near indirect JMP. | EventSel=C4H UMask=EBH Counter=0,1 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.NON_RETURN_IND_PS | Counts the number of branch instructions retired that were near indirect CALL or near indirect JMP. (Precise Event) | EventSel=C4H UMask=EBH Counter=0 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.REL_CALL | Counts the number of near relative CALL branch instructions retired. | EventSel=C4H UMask=FDH Counter=0,1 PEBS:[PreciseEventingIP] |
| BR_INST_RETIRED.REL_CALL_PS | Counts the number of near relative CALL branch instructions retired. (Precise Event) | EventSel=C4H UMask=FDH Counter=0 PEBS:[PreciseEventingIP] |
。。。还有更多。。。
针对TDT，我们再看下其api接口，以C++为例：
/*
********************************************************************************
** Copyright (C) 2019 Intel Corporation
** SPDX-License-Identifier: BSD-3-Clause
********************************************************************************
**
** @file tdt_agent.hpp
**
** @brief Defines the C++ entry point for ThreatDetection Technology Library.
**
**
********************************************************************************
*/
#ifndef TDT_AGENT_HPP
#define TDT_AGENT_HPP
#include <string>
#include <memory>
#include <functional>
#if defined(_WIN32)
# pragma warning(disable : 4251) // class needs to have a dll interface
# ifdef DLL_EXPORTS
# define TDT_API_EXPORT __declspec(dllexport)
# else
# define TDT_API_EXPORT __declspec(dllimport)
# endif
#else
# define TDT_API_EXPORT
#endif
namespace tdt_library
{
/**
* @brief The notification callback signature.
*
* @param[in] context the context that was passed to set_notification_callback.
* @param[in] msg the notification message.
*/
using notification_t = std::function<void(const long long context, const std::string& msg)>;
/**
* @brief Return codes from agent APIs.
* @note Update in tdt_agent.h/.go if updated.
*/
enum tdt_return_code : uint32_t
{
TDT_ERROR_SUCCESS,
TDT_ERROR_NULL_PARAM,
TDT_ERROR_INVALID_PARAM,
TDT_ERROR_OUT_OF_MEMORY,
TDT_ERROR_INTERNAL,
TDT_ERROR_INSUFFICIENT_BUFFER_SIZE,
TDT_ERROR_NOT_IMPLEMENTED,
TDT_ERROR_STARTUP_FAILURE,
TDT_ERROR_INVALID_PLUGIN,
TDT_ERROR_INVALID_CONFIG,
TDT_ERROR_NO_EXECUTION,
TDT_ERROR_AGENT_RUNNING,
TDT_ERROR_AGENT_NOT_RUNNING,
TDT_ERROR_AGENT_ABORTED,
TDT_ERROR_SIGNVERIFY_FAILED,
TDT_ERROR_NO_PROFILES_AVAILABLE,
TDT_WARNING_NOT_ALL_PROFILES_LOADED,
TDT_ERROR_PROFILES_DIR_NOT_EXISTS,
TDT_ERROR_AGENT_UNABLE_TO_STOP,
TDT_ERROR_PIPELINE_NOT_FUNCTIONAL,
TDT_ERROR_MAX
};
/**
* @brief Communication protocol formats supported by the library.
*
*/
enum tdt_protocol_format : uint32_t
{
TDT_PROTO_FORMAT_JSON,
TDT_PROTO_FORMAT_XML,
TDT_PROTO_FORMAT_MAX
};
// forward declaration
class tdt_agent_impl;
/**
* @brief The tdt agent class.
*
* The interface to configure various threat profiles and detect different threats.
*/
class TDT_API_EXPORT agent
{
public:
/**
* @brief Constructor.
*
*/
agent();
/**
* @brief Constructor.
*
* @param[in] proto_fmt Protocol format the user wishes to use. JSON is default if proto_fmt
* is not specified.
* @throw std::invalid_argument if proto_fmt is invalid.
* @throw other exceptions like std::bad_alloc.
*/
agent(tdt_protocol_format proto_fmt);
/**
* @brief Destructor.
*/
~agent();
// No Copy constructor or copy assignment
agent(const agent& orig) = delete;
agent& operator=(const agent&) = delete;
/**
* @brief discover version, build and supported profiles.
*
* @param[out] capabilities on return from this API it will contain a JSON object
* containing version, build, available profiles with properties.
* @code{.json} output: {"version": "1.2.1", "build":{"date":"Feb 18 2019", time:
* "23:59:01"},"profiles":
* [{"rfc_ml_sc":{"description":"side channel","state":"active"}},
* {"rfc_ml_cj":{"description":"crypto mining","state":"inactive"}}]}
* @endcode
*
* @return TDT_ERROR_SUCCESS on success or an error code.
* if TDT_WARNING_NOT_ALL_PROFILES_LOADED is returned then partial list of
* discovered profiles is returned. even on a error code minimum versiona and build
* information will be returned in capabilities.
*/
const tdt_return_code discover(std::string& capabilities);
/**
* @brief get current supported configurations for all profiles or a specific profile.
*
* @param[in] opt_profile optional name of profile for which to get the configuration. this
* can be empty.
* @param[out] gconfig on return will contain a JSON object describing current
* configurations for profile(s).
* @code{.json} input: opt_profile: "rfc_ml_cj"
* @endcode
* @code{.json} output: gconfig: {"configurations" : [{"rfc_ml_cj":{ "normalizer": {"model":
* {"t0_features_per_tid": false}}}}]}
* @endcode
* @code{.json} input: opt_profile: ""
* @endcode
* @code{.json} output: gconfig: {"configurations" : [{"rfc_ml_sc":{ "max_detections":
* 1200}}, {"rfc_ml_cj":{ "report_rate": 500}}]}
* @endcode
*
* @return TDT_ERROR_SUCCESS on success or an error code.
* if TDT_WARNING_NOT_ALL_PROFILES_LOADED is returned then partial list of profiles
* with their configurations is returned.
*/
const tdt_return_code get_configuration(const std::string& opt_profile,
std::string& gconfig);
/**
* @brief set configurations for profiles or a specific profile that need to be started when
* start is called.
*
* @param[in] opt_profile optional name of a specific profile for which to set the
* configuration. if empty then sconfig object should contain profile name with the
* configuration properties to set.
* @param[in] sconfig a JSON object to modify current configuration for the profile.
* if empty then opt_profile can't be empty and current/default configuration
* will be applied for profile specified in opt_profile.
* @code{.json} input: opt_profile: ""
* @endcode
* @code{.json} input: sconfig: {"configurations" : [{"rfc_ml_sc":{ "max_detections":
* 1200}}, {"rfc_ml_cj":{ "report_rate": 500}}]} input: sconfig: {{"rfc_ml_sc":{
* "max_detections": 1200}}, {"rfc_ml_cj":{ "report_rate": 500}}}
* @endcode
* @code{.json} input: opt_profile: "rfc_ml_cj"
* @endcode
* @code{.json} input: sconfig: {"normalizer": {"model": {"t0_features_per_tid": true}}}
* input: sconfig: {"rfc_ml_sc":{ "normalizer": {"model": {"t0_features_per_tid": true}}}}
* @endcode
*
* @return TDT_ERROR_SUCCESS on success or an error code. On success existing profiles
* configurations will be cleared and the new ones will be set to run after calling start.
*/
const tdt_return_code set_configuration(const std::string& opt_profile,
const std::string& sconfig);
/**
* @brief Applies profiles set by set_configuration and starts detection process.
*
* @return TDT_ERROR_SUCCESS on success or an error code.
*/
const tdt_return_code start();
/**
* @brief Stops current detection process for specified profile or all profiles.
* @param[in] opt_profile optional name of profile for which to stop detections.
*
* @return TDT_ERROR_SUCCESS on success or an error code.
*/
const tdt_return_code stop(const std::string& opt_profile);
/**
* @brief set callback for notifications.
*
* @param[in] callback to send notifications.
* @param[in] context to send with notifications. cannot be 0.
*
* @return TDT_ERROR_SUCCESS on success or an error code.
*/
const tdt_return_code set_notification_callback(notification_t callback,
const long long context);
/**
* @brief get string describing an error code.
*
* @param[in] code an error code.
* @code{.c} output: "out of memory"
* @endcode
*
* @return null terminated string describing the error code for valid error codes.
* returns nullptr for invalid error codes.
*
*/
static const char* get_error_string(const tdt_return_code code);
private:
std::unique_ptr<tdt_agent_impl> m_private;
};
} // namespace tdt_library
#endif /* TDT_AGENT_HPP */
其实非常简单，无非是一个配置设置，start检测，stop检测，notify callback。
