使用 FIO 对 PMEM 设备进行测试；解决问题 fio: engine libpmem not loadable；

基础知识：

FIO 工具使用：https://www.cnblogs.com/xuyaowen/p/fio-usage.html

PMEM 模型：NVM (非易失型存储-持久性内存) 编程模型与访问模式; PMDK 架构; App Direct

官方博客：Running FIO with pmem engines

持久内存工具集：2022年01月09日16:13:40

IPMCTL - Utility for managing Intel Optane DC persistent memory modules（专用于 Optane DC）

• Discover persistent memory modules in the platform.
• Provision the platform memory configuration.
• View and update the firmware on PMMs.
• Configure data-at-rest security on PMMs.
• Monitor PMM health.
• Track performance of PMMs.
• Debug and troubleshoot PMMs.

NDCTL - Utility to manage “libnvdimm” subsystem devices (Non-volatile Memory) (通用，适用于NVDIMM) 

此图机器中是双路服务器，并且每个socket关联的内存DIMM槽是交错插满的；

设备概览：

➜  examples sudo ndctl list -N -D -u --regions -i
{
  "dimms":[
    {
      "dev":"nmem1",
      "id":"8089-a2-1930-000008e9",
      "handle":"0x10",
      "phys_id":"0x45"
    },
    {
      "dev":"nmem3",
      "id":"8089-a2-1930-000011fe",
      "handle":"0x1010",
      "phys_id":"0x51"
    },
    {
      "dev":"nmem0",
      "id":"8089-a2-1930-00000b6f",
      "handle":"0",
      "phys_id":"0x43"
    },
    {
      "dev":"nmem2",
      "id":"8089-a2-1930-00000fc8",
      "handle":"0x1000",
      "phys_id":"0x4f"
    }
  ],
  "regions":[
    {
      "dev":"region1",
      "size":"252.00 GiB (270.58 GB)",
      "available_size":0,
      "type":"pmem",
      "numa_node":1,
      "iset_id":"0xc0f2eeb826532444",
      "mappings":[
        {
          "dimm":"nmem3",
          "offset":"0x10000000",
          "length":"0x1f80000000",
          "position":1
        },
        {
          "dimm":"nmem2",
          "offset":"0x10000000",
          "length":"0x1f80000000",
          "position":0
        }
      ],
      "persistence_domain":"memory_controller",
      "namespaces":[
        {
          "dev":"namespace1.0",
          "mode":"fsdax",
          "map":"dev",
          "size":"248.06 GiB (266.35 GB)",
          "uuid":"a4ec89a0-0b3c-4455-9528-25c73b635a51",
          "raw_uuid":"f42aca99-c2fa-4d4d-af09-95ea3815324e",
          "sector_size":512,
          "blockdev":"pmem1",
          "numa_node":1
        },
        {
          "dev":"namespace1.1",
          "mode":"raw",
          "size":0,
          "uuid":"00000000-0000-0000-0000-000000000000",
          "sector_size":512,
          "state":"disabled",
          "numa_node":1
        }
      ]
    },
    {
      "dev":"region0",
      "size":"252.00 GiB (270.58 GB)",
      "available_size":"252.00 GiB (270.58 GB)",
      "type":"pmem",
      "numa_node":0,
      "iset_id":"0x4840eeb8b8462444",
      "mappings":[
        {
          "dimm":"nmem1",
          "offset":"0x10000000",
          "length":"0x1f80000000",
          "position":1
        },
        {
          "dimm":"nmem0",
          "offset":"0x10000000",
          "length":"0x1f80000000",
          "position":0
        }
      ],
      "persistence_domain":"memory_controller",
      "namespaces":[
        {
          "dev":"namespace0.0",
          "mode":"raw",
          "size":0,
          "uuid":"00000000-0000-0000-0000-000000000000",
          "sector_size":512,
          "state":"disabled",
          "numa_node":0
        }
      ]
    }
  ]
}

JOB文件：修改自 LibpmemSeqR.fio  https://github.com/axboe/fio/blob/master/examples/libpmem.fio 

#
# run command for this workload:
# numactl -N 1 fio Libpmemjob.fio
# numactl -N 1 fio --output=Libpmemjob.json --output-format=json Libpmemjob.fio
# 
# it binds this FIO workload to CPU 1 only. It should be set according to /dev/pmem1 socket's number
#
[global]
bs=4k
thread
group_reporting
norandommap
overwrite=1
thinktime=0
sync=1
direct=1

ioengine=libpmem
iodepth=1

# by yaowenxu
# Preparing namespace with commands:
# mkfs.ext4 /dev/pmem1
# mount /dev/pmem1 /mnt/pmem/ -o dax
#

directory=/mnt/pmem/yx  # I'm working on directory; each of 'numjobs' gets its own file to work with
size=10GB  # Each file will have this size; so in summary they will fill the whole namespace
unlink=0

time_based
runtime=300
ramp_time=30

[libpmem-seqwrite]
rw=write
stonewall

[libpmem-seqread]
rw=read
stonewall

#[libpmem-randwrite]
#rw=randwrite
#stonewall

#[libpmem-randread]
#rw=randread
#stonewall

FIO版本：（需要对fio重新编译，开启libpmem引擎）

libpmem engine
IO engine that uses libpmem (part of PMDK collection) to write data and libc's memcpy to read. It requires PMDK >= 1.5.

# 编译命令(fio脚本configure文件编写的稍微有些问题，不能正确配置ldflags和cflags，在此处我使用环境变量解决)
yaowen@zju:~/packages/fio-fio-3.27$ export LDFLAGS='-L/home/yaowen/local/lib/'
yaowen@zju:~/packages/fio-fio-3.27$ export CFLAGS='-I/home/yaowen/local/include/'
yaowen@zju:~/packages/fio-fio-3.27$ ./configure --prefix=/home/yaowen/local

# engines展示

测试命令：(针对fsdax模式进行测试) 

numactl -N 1 fio --output=Libpmemjob.json --output-format=json Libpmemjob.fio

测试结果：如需技术交流，欢迎留言讨论；

更为专业的测试结果可以关注：2022年01月09日16:17:11

Basic Performance Measurements of the Intel Optane DC Persistent Memory Module

保持更新，转载请注明出处；cnblogs.com/xuyaowen;