debain虚拟机内存热插拔不生效问题处理

vmware中的debain虚拟机开启了内存热插把功能，但是在vmware中给虚拟机增加了内存之后，登录虚拟机发现没有添加成功。

# free -g
               total        used        free      shared  buff/cache   available
Mem:              31          15           9           0           5          12
Swap:              0           0           0
# more /proc/meminfo
MemTotal:       32847704 kB
MemFree:        10201676 kB
MemAvailable:   13127260 kB
Buffers:               0 kB
Cached:          5881772 kB
SwapCached:            0 kB
Active:          1547544 kB
Inactive:       20420504 kB
Active(anon):        492 kB
Inactive(anon): 16086724 kB
Active(file):    1547052 kB
Inactive(file):  4333780 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 4 kB
Writeback:             0 kB
AnonPages:      16086408 kB
Mapped:            70236 kB
Shmem:               940 kB
KReclaimable:     119396 kB
Slab:             207268 kB
SReclaimable:     119396 kB
SUnreclaim:        87872 kB
KernelStack:        7392 kB
PageTables:        34600 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    16423852 kB
Committed_AS:   27377088 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       23184 kB
VmallocChunk:          0 kB
Percpu:            75776 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:      347968 kB
DirectMap2M:    23769088 kB
DirectMap1G:    28311552 kB

列出可用的内存块

$ lsmem
RANGE                                 SIZE   STATE REMOVABLE   BLOCK
0x0000000000000000-0x00000000bfffffff   3G  online       yes    0-23
0x0000000100000000-0x000000083fffffff  29G  online       yes  32-263
0x0000000840000000-0x0000000c3fffffff  16G offline           264-391

Memory block size:       128M
Total online memory:      32G
Total offline memory:     16G

添加之前是32G，添加之后还是32G。新加的16G处于offline状态。

 

内存热插拔分为两个阶段：

1.物理内存热插拔阶段，在VMware中给debain的配置中添加内存后，VMware和内核共同完成了物理内存热插拔

2.逻辑内存热插拔阶段，完成内存状态的改变，使得新增内存可用

 

系统将内存划分成大小相同的块，可以通过/sys/devices/system/memory/block_size_bytes来指定块大小。内存被按照该块大小分成了多个文件，以/sys/devices/system/memory/memoryXXX的形式存放。逻辑内存热插拔就是通过将这些文件进行online/offline操作实现的。

# more /sys/devices/system/memory/block_size_bytes
8000000

 这里的值是16进制格式的。十六进制的8000000就是128M

 

1.查看一下可用内存

#  lsmem -a
RANGE                                  SIZE   STATE REMOVABLE BLOCK
0x0000000000000000-0x0000000007ffffff  128M  online       yes     0
0x0000000008000000-0x000000000fffffff  128M  online       yes     1
0x0000000010000000-0x0000000017ffffff  128M  online       yes     2
0x0000000018000000-0x000000001fffffff  128M  online       yes     3
0x0000000020000000-0x0000000027ffffff  128M  online       yes     4
0x0000000028000000-0x000000002fffffff  128M  online       yes     5
0x0000000030000000-0x0000000037ffffff  128M  online       yes     6
0x0000000038000000-0x000000003fffffff  128M  online       yes     7
0x0000000040000000-0x0000000047ffffff  128M  online       yes     8
......
0x0000000840000000-0x0000000847ffffff  128M offline             264
0x0000000848000000-0x000000084fffffff  128M offline             265
0x0000000850000000-0x0000000857ffffff  128M offline             266
0x0000000858000000-0x000000085fffffff  128M offline             267
......

Memory block size:       128M
Total online memory:      32G
Total offline memory:     16G

也可以从/sys/devices/system/memory出查看。不过不如lsmem的结果直观

# grep -r --include="state" "line" /sys/devices/system/memory/
grep -r --include="state" "line" /sys/devices/system/memory/
/sys/devices/system/memory/memory188/state:online
/sys/devices/system/memory/memory99/state:online
/sys/devices/system/memory/memory121/state:online
/sys/devices/system/memory/memory32/state:online
/sys/devices/system/memory/memory329/state:offline
/sys/devices/system/memory/memory272/state:offline
......

 

2.将offline的内存块改成online

用户可以将未使用的内存块改为online状态，实现热插拔。还可以为内存块选择哪个内存zone。

读取有效的内存zone

# cat /sys/devices/system/memory/memory32/valid_zones 
Normal
# cat /sys/devices/system/memory/memory329/valid_zones 
Normal Movable

如果后面还想通过热插拔移除内存，建议将zone设置成Movable zone

内存zone请参考：https://www.ibm.com/docs/en/linux-on-systems?topic=wysk-zones-2#memory_zones

 

设置成online状态

# echo online_value > /sys/devices/system/memory/memory<n>/state

这里的n换成上面对应的数字即可。

比如：

# echo online > /sys/devices/system/memory/memory272/state

online_value的取值有：

·online：sets the memory block online to the default zone. The default zone is the first zone listed in the valid_zones sysfs attribute.
·online_movable：sets the memory block online to the Movable zone. Setting the block online fails if the Movable zone is not listed in the valid_zones sysfs attribute.
·online_kernel：sets the memory block online to the first non-Movable zone listed in the valid_zones directory. Setting the block online fails if the Movable zone is the only zone listed in the valid_zones sysfs attribute.

也可以通过命令chmem -e来修改。具体请参考手册。

对相应的offline内存块进行online操作，完成后内存容量恢复正常。