hpsa 0000:0a:00.0: out of memory

Hpsa OOM错误：

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190266] active_anon:850084 inactive_anon:1372017 isolated_anon:0

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190267]  active_file:3177085 inactive_file:1515641 isolated_file:0

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190269]  unevictable:0 dirty:67378 writeback:0 unstable:0

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190270]  free:448723 slab_reclaimable:293611 slab_unreclaimable:346392

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190272]  mapped:6605 shmem:1750450 pagetables:16953 bounce:0

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190279] Node 0 DMA free:15728kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0k

B present:15324kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190301] lowmem_reserve[]: 0 1814 15954 15954

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190326] Node 0 DMA32 free:259816kB min:5084kB low:6352kB high:7624kB active_anon:115648kB inactive_anon:229520kB active_file:438304kB inactive_file:193924kB unevictable:0kB isolate

d(anon):0kB isolated(file):0kB present:1857552kB mlocked:0kB dirty:11844kB writeback:0kB mapped:592kB shmem:277680kB slab_reclaimable:115160kB slab_unreclaimable:104852kB kernel_stack:1136kB pagetables:1192kB unstable:0kB bounce:0kB writ

eback_tmp:0kB pages_scanned:0 all_unreclaimable? no

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190348] lowmem_reserve[]: 0 0 14140 14140

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190373] Node 0 Normal free:1204844kB min:39656kB low:49568kB high:59484kB active_anon:1553884kB inactive_anon:2515388kB active_file:5868168kB inactive_file:2222352kB unevictable:0k

B isolated(anon):0kB isolated(file):0kB present:14479360kB mlocked:0kB dirty:125688kB writeback:0kB mapped:10824kB shmem:3224972kB slab_reclaimable:394928kB slab_unreclaimable:552088kB kernel_stack:4312kB pagetables:45636kB unstable:0kB 

bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190396] lowmem_reserve[]: 0 0 0 0

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190420] Node 1 Normal free:314064kB min:45320kB low:56648kB high:67980kB active_anon:1731220kB inactive_anon:2743160kB active_file:6401336kB inactive_file:3646288kB unevictable:0kB

 isolated(anon):0kB isolated(file):0kB present:16547836kB mlocked:0kB dirty:131980kB writeback:0kB mapped:15004kB shmem:3499148kB slab_reclaimable:664356kB slab_unreclaimable:728628kB kernel_stack:3144kB pagetables:20984kB unstable:0kB b

ounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190442] lowmem_reserve[]: 0 0 0 0

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190466] Node 0 DMA: 2*4kB 1*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15728kB

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190525] Node 0 DMA32: 57072*4kB 2712*8kB 520*16kB 49*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 259936kB

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190591] Node 0 Normal: 246319*4kB 19953*8kB 3232*16kB 313*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1206628kB

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190649] Node 1 Normal: 24071*4kB 17889*8kB 4013*16kB 402*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 316596kB

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190706] 6444684 total pagecache pages

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190711] 1209 pages in swap cache

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190718] Swap cache stats: add 20603325, delete 20602116, find 248979552/249268961

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190724] Free swap  = 1280992kB

Aug 24 03:47:36 LIVE-117 kernel: [3983420.190729] Total swap = 6143992kB

Aug 24 03:47:36 LIVE-117 kernel: [3983420.385835] 8355838 pages RAM

Aug 24 03:47:36 LIVE-117 kernel: [3983420.385841] 178918 pages reserved

Aug 24 03:47:36 LIVE-117 kernel: [3983420.385844] 4937078 pages shared

Aug 24 03:47:36 LIVE-117 kernel: [3983420.385846] 3055834 pages non-shared

Aug 24 03:47:36 LIVE-117 kernel: [3983420.386050] hpsa 0000:0a:00.0: out of memory

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914301] .hpacucli: page allocation failure. order:4, mode:0xd0

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914310] Pid: 12573, comm: .hpacucli Not tainted 2.6.32-279.9.1.el6.x86_64 #1

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914315] Call Trace:

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914335]  [<ffffffff8112789f>] ? __alloc_pages_nodemask+0x77f/0x940

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914347]  [<ffffffff811620a2>] ? kmem_getpages+0x62/0x170

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914355]  [<ffffffff81162cba>] ? fallback_alloc+0x1ba/0x270

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914363]  [<ffffffff8116270f>] ? cache_grow+0x2cf/0x320

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914370]  [<ffffffff81162a39>] ? ____cache_alloc_node+0x99/0x160

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914378]  [<ffffffff8116327a>] ? kmem_cache_alloc_notrace+0xfa/0x130

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914402]  [<ffffffffa005746d>] ? hpsa_update_scsi_devices+0x8d/0x1c60 [hpsa]

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914410]  [<ffffffff8115c51a>] ? alloc_pages_current+0xaa/0x110

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914420]  [<ffffffffa00f9b6b>] ? sg_build_indirect+0x18b/0x320 [sg]

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914432]  [<ffffffffa0059147>] ? hpsa_scan_start+0x107/0x150 [hpsa]

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914441]  [<ffffffffa00f9e1a>] ? sg_open+0x9a/0x630 [sg]

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914450]  [<ffffffff81193bca>] ? dput+0x9a/0x150

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914461]  [<ffffffffa005a9cc>] ? hpsa_ioctl+0x12c/0x8d0 [hpsa]

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914470]  [<ffffffff8117f3b5>] ? chrdev_open+0x125/0x230

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914478]  [<ffffffff8117f290>] ? chrdev_open+0x0/0x230

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914488]  [<ffffffff81365bd3>] ? scsi_ioctl+0xc3/0x380

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914497]  [<ffffffffa00faebe>] ? sg_ioctl+0x20e/0xd20 [sg]

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914506]  [<ffffffff8118c500>] ? do_filp_open+0x780/0xd60

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914513]  [<ffffffff8118e25c>] ? vfs_ioctl+0x7c/0xa0

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914520]  [<ffffffff8118e3a4>] ? do_vfs_ioctl+0x84/0x580

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914527]  [<ffffffff8118e921>] ? sys_ioctl+0x81/0xa0

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914538]  [<ffffffff8100b0f2>] ? system_call_fastpath+0x16/0x1b

Aug 24 03:52:43 LIVE-117 kernel: [3983726.914542] Mem-Info:

解决办法：

系统无法分配连续的内存地址，增加系统预留内存可降低该问题的出现概率。

# echo "vm.min_free_kbytes = 1048576"  >> /etc/sysctl.conf         /预留1G。

# sysctl -p