[zz]Adventures In A 32-bit Minidump, Part 1

http://www.sevenforums.com/crash-lockup-debug-how/34871-3-advanced-principles-debugging.html#post287662

What's a "loaded modules" list?

Another of the most important bits of information in a minidump is the list of all modules (executable files) loaded at the time of the crash.

Such a list becomes particularly important when deterministic minidump analysis cannot specifically pinpoint a crash cause. By examining the list of modules, it is possible to make educated guesses regarding those modules most likely to be involved, and hence most likely to require updating or even removal.

The objective of the debugger's 'lm' command ("list modules") is self-explanatory:

0: kd> lm
start end module name
80bb0000 80bb8000 kdcom (deferred)
82c03000 83003000 nt (pdb symbols) s:\symCachePub\ntkrnlmp.pdb\998A3472EEA6405CB8C089DE868F26222\ntkrnlmp.pdb
83003000 8303a000 hal (deferred)
87800000 87811000 fileinfo (deferred)
88035000 8817e000 tcpip (pdb symbols) c:\symcachepub\tcpip.pdb\416A30179D3D4AF4A8278FA8146232A92\tcpip.pdb
...

In the truncated display above, the debugger shows several columns of information about each module:

• The "start" and "end" addresses in memory where the module's code has been loaded. For various reasons, that range doesn't necessarily match the file's on-disk size.
• The name of the executable binary (module), without its extension. (Note that the debugger always refers to the kernel image as "nt".)
• The module's symbols status - loaded or "deferred".

"Symbols" are oft-misunderstood by those starting off in debugging. In a nutshell, a module's symbols contain information about the module which is irrelevant to the computer's execution of code in that module, but useful during subsequent debugging. One analogy is a car repair manual - it's not required for driving the car, but almost indispensable for troubleshooting, and entirely car model-specific.

In order to minimise module size, Microsoft strips off symbolic information from modules when they are compiled; otherwise, the OS would consume far more space. Instead of being included with the OS, each module's symbols are made available for download from a HTTP server (preferred) and in symbols "packs" for various OSs and Service Packs. Note that every module has its own specific symbols library (a PDB file), and different versions of the same module will have correspondingly different symbols.

In our example, the "tcpip" module (tcpip.sys) already has symbols loaded by the debugger:

88035000 8817e000 tcpip (pdb symbols) s:\SymCachePub\tcpip.pdb\416A30179D3D4AF4A8278FA8146232A92\tcpip.pdb

The long number is a unique identifier constructed from the module's attributes. The name of the module, plus that number, is what the debugger uses to search Microsoft's public symbol server for a particular PDB when it's required. The rest of the path above is specific to the machine which was used to generate the example.

The debugger performs "lazy" symbol loading by default, which means that (for performance reasons) it will not bother to search Microsoft's symbol server for symbols corresponding to a specific module until that module's code is encountered during debugging. As an example, here is the stack from the corresponding minidump:

0: kd> k
ChildEBP RetAddr
80d8a564 880a67f8 nt!KiTrap0E+0x2cf
80d8a5d8 880a62a6 tcpip!IN6_IS_ADDR_ISATAP+0x18
80d8a5e8 880a36c9 tcpip!Ipv6AddressType+0x27
80d8a658 880a4bde tcpip!IppValidateSetAllRouteParameters+0x56
80d8a698 880a4b88 tcpip!IppUpdateUnicastRouteUnderLock+0x34

Because "nt" (actually NTOSKRNL.EXE) and TCPIP.SYS are the only modules whose functions have been encountered during the stack unwind, their's are the only symbols loaded. All other modules in the listing above have their symbols status specified as "deferred".

The debugger's 'lm' command can optionally take switches. For example, 'lm t' displays the timestamp for each module, in addition to the basic information already discussed above:

0: kd> lm t
start end module name
80bb0000 80bb8000 kdcom Tue Jul 14 11:08:58 2009 (4A5BDAAA)
82c03000 83003000 nt Tue Jul 14 09:15:08 2009 (4A5BBFFC)
83003000 8303a000 hal Tue Jul 14 09:11:03 2009 (4A5BBF07)
...

The textual representation of a module's timestamp (red) is converted for display from the "unix time" value (green) which stored in the minidump for each module. (Unix time is defined as the number of seconds since midnight, Jan 1, 1970.)

Lastly, the 'lm' command can optionally take 'm' (match name pattern) and 'v' (display verbose information) switches. In that combination, it is very useful for revealing as much information as possible about a specific module:

0: kd> lm vm tcpip
start end module name
88035000 8817e000 tcpip (pdb symbols) c:\symcachepub\tcpip.pdb\416A30179D3D4AF4A8278FA8146232A92\tcpip.pdb
Loaded symbol image file: tcpip.sys
Mapped memory image file: c:\symcachepub\tcpip.sys\4A5BBF8E149000\tcpip.sys
Image path: \SystemRoot\System32\drivers\tcpip.sys
Image name: tcpip.sys
Timestamp: Tue Jul 14 09:13:18 2009 (4A5BBF8E)
CheckSum: 00146D18
ImageSize: 00149000
File version: 6.1.7600.16385
Product version: 6.1.7600.16385
... (parts omitted for clarity)...
FileVersion: 6.1.7600.16385 (win7_rtm.090713-1255)
FileDescription: TCP/IP Driver
LegalCopyright: © Microsoft Corporation. All rights reserved.

In situations where the cause of a crash is suspected to be an unknown buggy driver which is corrupting memory, a valid approach is to list all loaded modules, create a sub-list consisting of only those drivers which are not part of the OS, and then rely on each driver's age as a general indicator as to whether an update is likely to be available.

Updating the oldest drivers frequently brings positive results and resolves crashes.