symbol加载失败的case分析
最新一直在分析产品反馈回来的dump,分析过程中发现有一些dump的symbol无法加载。比例很小,1000个dump,只有7个没法加载symbol。
即使使用.reload /i也无法加载:
0:000> .reload /i alibrowser.exe
Unable to load image C:\Program Files\AliBrowser\Alibrowser.exe, Win32 error 0n2
*** WARNING: Unable to verify timestamp for Alibrowser.exe
*** ERROR: Module load completed but symbols could not be loaded for Alibrowser.exe
我们知道!sym noisy可以将整个加载symbol的过程打印出来。所以执行!sym noisy 之后再次执行.reload /i alibrowser.exe
0:000> !sym noisy
noisy mode - symbol prompts on
0:000> .reload /i alibrowser.exe
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://10.1.170.157/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://msdl.microsoft.com/download/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
DBGHELP: D:\Projects\aliyun\CrashTracker\CrashTracker\bin\Release\Alibrowser.exe - file not found
DBGHELP: Alibrowser.exe not found in e:\symbols
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://10.1.170.157/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://msdl.microsoft.com/download/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
DBGENG: C:\Program Files\AliBrowser\Alibrowser.exe - Couldn't map image from disk.
Unable to load image C:\Program Files\AliBrowser\Alibrowser.exe, Win32 error 0n2
DBGHELP: No header for Alibrowser.exe. Searching for dbg file
DBGHELP: .\Alibrowser.dbg - file not found
DBGHELP: .\exe\Alibrowser.dbg - path not found
DBGHELP: .\symbols\exe\Alibrowser.dbg - path not found
DBGHELP: Alibrowser.exe missing debug info. Searching for pdb anyway
DBGHELP: Can't use symbol server for
Alibrowser.pdb - no header information available
DBGHELP: Alibrowser.pdb - file not found
*** WARNING: Unable to verify timestamp for Alibrowser.exe
*** ERROR: Module load completed but symbols could not be loaded for Alibrowser.exe
DBGHELP: Alibrowser - no symbols loaded
整个加载过程很长,直接一眼也看不出来哪里有问题。不过我们可以对比一下正常的加载过程。下面的代码是在一个可以正常加载symbol的dump里面执行的:
0:024> !sym noisy
noisy mode - symbol prompts on
0:024> .reload /i alibrowser.exe
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://10.1.170.157/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://msdl.microsoft.com/download/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
DBGHELP: D:\Projects\aliyun\CrashTracker\CrashTracker\bin\Release\Alibrowser.exe - file not found
DBGHELP: Alibrowser.exe not found in e:\symbols
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://10.1.170.157/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://msdl.microsoft.com/download/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
DBGENG: C:\Program Files\AliBrowser\Alibrowser.exe - Couldn't map image from disk.
Unable to load image C:\Program Files\AliBrowser\Alibrowser.exe, Win32 error 0n2
DBGENG: Alibrowser.exe - Partial symbol image load missing image info
DBGHELP: Module is not fully loaded into memory.
DBGHELP: Searching for symbols using debugger-provided data.
*** WARNING: Unable to verify timestamp for Alibrowser.exe
DBGHELP: Alibrowser - private symbols & lines
e:\symbols\Alibrowser.pdb\4A9A20AE996E4E5590C000C41FF40AB81\Alibrowser.pdb
仔细对比两者的加载过程,可以看到,前面大部分都完全一样,从红色部分开始,两者不一样。
从后者的这句话(DBGHELP: Searching for symbols using debugger-provided data)来看,加载symbol利用的是dump里面的symbol信息。
而前者里面的这块信息可能丢失或者损坏,导致没法正常加载symbol。
那么如何能够使前者能够加载symbol呢?我们可以看到前面一大段的日志都是在寻找alibrowser.exe,我猜想windbg一开始是想找到alibrowser.exe然后利用exe的信息来定位具体的symbol。
如果我们在他寻找的地方放一个exe,是不是就可以加载成功呢?
于是我在C:\Program Files\AliBrowser\下面放一个跟dump里面版本一样的alibrowser.exe。果然就加载成功了:
0:000> .reload /i alibrowser.exe
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://10.1.170.157/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://msdl.microsoft.com/download/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
DBGHELP: D:\Projects\aliyun\CrashTracker\CrashTracker\bin\Release\Alibrowser.exe - file not found
DBGHELP: Alibrowser.exe not found in e:\symbols
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://10.1.170.157/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://msdl.microsoft.com/download/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
DBGENG: C:\Program Files\AliBrowser\Alibrowser.exe - Mapped image memory
DBGHELP: Alibrowser - private symbols & lines
e:\symbols\Alibrowser.pdb\4A9A20AE996E4E5590C000C41FF40AB81\Alibrowser.pdb
在C:\Program Files\AliBrowser放置了对应版本的alibrowser.exe之后,发现大部分(7个中的6个)都可以正常加载symbol了,但是还有一个没法正常加载symbol。
我们看一下这个有问题的dump的加载过程:
0:004> !sym noisy
noisy mode - symbol prompts on
0:004> .reload /i alibrowser.exe
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://10.1.170.157/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
SYMSRV: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://msdl.microsoft.com/download/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
DBGHELP: D:\Projects\aliyun\CrashTracker\CrashTracker\bin\Release\Alibrowser.exe - file not found
DBGHELP: Alibrowser.exe not found in e:\symbols
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://10.1.170.157/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
SYMSRV: E:\Symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe not found
SYMSRV: http://msdl.microsoft.com/download/symbols/Alibrowser.exe/4F0C2D28e9e000/Alibrowser.exe not found
DBGENG: D:\Program Files\浏览器\AliBrowser\Alibrowser.exe - Couldn't map image from disk.
Unable to load image D:\Program Files\浏览器\AliBrowser\Alibrowser.exe, Win32 error 0n2
DBGHELP: No header for Alibrowser.exe. Searching for dbg file
DBGHELP: .\Alibrowser.dbg - file not found
DBGHELP: .\exe\Alibrowser.dbg - path not found
DBGHELP: .\symbols\exe\Alibrowser.dbg - path not found
DBGHELP: Alibrowser.exe missing debug info. Searching for pdb anyway
DBGHELP: Can't use symbol server for Alibrowser.pdb - no header information available
DBGHELP: Alibrowser.pdb - file not found
*** WARNING: Unable to verify timestamp for Alibrowser.exe
*** ERROR: Module load completed but symbols could not be loaded for Alibrowser.exe
DBGHELP: Alibrowser - no symbols loaded
可以看到,加载symbol的时候是去D:\Program Files\浏览器\AliBrowser\,也就是产生dump时alibrowser.exe所在路径去寻找的,而这个用户没有安装在默认路径,所以我们放在默认路径的alibrowser.exe没有作用。
另外,我们可以看到,windbg其实是首先到e:\symbols\Alibrowser.exe\4F0C2D28e9e000里面是寻找alibrowser.exe的,所以在里面放一个alibrowser.exe即可。
0:004> .reload /i alibrowser.exe
DBGHELP: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe - OK
DBGENG: e:\symbols\Alibrowser.exe\4F0C2D28e9e000\Alibrowser.exe - Mapped image memory
DBGHELP: Alibrowser - private symbols & lines
e:\symbols\Alibrowser.pdb\4A9A20AE996E4E5590C000C41FF40AB81\Alibrowser.pdb