Using WinDBG to Identify Defective Function

Question

I've installed WinDBG from the 7.1 Windows SDK. Then with VC++ 2008 I made a program 'CleanPayload.exe' which contains nothing but a 'main' and an invocation to a function that intentionally contains a defect. It is a release build which includes debug symbols. I opened that program into WindDBG and then

1. did a .sympath+ to indicate where the PDB was for that program.
2. did a ld * to load all symbols
3. did a lm to verify all symbols were loaded (private symbols for my program, public symbols for Windows libraries).

I then ran the program and it threw a first-chance exception, which was rather expected. As follows:

(910.12a0): WOW64 breakpoint - code 4000001f (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
ntdll32!LdrpDoDebuggerBreak+0x2c:
771e0f2b cc              int     3 

But when I ask WinDBG to show me the stack, it doesn't show me anything of my program 'CleanPayload.exe'. Instead it shows me this:

0:000:x86> kb
ChildEBP RetAddr  Args to Child              
004bf5ec 771c122b 7efdd000 7efde000 7724206c ntdll32!LdrpDoDebuggerBreak+0x2c
004bf764 77192187 004bf7d8 77140000 7c185e6a ntdll32!LdrpInitializeProcess+0x132f
004bf7b4 77179e89 004bf7d8 77140000 00000000 ntdll32!_LdrpInitialize+0x78
004bf7c4 00000000 004bf7d8 77140000 00000000 ntdll32!LdrInitializeThunk+0x10

What do I need to do so that it will show me a stack trace which (1) includes my program and (2) the function where the exception was thrown?

Update I followed Larry's suggestion, to run past the first exception, and got the following results:

0:000:x86> g
ntdll!NtTerminateProcess+0xa:
00000000`76faf97a c3              ret
0:000> kb
RetAddr           : Args to Child                                                           : Call Site
00000000`74c6601a : 00000000`00000000 00000000`000de600 00000000`000ddc80 00000000`74c60304 : ntdll!NtTerminateProcess+0xa
00000000`74c5cf87 : 00000000`0030f988 00000000`0030dba8 00000000`7efdb000 00000000`0030f934 : wow64!whNtTerminateProcess+0x46
00000000`74be276d : 00000000`77150190 00000000`74c50023 00000000`00000000 00000000`0030fab8 : wow64!Wow64SystemServiceEx+0xd7
00000000`74c5d07e : 00000000`00000000 00000000`74be1920 00000000`000de820 00000000`76f93501 : wow64cpu!TurboDispatchJumpAddressEnd+0x24
00000000`74c5c549 : 00000000`00000000 00000000`00000000 00000000`74c54ac8 00000000`7ffe0030 : wow64!RunCpuSimulation+0xa
00000000`76faae27 : 00000000`004a3100 00000000`00000000 00000000`7707a1e0 00000000`7efdf000 : wow64!Wow64LdrpInitialize+0x429
00000000`76fa72f8 : 00000000`00000000 00000000`76fa8641 00000000`76fb84e0 00000000`00000000 : ntdll!LdrpInitializeProcess+0x1780
00000000`76f92ace : 00000000`000df1b0 00000000`00000000 00000000`7efdf000 00000000`00000000 : ntdll! ?? ::FNODOBFM::`string'+0x2af20
00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!LdrInitializeThunk+0xe

Thus, unfortunately, I'm still not seeing relevant stack trace information. I also tried the .effmach x86 command, before the steps above, and it didn't seem to have an impact. Incidentally, I also then re-executed the whole test with app verifier activated for the target program I'm testing. I got very conflicting results:

0:000> g
ModLoad: 00000000`76d40000 00000000`76e5f000   WOW64_IMAGE_SECTION
ModLoad: 00000000`74f90000 00000000`75090000   WOW64_IMAGE_SECTION
ModLoad: 00000000`76d40000 00000000`76e5f000   NOT_AN_IMAGE
ModLoad: 00000000`76e60000 00000000`76f5a000   NOT_AN_IMAGE
ModLoad: 00000000`71160000 00000000`711c0000   C:\Windows\syswow64\verifier.dll
Page heap: pid 0x1A54: page heap enabled with flags 0x3.
AVRF: CleanPayload.exe: pid 0x1A54: flags 0x80643027: application verifier enabled
ModLoad: 00000000`71130000 00000000`7115b000   C:\Windows\SysWOW64\vrfcore.dll
ModLoad: 00000000`710d0000 00000000`71128000   C:\Windows\SysWOW64\vfbasics.dll
ModLoad: 00000000`74f90000 00000000`75090000   C:\Windows\syswow64\kernel32.dll
ModLoad: 00000000`76830000 00000000`76876000   C:\Windows\syswow64\KERNELBASE.dll
ModLoad: 00000000`715c0000 00000000`7164e000   C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.4926_none_508ed732bcbc0e5a\MSVCP90.dll
ModLoad: 00000000`73dc0000 00000000`73e63000   C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.4926_none_508ed732bcbc0e5a\MSVCR90.dll
(1a54.17dc): WOW64 breakpoint - code 4000001f (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
ntdll32!LdrpDoDebuggerBreak+0x2c:
771e0f2b cc              int     3

0:000:x86> !avrf
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Your debugger is not using the correct symbols                 ***
***                                                                   ***
***    In order for this command to work properly, your symbol path   ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: wow64!_TEB32                                  ***
***                                                                   ***
*************************************************************************
Application verifier is not enabled for this process.
Use appverif.exe tool to enable it.

The above execution saysAVRF: Cleanpayload.exe ... application verifier enabled, which indicates that it is locked-on to the target. But then the subsequent !avrf command reveals that debug symbols are bad, even though the lm command shows they are all properly loaded! What on earth is happening here?

Solution 1

You're running the 64bit version of windbg and a 32bit application. The initial breakpoint is running in 64bit code.

If you hit "g" you should hit the initial breakpoint for the 32bit application, you should be able to go from there.

To switch from 64bit debugging to 32bit debugging (if you hit CTRL-C for example), type in:

.effmach x86

which will switch the debugger from 64bit mode to 32bit mode.

Solution 2

Are you trying to figure out how to debug real problems once your software is packaged up and sent to QA or customers? If yes, there's another tool you could use, adplus. Adplus launches the debugger under the hood and has only one purpose (actually two, if you ran in hang mode, but that's not what you want here), which is to wait for exceptions. When an exception happens, it will generate a process memory dump file which can be loaded into WinDbg.

Using this approach, you don't have to rely on QA or your customers to know how to work with WinDbg. You simply give them instructions how to run one command line. After they run it, they just zip up the entire output directory and send it to you for analysis.

Once loaded into WinDbg, the memory dump file will show you the exact place of the exception and all local/member variables at the time (although you may have to fish a bit for those values if your code is optimized).

Comments

[Brent Arias]

Evidently, 32 bit debuggin in 64 bit WinDBG is half the problem. The other half is finding the right debug symbols for the 32 bit program. Note this discussion: http://www.eggheadcafe.com/software/aspnet/29430292/teb32-an‌​d-peb32-type-info-mi‌​ssing-from-public-wo‌​w64pdb.aspx. That said, if I'm debugging a 64 bit application, in order to make 'application verifier' happy, I must be sure to place 'c:\windows\system32' in the symbol path before 'srv*'.

[Larry Osterman]

Good one, I'd forgotten the symbol path issue as well.

[Sabuncu]

@LO: Can you explain this a bit further please? What do you mean by the initial breakpoint being in 64bit code? Specifically, can you please compare it against the same scenario under windbg32? Also, this initial exception that is generated: is it inserted by the debugger so that the debuggee will stop upon invocation? THANKS.

[Larry Osterman]

When you launch a 32bit app in 64bit windbg, the initial breakpoint is in the WOW64 (Windows-On-Windows 64bit) initialization. You need to hit "G" once more to hit the 32bit initial breakpoint. It's an artifact of how 32bit apps work under a 64bit debugger. I'm not sure of the exact reasons why, but I know that it's just something you have to do.

[Yauheni Sivukha]

Author already had acquired the dump. He has problems analyzing it though.

[DXM]

@YauheniSivukha - probably not when I posted this answer... over 3 years ago