On Windows 7, Outlook 2010 Hangs and can not be killed

Question

I'm having a strange issue with one of my users. The user upgraded to Office 2010, after some use, Outlook hangs up. When it does this the task can not be killed with task manager or the command taskkill /f /im outlook.exe . It reports a success, but it remains in the task list.

In addition, Outlook cannot be reopened while the Zombie task exists. Worst of all, Windows 7 cannot be shutdown properly while this task exists and sits at waiting for task to close and will never pass this screen forcing a hard reboot.

I've tried several things including repairing Office 2010, updating drivers, recreating his profile completely and disabling all programs and unneeded services including AV.

I ran a dump and ran a driver trace as seen below:

Microsoft (R) Windows Debugger Version 6.12.0002.633 X86
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\Windows\livekd.dmp]
Kernel Complete Dump File: Full address space is available

Comment: 'LiveKD live system view'
Symbol search path is: srv*c:\Symbols*http://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows 7 Kernel Version 7600 MP (2 procs) Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 7600.16617.x86fre.win7_gdr.100618-1621
Machine Name:
Kernel base = 0x82a4c000 PsLoadedModuleList = 0x82b94810
Debug session time: Sat Feb 12 21:34:57.897 17420 (UTC - 5:00)
System Uptime: 0 days 0:41:52.019
WARNING: Process directory table base 00185000 doesn't match CR3 DC36AA40
WARNING: Process directory table base 00185000 doesn't match CR3 DC36AA40
Loading Kernel Symbols
...............................................................
................................................................
.............................
Loading User Symbols

Loading unloaded module list
.....
*** ERROR: Module load completed but symbols could not be loaded for LiveKdD.SYS
0: kd> !process 0 7 outlook.exe
PROCESS 85eeaa58  SessionId: 1  Cid: 16cc    Peb: 7ffdb000  ParentCid: 0cc8
    DirBase: dc36a9a0  ObjectTable: 00000000  HandleCount:   0.
    Image: OUTLOOK.EXE
    VadRoot 00000000 Vads 0 Clone 0 Private 1. Modified 2415. Locked 0.
    DeviceMap 9c22b480
    Token                             af00c910
    ElapsedTime                       00:30:08.202
    UserTime                          00:00:09.297
    KernelTime                        00:00:04.087
    QuotaPoolUsage[PagedPool]         0
    QuotaPoolUsage[NonPagedPool]      0
    Working Set Sizes (now,min,max)  (6, 50, 345) (24KB, 200KB, 1380KB)
    PeakWorkingSetSize                31194
    VirtualSize                       470 Mb
    PeakVirtualSize                   489 Mb
    PageFaultCount                    58161
    MemoryPriority                    BACKGROUND
    BasePriority                      8
    CommitCharge                      0

No active threads

PROCESS 87f06d40  SessionId: 1  Cid: 08f0    Peb: 7ffdc000  ParentCid: 0cc8
    DirBase: dc36a840  ObjectTable: caa1c100  HandleCount: 803.
    Image: OUTLOOK.EXE
    VadRoot 85fe95b8 Vads 270 Clone 0 Private 2357. Modified 820. Locked 0.
    DeviceMap 9c22b480
    Token                             9c2e1910
    ElapsedTime                       00:26:13.180
    UserTime                          00:00:00.031
    KernelTime                        00:00:00.015
    QuotaPoolUsage[PagedPool]         0
    QuotaPoolUsage[NonPagedPool]      0
    Working Set Sizes (now,min,max)  (59485, 50, 345) (237940KB, 200KB, 1380KB)
    PeakWorkingSetSize                59485
    VirtualSize                       287 Mb
    PeakVirtualSize                   290 Mb
    PageFaultCount                    61763
    MemoryPriority                    BACKGROUND
    BasePriority                      8
    CommitCharge                      3627

        THREAD 85d753a0  Cid 08f0.0f70  Teb: 7ffdf000 Win32Thread: fd0c54f8 WAIT: (WrResource) KernelMode Non-Alertable
            8532a7b8  SynchronizationEvent
        IRP List:
            85e405a8: (0006,0100) Flags: 00000884  Mdl: 00000000
        Not impersonating
        DeviceMap                 9c22b480
        Owning Process            87f06d40       Image:         OUTLOOK.EXE
        Attached Process          N/A            Image:         N/A
        Wait Start TickCount      161805       
        Context Switch Count      1258             
        UserTime                  00:00:00.109
        KernelTime                00:00:00.202
        Win32 Start Address 0x2f411f08
        Stack Init b9bb9fd0 Current b9bb9558 Base b9bba000 Limit b9bb7000 Call 0
        Priority 11 BasePriority 8 UnusualBoost 0 ForegroundBoost 2 IoPriority 2 PagePriority 5
        ChildEBP RetAddr  Args to Child              
        b9bb9570 82abab25 85d753a0 00000000 807c6120 nt!KiSwapContext+0x26 (FPO: [Uses EBP] [0,0,4])
        b9bb95a8 82ab9423 85d75460 85d753a0 8532a7b8 nt!KiSwapThread+0x266
        b9bb95d0 82ab32cf 85d753a0 85d75460 0000000e nt!KiCommitThreadWait+0x1df
        b9bb964c 82aeb1a3 8532a7b8 0000001b 00000000 nt!KeWaitForSingleObject+0x393
        b9bb96a4 82ab66ce 8532a7b8 c0000055 b4542008 nt!ExpWaitForResource+0x16f
        b9bb96f4 9050e8ea 85fc0c88 d082ab01 29ebd2f3 nt!ExAcquireResourceExclusiveLite+0x1cf
        b9bb973c 90509bda b4542008 85ff25d0 00000001 rdbss!__RxAcquireFcb+0x18f (FPO: [Non-Fpo])
        b9bb97a8 9050e23f 85ff25d0 85e405a8 b9bb9868 rdbss!RxFindOrCreateFcb+0x341 (FPO: [Non-Fpo])
        b9bb9848 9050c920 85ff25d0 85e405a8 b9bb9868 rdbss!RxCreateFromNetRoot+0x3df (FPO: [Non-Fpo])
        b9bb987c 904f3fb1 00000000 85e405a8 29ebdccb rdbss!RxCommonCreate+0x2ae (FPO: [Non-Fpo])
        b9bb9904 9050ee2b 90504240 85e405a8 85e93a58 rdbss!RxFsdCommonDispatch+0x646 (FPO: [Non-Fpo])
        b9bb9934 94ce4298 87b5d020 00e405a8 85e40660 rdbss!RxFsdDispatch+0x1ab (FPO: [Non-Fpo])
        b9bb9950 82a884bc 87b5d020 01e405a8 85e40684 mrxsmb!MRxSmbFsdDispatch+0x9a (FPO: [Non-Fpo])
        b9bb9968 8c3f3bb0 00000103 85b983b8 85e93a58 nt!IofCallDriver+0x63
        b9bb9984 8c3f2b52 85b983b8 c0000016 85e40660 mup!MupiCallUncProvider+0x10f (FPO: [Non-Fpo])
        b9bb999c 8c3f2f5b 85b983b8 358489ee 00000000 mup!MupStateMachine+0x9b (FPO: [Non-Fpo])
        b9bb99e8 82a884bc 853422d8 85e405a8 85e405a8 mup!MupCreate+0x109 (FPO: [Non-Fpo])
        b9bb9a00 8c0fe20c 85e405a8 00000000 85e40684 nt!IofCallDriver+0x63
        b9bb9a24 8c1118c9 b9bb9a44 86a48c90 00000000 fltmgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x2aa (FPO: [Non-Fpo])
        b9bb9a70 82a884bc 86a48c90 86a48760 85e93ab4 fltmgr!FltpCreate+0x2db (FPO: [Non-Fpo])
        b9bb9a88 82c8c6ad 893d5e2e b9bb9c30 00000000 nt!IofCallDriver+0x63
        b9bb9b60 82c6d26b 853422d8 8530f488 85f76678 nt!IopParseDevice+0xed7
        b9bb9bdc 82c932d9 00000000 b9bb9c30 00000040 nt!ObpLookupObjectName+0x4fa
        b9bb9c38 82c8b62b 001bdf60 8530f488 00000001 nt!ObOpenObjectByName+0x165
        b9bb9cb4 82c96f42 001bdfbc c0100080 001bdf60 nt!IopCreateFile+0x673
        b9bb9d00 82a8f44a 001bdfbc c0100080 001bdf60 nt!NtCreateFile+0x34
        b9bb9d00 772164f4 001bdfbc c0100080 001bdf60 nt!KiFastCallEntry+0x12a (FPO: [0,3] TrapFrame @ b9bb9d34)
WARNING: Frame IP not in any known module. Following frames may be wrong.
        001bdf1c 5a3e4000 00000007 00008000 00000000 0x772164f4
        001bdfc4 00000000 00000000 06b84000 00000021 0x5a3e4000

0: kd> !irp 85e405a8
Irp is active with 4 stacks 2 is current (= 0x85e4063c)
 No Mdl: No System Buffer: Thread 85d753a0:  Irp stack trace.  
     cmd  flg cl Device   File     Completion-Context
 [  0, 0]   0  0 00000000 00000000 00000000-00000000    

   Args: 00000000 00000000 00000000 00000000
>[  0, 0]   0 e0 87b5d020 85e93a58 8c3ee3be-85b983b8 Success Error Cancel 
        \FileSystem\c mup!MupiUncProviderCompletion
   Args: b9bb9aac 01000040 00010080 00000000
 [  0, 0]   0 e0 853422d8 85e93a58 8c0fd4de-85f84448 Success Error Cancel 
        \FileSystem\Mup cr!FltpSynchronizedOperationCompletion
   Args: b9bb9aac 01000040 00010080 00000000
 [  0, 0]   0  0 86a48c90 85e93a58 00000000-00000000    
        \FileSystem\FltMgr
   Args: b9bb9aac 01000040 00010080 00000000

System Info:

S Name:                   Microsoft Windows 7 Professional 
OS Version:                6.1.7600 N/A Build 7600
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Member Workstation
OS Build Type:             Multiprocessor Free
Registered Owner:          ~~~~~~
Registered Organization:   Hewlett-Packard Company
Product ID:                ~~~~~~
Original Install Date:     1/14/2010, 1:26:39 PM
System Boot Time:          11/9/2010, 3:07:05 PM
System Manufacturer:       Hewlett-Packard
System Model:              HP Compaq dx7500 Microtower
System Type:               X86-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: x64 Family 6 Model 23 Stepping 10 GenuineIntel ~2936 Mhz
BIOS Version:              American Megatrends Inc. 5.14   , 5/11/2009
Windows Directory:         C:\Windows
System Directory:          C:\Windows\system32
Boot Device:               \Device\HarddiskVolume1
System Locale:             en-us;English (United States)
Input Locale:              en-us;English (United States)
Time Zone:                 (UTC-05:00) Eastern Time (US & Canada)
Total Physical Memory:     3,549 MB
Available Physical Memory: 2,145 MB
Virtual Memory: Max Size:  7,097 MB
Virtual Memory: Available: 5,870 MB
Virtual Memory: In Use:    1,227 MB

Solution

The only reasonable thing I see is this:

b9bb9a88 82c8c6ad 893d5e2e b9bb9c30 00000000 nt!IofCallDriver+0x63
b9bb9b60 82c6d26b 853422d8 8530f488 85f76678 nt!IopParseDevice+0xed7
b9bb9bdc 82c932d9 00000000 b9bb9c30 00000040 nt!ObpLookupObjectName+0x4fa
b9bb9c38 82c8b62b 001bdf60 8530f488 00000001 nt!ObOpenObjectByName+0x165
b9bb9cb4 82c96f42 001bdfbc c0100080 001bdf60 nt!IopCreateFile+0x673
b9bb9d00 82a8f44a 001bdfbc c0100080 001bdf60 nt!NtCreateFile+0x34

It's trying to open a file (Create File creates a File Object, not a File on the Drive!), it first looks up the Object Name and then it calls the Driver of the Device being referred to. (From the \\?\Device\File syntax)

It doesn't look like that it is something on the hard drive as that would be different, by going through MUP, yet another driver and RDBSS it seems that it is trying to access another kind of File. (cache, net?)

You will need to figure out the Object or Device Name to have a clue what might be going on,
you can do this by scanning the stack for strings.

You should check my explorer.exe hang where I capture the hang itself, use !analyze -hang -v and other commands to get more wise of the hang, the scanning helped me get to know what was wrong (something was rewriting C: to D:) and lucky someone showed me it was VCD doing this, I probably would've found it in the stack sooner or later... I guess something out of it might help you troubleshoot.

Comments

[Tamara Wijsman]

From what I see it looks like a blocked or inaccessible resource, I don't see a deadlock problem. He could try to identify the resource, it could help him find a solution...

[Rhys]

Great analysis, I was looking at that too. MUP.sys, from what I understand, controls UDP. So it may be attempting to write to a remote source when it dies? Anyway, I hate losing to an issue like this, but it was not practical to proceed with finding the issue. We are reformatting instead. If the issue still exists then I will have fun trying to figure this out.

[Rhys]

You know, I'm also wondering if it may be a FS problem. fltmgr was showing in there as well. microsoft.com/whdc/driver/filterdrv/default.mspx I'm going to do a quick chkdsk before formatting.