IISAdmin won't start, error 0x80090006 - how to troubleshoot further?

Question

A while back my Windows Server 2003 machine hung up and I needed to force-reboot by power-cycling it. When it was restarting it ran chkdsk, and chkdsk reported some problems.

Since then IISAdmin has not been able to start (which means that IIS doesn't work at all):

C:\WINDOWS\system32>sc query iisadmin

    SERVICE_NAME: iisadmin
    TYPE               : 20  WIN32_SHARE_PROCESS
    STATE              : 1  STOPPED
                            (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN))
    WIN32_EXIT_CODE    : 1066  (0x42a)
    SERVICE_EXIT_CODE  : -2146893818  (0x80090006)
    CHECKPOINT         : 0x0
    WAIT_HINT          : 0x0

I suspected a corrupt Metabase so I tried restoring from a Metabase History file dated earlier than the crash -- but no luck.

Not really sure how to troubleshoot this further. There's one apparently-related error in the SYSTEM log that comes up regularly, event ID 10010 from DCOM:

The server {A9E69610-B80D-11D0-B9B9-00A0C922E750} did not register with DCOM
within the required timeout.

But I'm not sure if this actually sheds additional light or if it's just another warning that IISAdmin can't start.

Otherwise the Event logs are pretty empty of help. What's the next step to isolate this problem and fix it? I should point out that I don't have the luxury of rebooting this server very often -- it's an app server, and the users who depend on the app work until late at night, and I would need to be on-site to reboot it because it doesn't always restart reliably -- but I normally work off-site.

So if there's any way to keep investigating this without needing to reboot, I'm eager to hear it.

Solution

According to winerror.h, 0x80090006 means "NTE_BAD_SIGNATURE", which is a return from the Win32 API call "CryptVerifyMessageSignature", and means that "The message's signature was not verified." Looks like a failure of a cryptographic signature check on some file related to the iisadmin startup, to me.

I'd consider doing an "SFC /SCANNOW" to see if you can replace the screwed-up file, assuming that the System File Checker can get to some Windows installation files, and assuming that it is a screwed-up file. (Yes-- "screwed-up" is a technical term. Sounds better than "corrupt" to me. Corrupt? Was the file taking bribes?)

If you're not getting it to work after an "SFC /SCANNOW", I'd probably fire up "Process Monitor" and watch what files are getting loaded. There will be a LOT of log entries generated, but you might find the needle in that haystack.

Given that the machine took a hard power-down and had some filesystem corruption, I'd put my money on "SFC /SCANNOW" finding one or more damaged files.

Aside: So, it won't restart reliably, eh? Gives me a "The web site is down!" kind of feeling... (DO NOT REBOOT THE WEB SERVER!!!!!! Heh heh...)

Comments

[Ben Dunlap]

Oh. Didn't realize I could do that. I thought I had to remove and then re-add the "application server" role -- which would also turn off some other stuff that our non-web-based apps rely on, thus being the practical equivalent of a reboot. I will try the Windows Components approach if I can't dig anything up with SFC or Procmon, thanks.

[Spence]

"SFC /SCANNOW", from a command prompt, will do what you want. I'd do that before I'd reinstall IIS.

[Ben Dunlap]

It just occurred to me -- do you happen to know if the SFC utility uses VSS? Because that doesn't work on this server (yes, I know, the server is a wreck. Unfortunately it's also our primary machine and the most expensive piece of equipment in the building). It was actually a VSS operation that caused the lock-up that led to the hard-restart... ugh.

[Ben Dunlap]

OK it's not as expensive as our digital printing press. Most expensive piece of strictly-IT equipment in the building.

[Spence]

No, SFC doesn't use VSS.

[Ben Dunlap]

OK, thanks. I've found a workaround for the time being that doesn't require IIS but I will almost certainly need to get IIS running on this server soon - so I will fiddle with it during off-peak hours and report back.