How to debug what driver causes a kernel-panic?

Question

After: Fixing recursive fault but reboot is needed on x3850 x6 SLES12

Q: I was thinking that if I get a kernel panic on a linux server, how can I debug, what driver causes the kernel-panic?

In my case, it seems to have been due to an emulex driver issue, in that since updating that driver, kernel panics have no longer occurred.

Can someone elaborate the step-by-step recommended procedure for investigating the cause of a kernel panic? Please include where to save/how the coredump from a kernel panic, ex.: if a liveCD paniced until where I can see what driver did the crash.

Solution

In regards to your previous issue, I would add debug to your kernel cmdline to see more verbose output.

As far as recovering from a kernel crash, I usually boot into a livecd and check the logs. A simple way to troubleshoot drivers is to blacklist the ones you think may be causing the crash, working backwards until the system boots again (or at all), and then enabling each one individually until you get your system to crash again.

Here are some other resources for debugging the linux kernel:

• RedHat Linux: Kernel Crash Dump Guide
• Ubuntu Wiki: Kernel Debugging Tricks