Module vfat fails to load (I've tried the obvious fixes)

Question

The module vfat doesn't load on boot, and attempts to force the issue with a modprobe vfat produce the error

modprobe: ERROR: could not insert 'vfat': Unknown symbol in module, or unknown parameter (see dmesg)

with the dmesg lines

[  663.227894] fat: Unknown symbol __bread_gfp (err 0)
[  663.227924] fat: Unknown symbol __getblk_gfp (err 0)

There are also two [FAILED] messages at boot-time advising me to run systemctl status systemd-modules-load.service; the result of doing so is:

● systemd-modules-load.service - Load Kernel Modules
   Loaded: loaded (/lib/systemd/system/systemd-modules-load.service; static)
   Active: failed (Result: exit-code) since Fri 2016-02-12 12:55:11 EST; 18min ago
     Docs: man:systemd-modules-load.service(8)
           man:modules-load.d(5)
 Main PID: 502 (code=exited, status=1/FAILURE)

Feb 12 12:55:11 aleph systemd-modules-load[502]: Failed to insert 'fuse': No such file or directory
Feb 12 12:55:11 aleph systemd[1]: systemd-modules-load.service: main process exited, code=exited, status=1/FAILURE
Feb 12 12:55:11 aleph systemd[1]: Failed to start Load Kernel Modules.
Feb 12 12:55:11 aleph systemd[1]: Unit systemd-modules-load.service entered failed state.

I am running a basically vanilla Debian Jessie and haven't hand-tweaked anything about my kernel. uname -a returns

Linux aleph 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) x86_64 GNU/Linux

and modinfo fat vfat:

filename:       /lib/modules/3.16.0-4-amd64/kernel/fs/fat/fat.ko
license:        GPL
depends:        
intree:         Y
vermagic:       3.16.0-4-amd64 SMP mod_unload modversions 
filename:       /lib/modules/3.16.0-4-amd64/kernel/fs/fat/vfat.ko
author:         Gordon Chaffee
description:    VFAT filesystem support
license:        GPL
alias:          fs-vfat
depends:        fat
intree:         Y
vermagic:       3.16.0-4-amd64 SMP mod_unload modversions 

Everything I've read from Google searches on the specifics of the error suggests that the problem here is a mismatch between the versions of the running kernel and the modules selected by kmod. To that end I've taken the two obvious steps suggested in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=808380 and vfat not recognized in debian for fixing that problem: first, I tried rebooting, and then I used apt-get install --reinstall linux-image-3.16.0-4-amd64 to force a reinstall and rebooted after. debsums linux-image-3.16.0-4-amd64 also indicates that my current kernel should be OK. The problem persists, however.

I could probably fix this by compiling my own kernel and modules, but I'd really consider going outside of the debian binaries a bit of a last resort.

Solution

OK, the problem turned out to be the usual (i.e., wrong kernel), with a slight wrinkle: for some reason which no doubt made sense at the time I did it, I had grub-pc installed as a debian package, but had LILO (not installed as a package) running as my actual bootloader, so kernel installs (and reinstalls, and suchlike) cheerfully updated grub, which had no effect on the kernel image actually being loaded on boot time. There's still a known bug that a specific Debian kernel/module update didn't increment the version number which throws kmod's version selection off (and contributed towards my impression that I had no kernel/module mismatch, since lsmod and uname were giving me the same version number), but that bug's normally easily fixed by rebooting to load the right kernel---but not in this instance, where the bootloader still had the old kernel.

Comments

[Rui F Ribeiro]

try doing a sudo depmod -a and then loading vfat again

[Rui F Ribeiro]

Would you provide the output of uname -r?

[Jake Wildstrom]

sudo depmod -a produced no output, didn't affect the result of a sudo modprobe vfat described above. uname -r returns 3.16.0-4-amd64.

[Rui F Ribeiro]

have you tried to reboot?

[Jake Wildstrom]

If you mean a reboot since running depmod -a, yup, just tried and the situation remains unchanged.

[Giacomo Catenazzi]

Kernel and modules doesn't match. Try reinstalling the kernel. (check also if your disk is full). If all fails, write a bug report.

[Jake Wildstrom]

Plenty of free disk space, and I thought the apt-get install --reinstall linux-image-3.16.0-4-amd64 invocation did reinstall the kernel; is there something else I should be doing instead? And if it's actually a mismatch between my kernel and modules, why are the reported versions on modinfo and uname the same?

[Jake Wildstrom]

As mentioned in the post, I have (and have confirmed the integrity of) the installed package linux-image-3.16.0-4-amd64; linux-headers-3.16.0-4-amd64 was not installed, so I've gone ahead and installed it, although it's not clear to me how that will help --- isn't it relevant chiefly to building a kernel or kernel modules from source?