unmounted logical volume is "busy"?

Question

I'm trying to shrink an ext4 filesystem on a CentOS 6 server. I did a lazy unmount of the filesystem while I waited for some processes to finish running. They've all finished running but I can't seem to do anything with the filesystem. How can I see what is using the volume and stop it?

resize2fs

[root@planck ~]# resize2fs -P /dev/vg_dev/lv_home 
resize2fs 1.42.9 (28-Dec-2013)
resize2fs: Device or resource busy while trying to open /dev/vg_dev/lv_home
Couldn't find valid filesystem superblock.

fsck

[root@planck ~]# fsck /dev/vg_dev/lv_home 
fsck from util-linux-ng 2.17.2
e2fsck 1.42.9 (28-Dec-2013)
/dev/mapper/vg_dev-lv_home is in use.
e2fsck: Cannot continue, aborting.

umount

[root@planck ~]# umount /dev/vg_dev/lv_home
umount: /dev/vg_dev/lv_home: not mounted

lvs

  LV      VG     Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  lv_home vg_dev -wi-ao----  5.86t                                                    
  lv_root vg_dev -wi-ao---- 50.00g                                                    
  lv_swap vg_dev -wi-ao----  5.44g                                                    

Solution

fuser -mv /dev/vg_dev/lv_home should show you the process PID you need to kill to free up the device.

For example:

 # fuser -mv /dev/vg_dev/lv_home
                         USER        PID ACCESS COMMAND
    /dev/vg_dev/lv_home:
                         sbonds     9627 ..c.. bash

Now, you may not be able to actually kill it if it's blocked on I/O or some other uninterruptible reason.

Comments

[Aaron]

what is the output of lvs?

[b3m2a1]

@Aaron I added the output of lvs. looks the same as the mounted volumes.

[Aaron]

That suggests home is mounted. What do you get with fuser -c /home and umount /home and grep home /etc/mtab

[b3m2a1]

Sorry @Aaron, I've already rebooted. fuser -mv /home wasn't showing anything. umount /home gave the same not mounted error as umount /dev/vg_dev/lv_home, and mount did not show it as mounted (i assume mtab would have shown the same.)

[b3m2a1]

no output :( nice to know fuser accepts a block device, though.

[Steve Bonds]

One common non-process reason for a mount that won't move is another filesystem mounted on top, e.g. /home/some-other-mount. If that's not the case, you probably have a reboot in your near future. You need to shrink ext4 offline anyhow.

[b3m2a1]

yeah there wasn't another system mounted on top. i rebooted, and kept killing processes until i could non-lazy unmount. /home is offline, but there are many services running out of /var that i can keep online during the resize.

[rubo77]

now I see the process:/dev/dm-36: root kernel mount /media/snapshots/tmp-vm06.docker-disk – but how do i kill a process insde a snapshot?