Why would a partition be in use by the system if it isn't mounted?

Question

I've been migrating my system from btrfs to ext4 after running into performance issues with VMs. I have two hard drives in my laptop to work with. I've successfully moved my home partition, but the same steps I used aren't working for root.

Progress so far:

I've dd'd my root partition from /dev/sda3 into /dev/sdb3. I modified /etc/fstab to the following:

$ cat /etc/fstab
# 
# /etc/fstab: static file system information
#
# <file system> <dir>   <type>  <options>       <dump>  <pass>
# UUID=95f13c34-96ca-49e3-bcb2-ff594df31506
/dev/sdb3               /               btrfs           rw,noatime,ssd,space_cache,discard      0 0

# UUID=0fe04f59-599f-41e2-ac30-2ad0f17a9727
/dev/sda2               /boot           ext2            rw,relatime     0 2

# UUID=44741e0f-924a-4841-80ef-2132bef84182
/dev/sda4               /home           ext4            rw,noatime,discard      0 0

and run sudo mkinitcpio -p linux. It seems to work. I'm able to boot by mounting the partition on the second disk. df shows:

$ df
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb3        28G   18G  9.8G  65% /

So, clearly, sdb3 is mounted, not sda3. Here's the problematic step: When I try to format sda3, which is supposedly unused, I get the following:

$ sudo mkfs.ext4 /dev/sda3
[sudo] password for stew: 
mke2fs 1.42.11 (09-Jul-2014)
/dev/sda3 contains a btrfs file system
Proceed anyway? (y,n) y
/dev/sda3 is apparently in use by the system; will not make a filesystem here!

sda3 is in use. How and why could it possibly be in use?

As per casey's comment, the output of mount:

mount | grep sd
/dev/sdb3 on / type btrfs (rw,noatime,ssd,discard,space_cache)
/dev/sda4 on /home type ext4 (rw,noatime,discard,data=ordered)
/dev/sda2 on /boot type ext2 (rw,relatime)

As per Warwick's comment, unmounting:

$ sudo umount /dev/sda3
umount: /dev/sda3: not mounted

Mounting and umounting sda3 elsewhere works successfully, but changes nothing.

Update: More fishy behavior:

$ mount | grep sd
/dev/sdb3 on / type btrfs (rw,noatime,ssd,discard,space_cache)
/dev/sda4 on /home type ext4 (rw,noatime,discard,data=ordered)
/dev/sda2 on /boot type ext2 (rw,relatime)
$ sudo mount /dev/sda3 mnt
[sudo] password for stew: 
$ mount | grep sd
/dev/sda3 on / type btrfs (rw,noatime,ssd,discard,space_cache)
/dev/sda4 on /home type ext4 (rw,noatime,discard,data=ordered)
/dev/sda2 on /boot type ext2 (rw,relatime)
/dev/sda3 on /home/stew/mnt type btrfs (rw,relatime,ssd,discard,space_cache)

After mounting sda3, sdb3 is no longer mounter. Weird, huh?

As per mikeserv:

$ rmmod btrfs
rmmod: ERROR: Module btrfs is in use

This is very much expected, since sdb3 is btrfs and supposed to be mounted to root. From my mkinitcpio.conf file:

MODULES=""
HOOKS="base udev autodetect modconf block filesystems keyboard fsck"

Solution

I figured it out. My bootloader wasn't configured properly. Sounds obvious, right? Modifying fstab doesn't quite qualify as configuring the bootloader. I had to change a line in /boot/syslinux/syslinux.cgf to refer to correct boot partition.

That said, there was no need to boot off of the second disk in the first place. I could have avoided this problem by completing the whole process in a live environment and chroot-ing in to run mkinitcpio.

Comments

[Kyle Jones]

So you booted off sda3 then mounted sdb3 on / over top of it, which overwrote the mtab entry, hence the mount command output showing no sign of sda3.

[stewSquared]

@KyleJones Yup. That sums it up.