LUKS initramfs boot problem: "/dev/mapper/ubuntu-root does not exist." How can I fix it using a backup?

Question

First time asker, relatively new to Ubuntu/Linux. Haven't had much luck with the search function. Please forgive the essay below - the problem is a bit unique, I think...

I'm running 12.04 on a laptop (with UEFI BIOS), with full disk encryption set up up using the Alternate CD. I recently made a backup image of the entire hard drive using dd, to an external USB hdd. Having recently made some changes on the laptop, I tried to plug in the external USB clone to copy the files to it. Despite getting the password prompt and having it appear in the Disk Utility as a logical volume group, it would not actually mount the external drive, saying it was 'not a mountable file system'. Tried it on my desktop, and it mounted up just fine (once I installed lvm2).

Using pvdisplay, I noticed that both the external drive and internal laptop drive had the same UUID (duh, it's a clone!). So, on the desktop, I used pvchange -u to change the external drive's uuid. Plugged it into the laptop, alas, still no joy. Gave up, turned off the laptop (drive still plugged in), went and had dinner. Came back, unplugged external drive from now-off laptop, tried to boot. Problems!

It boots to the password prompt screen fine. Enter the password, and after a long wait, it drops to an initramfs prompt, with the error: "ALERT! /dev/mapper/ubuntu-root does not exist." Poop.

If I reboot, and plug back in the external drive, it boots up, seemingly running root off the USB drive. I have tried the solution here, entering /dev/sda3 (and yet more variants) instead. No bingo, still get exactly the same messages. The fact I'm using LUKS with a LVM seems to complicate things. I think I've muddled a config file somewhere, probably by plugging in two drives with the same UUID (stupid!) and now it thinks /root is on the external drive. I'm stumped as to how to get it back.

Solution

I FIXED IT!! For the references of future generations, heres how:

The problem was that by plugging the a clone of the system I was running into it, I'd "confused" my system as to which drive to write to, and somehow nuked the Volume Group metadata for the original internal drive. Luckily, I still had the metadata from the clone I'd made.

This meant that I could plug in the working backup, unlock it, and use vgcfgbackup to make a copy of its metadata.

~$ sudo vgcfgbackup -f /tmp/trousers
  Volume group "ubuntu" successfully backed up.

This creates a text file with the metadata in /tmp called trousers (called it whatever you like, makes no difference)

Then, I dismounted and unplugged the functioning backup clone, and plugged in the borked internal drive. Unlocked it like I did the backup, then run:

 sudo vgcfgrestore -f /tmp/trousers ubuntu
  Restored volume group ubuntu

Upon doing this, the volume group suddenly sprang to life in disk utility, and the borked internal was unborked, mountable and readable again!

Critical bit here is that 'ubuntu' was the name of the original volume group. You can verify this by opening up the 'trousers' file in gedit and looking at the entry on the line below "creation_time".

Also worth noting that this only worked because the backup was an exact clone of the original internal, and hence the VG metatdata file had all the correct UUID's in place. I actually got an error message first time round because I'd originally manually changed one of the UUID's on the external drive. I had to use pvdisplay on the original internal to find its UUID, and then manually edit the 'trousers' file to get it to work.

Anyway, hope that helps someone one day.

Comments

[Flimm]

Did you update /etc/crypttab after modifying the UUID? Also, remember that the LUKS volume UUID, the volume group UUID, the physical volume UUID, the logical volume UUID and the underlying filesystem UUID are all different, so all of them may need to be modified.

[jdeks]

No, I didn't edit /etc/cryptab - not sure what that is to be honest. Thanks for the info about UUID's I didn't realize that either. So how do I use this to get the laptop up and running again? I'm more focused on getting it going than going through and correcting all the UUID's on the external backup - that's going to be wiped and re-done once I fix this.

[jdeks]

Update: Removed the internal laptop drive, plugged it into my desktop and it wouldn't read it. Ran lvscan and pvscan, and it doesn't pick it up as being part of a logical volume group. Did the same thing to the backup clone: detects it just fine. Belongs to the 'ubuntu' volume group, mounts everything OK. Any ideas?

[jdeks]

C'mon, no-one? I'm really up the creek without a paddle here, and I need to get this thing back on-line for work in the new year. Even if I can just find a way to read the encrypted volume off another computer to I can salvage my un-backed up data. The crux of the issue seems to be that the internal laptop drive has a logical volume, but doesn't seem to have a logical volume group. Is there any way to re-create the volume group without nuking the logical volume?

[Flimm]

Hi jdeks. If you catch me online I'm willing to chat with you on IRC about it. My username is Flimm. Alternatively, try ubuntuforums.org, it's more suitable for troubleshooting requests. Ask Ubuntu is really about specific questions with specific answers.

[jdeks]

Hi Flimm, thanks very muc. I've already posted this on ubuntuforums, I put it here too because no-one there help. What server and channel are you on with IRC?

[Flimm]

Freenode, often on #ubuntu.

[Flimm]

Woot! Woot! Glad you fixed it. I didn't know about vgcfgbackup so I learnt something today.

[jdeks]

Quite the happy new year! Thanks for taking the time to help me, very much appreciated.