GRUB won't boot persistent - "hd0,4 not found"

Question

I'm trying to make a persistent USB flash drive of Ubuntu 17.04 with mkusb (dus). I've followed the directions on the mkusb website explicitly, and I've tried most variations I can think of, all with no joy. When I select the persistent USB in BIOS, grub boots from it; I select Ubuntu persistent; grub objects with hd0,4 not found. I've been searching online for a solution for several days now.

Some of the things I've tried: - Creating the persistent drive on a second flash drive while booted from a live-USB flash drive; - All combinations of BIOS legacy mode and secure boot enabled and disabled; - msdos and gpt partition tables on the target drive; - different flash drives; - different amounts of persistent memory; - etc.

Any suggestions will be greatly appreciated.

My system: HP laptop with AMD64 quad core, 16GB memory, booting (usually) in UEFI secure mode. BIOS is the latest. The installed OS is Ubuntu 17.04.

Thanks!! -Warren

Edit #1 to add:

On further investigation, it seems that grub loads from the flash drive as hd2, not hd0. This is probably because the two internal hard drives are discovered by the BIOS first and are recognized as hd0 and hd1. How to get grub on a persistent flash drive to recognize that it has been loaded from hd2 and not hd0 is presently above my pay grade.

Edit #2 to add:

@sudodus- thanks for the help. Some of the info requested:

The mkusb and dus components are all loaded from the ppa and are version 12.0.9. BIOS has legacy mode disabled and secure boot enabled. I tried editing the grub commands ("e") and changing hd0 to hd2. That works! The persistent flash drive boots as expected. (Wonder why grub doesn't just set root to the disk it was loaded from ...?)

Output from the commands--

warren@warren-laptop:~$ sudo lsblk -fm
NAME   FSTYPE  LABEL              UUID                                 MOUNTPOINT                       NAME     SIZE OWNER GROUP MODE
sda                                                                                                     sda    953.9G root  disk  brw-rw----
├─sda1 vfat                       1168-7BED                            /boot/efi                        ├─sda1   256M root  disk  brw-rw----
├─sda2 ext4                       182d8f07-567e-4514-9f99-49b78767f195 /                                ├─sda2 921.6G root  disk  brw-rw----
└─sda3 swap                       8187044e-0939-4868-b81a-b5bc64bd9e7c [SWAP]                           └─sda3    32G root  disk  brw-rw----
sdb                                                                                                     sdb    698.7G root  disk  brw-rw----
└─sdb1 ext4    devel              27a0c109-82be-4ceb-b572-dedbeb412744 /home/warren/devel               └─sdb1 698.6G root  disk  brw-rw----
sdc                                                                                                     sdc     14.5G root  disk  brw-rw----
├─sdc1 ntfs    usbdata            134E2DAF76E6A9A4                     /media/warren/usbdata            ├─sdc1   3.2G root  disk  brw-rw----
├─sdc2                                                                                                  ├─sdc2     1M root  disk  brw-rw----
├─sdc3 vfat    usbboot            2EA6-44A9                                                             ├─sdc3   122M root  disk  brw-rw----
├─sdc4 iso9660 Ubuntu 17.04 amd64 2017-04-12-03-44-04-00               /media/warren/Ubuntu 17.04 amd64 ├─sdc4   1.5G root  disk  brw-rw----
└─sdc5 ext4    casper-rw          8a27c645-3e5c-4e7f-bf92-eb51dc0bd19f /media/warren/casper-rw          └─sdc5   9.6G root  disk  brw-rw----
warren@warren-laptop:~$ sudo parted -ls
Model: ATA SanDisk SD8SB8U1 (scsi)
Disk /dev/sda: 1024GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system     Name  Flags
 1      1049kB  269MB   268MB   fat32           efi   boot, esp
 2      269MB   990GB   990GB   ext4
 3      990GB   1024GB  34.4GB  linux-swap(v1)


Model: ATA ST750LX003-1AC15 (scsi)
Disk /dev/sdb: 750GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags: 

Number  Start   End    Size   File system  Name   Flags
 1      16.8MB  750GB  750GB  ext4         devel


Model:  Patriot Memory (scsi)
Disk /dev/sdc: 15.5GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name     Flags
 2      1049kB  2097kB  1049kB               primary  bios_grub
 3      2097kB  130MB   128MB   fat32        primary  boot, esp
 4      130MB   1759MB  1629MB               primary
 5      1759MB  12.1GB  10.3GB  ext2         primary
 1      12.1GB  15.5GB  3436MB  ntfs         primary  msftdata


warren@warren-laptop:~$ df
Filesystem     1K-blocks      Used Available Use% Mounted on
udev             7776108         0   7776108   0% /dev
tmpfs            1559788     10204   1549584   1% /run
/dev/sda2      951092244 187079792 715676736  21% /
tmpfs            7798928        12   7798916   1% /dev/shm
tmpfs               5120         4      5116   1% /run/lock
tmpfs            7798928         0   7798928   0% /sys/fs/cgroup
/dev/sda1         258095      3496    254599   2% /boot/efi
/dev/sdb1      720011344 242201684 441165412  36% /home/warren/devel
tmpfs            1559784       168   1559616   1% /run/user/1000
/dev/sdc4        1571328   1571328         0 100% /media/warren/Ubuntu 17.04 amd64
/dev/sdc1        3355644     17388   3338256   1% /media/warren/usbdata
/dev/sdc5        9909328     36912   9352660   1% /media/warren/casper-rw
warren@warren-laptop:~$ 

Thanks!!

Solution 1

Trying to find a general bugfix

Normally the computer will identify the boot drive as hd0, but this is not the case in your computer. I will try to find a way to get around that problem. I tested with Lubuntu 17.04 in my Toshiba laptop in UEFI mode, and could not reproduce the bug. So I will need your help to test whatever bugfix I can find.

Edit: I have modified the shellscript dus-persistent of mkusb and uploaded the new version 12.1.1 to the unstable PPA. The main improvements address a change in the boot structure of Ubuntu 17.04 and the fact that your computer does not boot from hd0. The main difference is how to identify partition #4 with the image from the iso file (the iso9660 file system),

search --set=root --fs-uuid <the-UUID-of-partition-no-4>

and it is done automatically by mkusb-dus. Please test if it solves the problem! You get the new version according to this link,

help.ubuntu.com/community/mkusb/gui#from_the_unstable_PPA

The unstable version is still developed and debugged. It is available from ppa:mkusb/unstable via the following command lines,

sudo add-apt-repository universe  # this line only for standard Ubuntu

sudo add-apt-repository ppa:mkusb/unstable
sudo apt-get update
sudo apt-get install mkusb               # install
# sudo apt-get dist-upgrade              # upgrade to current version (with all other upgrades), only for installed systems
sudo apt-get install mkusb guidus dus mkusb-common
                                         # upgrade all mkusb basic components including dus

# for persistent live drives that work in UEFI and BIOS mode with 32-bit iso files, add
sudo apt-get install usb-pack-efi

Workaround - bugfix in your case

Thanks for the feedback (that you edited into the original question). It makes it easier to help.

"I tried editing the grub commands ("e") and changing hd0 to hd2. That works! The persistent flash drive boots as expected. (Wonder why grub doesn't just set root to the disk it was loaded from ...?)"

I would suggest editing the grub.cfg file in partition #3 of the 'USB drive to be persistent' (the 'usbboot' partition). This should make the bugfix persistent in your USB flash drive. But it would not work in other computers, that behave like my computers.

$ ls -l /media/lubuntu/usbboot/boot/grub/grub.cfg 
-rw-r--r-- 1 lubuntu lubuntu 1312 Apr 29 06:31 /media/lubuntu/usbboot/boot/grub/grub.cfg

grub.cfg has the following content, where you modify hd0,4 to hd2,4. (You might also modify hd0,3 to hd2,3, but Memtest86+ will not work in UEFI mode anyway.)

In your case you can expect 'Ubuntu' instead of 'Lubuntu', but otherwise it would look the same.

set timeout=10
set default=0

menuentry "Lubuntu-17.04-desktop-amd64.iso - persistent live" {
 set root=(hd0,4)
 linux ($root)/casper/vmlinuz.efi boot=casper quiet splash persistent --
 initrd ($root)/casper/initrd.lz
}
menuentry "Lubuntu-17.04-desktop-amd64.iso - persistent live to RAM" {
 set root=(hd0,4)
 linux ($root)/casper/vmlinuz.efi boot=casper quiet splash toram persistent --
 initrd ($root)/casper/initrd.lz
}
menuentry "Lubuntu-17.04-desktop-amd64.iso - live" {
 set root=(hd0,4)
 linux ($root)/casper/vmlinuz.efi boot=casper quiet splash --
 initrd ($root)/casper/initrd.lz
}
menuentry "Lubuntu-17.04-desktop-amd64.iso - recovery mode" {
 set root=(hd0,4)
 linux ($root)/casper/vmlinuz.efi boot=casper ro recovery nomodeset
 initrd ($root)/casper/initrd.lz
}
submenu "Memory test" {
 menuentry "Memtest86+ works in BIOS alias CSM mode, but not in efi (UEFI) mode"{
  echo "${grub_platform} mode; (memtest86+ does not work with efi (UEFI) mode)"
 }
 menuentry "Memtest86+ 4.20" {
  set root=(hd0,3)
  echo "${grub_platform} mode; (memtest86+ does not work with efi (UEFI) mode)"
  linux16 /boot/memtest/memtest.bin
 }
 menuentry "Memtest86+-5.01" {
  set root=(hd0,3)
  echo "${grub_platform} mode; (memtest86+ does not work with efi (UEFI) mode)"
  linux16 /boot/memtest/memtest86+-5.01.bin
 }
}

Alternative

An alternative is to copy the menuentries of grub.cfg and modify one of them to have two alternatives to make the USB flash drive boot easily in different computers.

...
menuentry "Ubuntu-17.04-desktop-amd64.iso - persistent live (hd2,4)" {
 set root=(hd2,4)
 linux ($root)/casper/vmlinuz.efi boot=casper quiet splash persistent --
 initrd ($root)/casper/initrd.lz
}
menuentry "Ubuntu-17.04-desktop-amd64.iso - persistent live (hd0,4)" {
 set root=(hd0,4)
 linux ($root)/casper/vmlinuz.efi boot=casper quiet splash persistent --
 initrd ($root)/casper/initrd.lz
}
...

Solution 2

Assuming you are booting with grub (UEFI mode) and not syslinux (legacy mode), you could just edit the grub commands from the grub menu screen by typing "e", then using the arrow keys move to the hd0 and change it to hd2. Instructions at bottom of grub screen, control X or F10 to then boot.

If the changes work, try to edit them into the grub.cfg file. Sudodus is right, update-grub will not work in this case.

I'll leave the mkusb discussion to sudodus, I was getting confused by looking at a very old USB stick created by an older version.

Comments

[sudodus]

I am willing to help you, and I need your help in order to identify what is happening. When you are booting the computer (from some other drive, internal or external and with the 'USB drive to be persistent' connected) please run the following commands and edit your original question to post the output (indent 4 spaces to render it as code), sudo lsblk -fm , sudo parted -ls , df

[sudodus]

Also, please tell me which version of mkusb you are using: what is the output of mkusb -v Furthermore, some HP computers want an MSDOS partition table (not the default GPT) in order to boot nicely via grub. There is an option in the 'settings' menu.

[sudodus]

Normally the computer will identify the boot drive as hd0, but if this is not the case, I agree that @ubfan1's manual method should work (to edit the grub.cfg file, the file in the 'USB drive to be persistent' ). But I think the system is not complete enough (in a persistent live drive made with mkusb) for sudo update-grub

[sudodus]

We can only guess until there is feedback from the original poster, but I would suggest editing the grub.cfg file in partition #3 of the 'USB drive to be persistent' (the 'usbboot' partition).

[Warren Severin]

at entry to the grub menu, the root hd is set correctly. eg.: grub> echo $root hd2,gpt3 grub> All that is needed in resetting $root is to change the partition, not the drive. Another possibility is to use the grub drivemap module, eg: grub> drivemap -s ($root) (hd0) However, this at minimum seems to require disabling secure boot. After that, insmod is in a conundrum, finding the correct drive to load the drivemap module from.

[sudodus]

Please explain when and how the root hd is set correctly.

[sudodus]

I have been testing a new way to find partition #4, and I will edit my answer to describe how it works and how to get it. Please test if that will solve your problem.

[Warren Severin]

re: "Please explain when and how the root hd is set correctly." At first appearance of the grub menu, if I enter "c" to get a command line and get a grub> prompt, I enter echo $root I get response hd2,gpt3

[sudodus]

If the method of version 12.1.1 does not work, I will try along what you describe. The problem might be that you do not really boot directly from the USB flash drive, but from the internal drive's EFI partition, and I still hope that the method to search for the UUID should find the correct partition (without having to specify the hd2 explicitly. I have edited the answer now. Good luck :-)

[Warren Severin]

Tried it. It seems to work! I installed the unstable release, created a new persistent drive, booted from it (from BIOS menu) and it came up just as expected. Nice work! Thanks!

[sudodus]

I'm glad it works for you. Thanks for taking your time to report this bug and to help me test it :-)

[Meninx - メネンックス]

I just tried mkusb 12.1.1, (with UUID's), I think this is much better than using (hd0,4).