grub-install error. failed to register the EFI boot entry: Operation not permitted. mount: /var/lib/grub/esp: mount point does not exist

Question

grub-install fails to reinstall grub with:

Installing for x86_64-efi platform.
grub-install: warning: Internal error.
grub-install: error: failed to register the EFI boot entry: Operation not permitted.

I noticed while performing an upgrade which tries to upgrade the package grub-efi-amd64-signed. This gives a more informative error message:

Installing grub to /boot/efi.
Installing for x86_64-efi platform.
grub-install: warning: Internal error.
grub-install: error: failed to register the EFI boot entry: Operation not permitted.
mount: /var/lib/grub/esp: mount point does not exist.

The system is trying to find the EFI partition on /var/lib/grub/esp

This is strange since my fstab looks correct,

/dev/sda7 has the boot flag and /dev/sda2 has the esp flag. I don't know why grub-install goes to /var/lib to try to find the efi partition. The error still persists even if I explicitly pass the directories to grub-install.

grub-install --efi-directory=/boot/efi/ --boot-directory=/boot/

Yes, I tried running boot repair. Got the following out: https://paste.ubuntu.com/p/THG9p4YQgQ/

Solution

After turning off Fast Boot UEFI settings, the grub-install problem vanished. I also turned off Fast Boot from withing Windows itself as advised by @oldfred. grub-install works. However, when I upgrade the grub-efi-amd64-signed package, I still get the same /var/lib/grub/esp error:

Setting up grub-efi-amd64-signed (1.142.9+2.04-1ubuntu26.7) ...
Installing grub to /boot/efi.
Installing for x86_64-efi platform.
Installation finished. No error reported.
mount: /var/lib/grub/esp: mount point does not exist.
dpkg: error processing package grub-efi-amd64-signed (--configure):
 installed grub-efi-amd64-signed package post-installation script subprocess returned error exit status 32
Errors were encountered while processing:
 grub-efi-amd64-signed

After that, purging grub and reinstalling grub by running the following commands solved the grub-efi-amd64-signed installation issue.

sudo apt-get purge grub\*
sudo apt-get install grub-efi
sudo apt-get autoremove
sudo update-grub

Comments

[oldfred]

Normally both boot & esp flags are on ESP with UEFI. Grub does not use boot flag, Windows does. Does fstab have correct mount of ESP - efi system partition? You can use Boot-Repair to do a total reinstall of grub. Use Advanced mode and be sure to boot in UEFI boot mode. Lets see details, use ppa version with your live installer (2nd option) or any working install, not Boot-Repair ISO: Please copy & paste the pastebin link to the Boot-info summary report ( do not post report), do not run the auto fix till reviewed. help.ubuntu.com/community/Boot-Repair

[pip1726]

Yes, actually both flags are on for /dev/sda2 (/boot/efi) and /dev/sda7 (/boot) partitions. I've tried switching them off for either one of them, but I haven't found seen any changes with regard to the error.

[oldfred]

You can only have one boot flag and one ESP per device. So put them on ESP. Turn off Windows fast start up. Check if UEFI has any settings locking/protecting ESP, some do. askubuntu.com/questions/843153/… & askubuntu.com/questions/145902/… Most desktops do not need separate /boot partition, more for servers. Just another partition to manage size.

[pip1726]

@oldfred Thanks for taking the time to look at this. I still don't understand what is the real issue, how to resolve it. From the links you are sharing, it looks like a Windows problem?

[oldfred]

Most likely a Windows hibernation/fast startup or UEFI setting issue.

[pip1726]

Thank you @oldfred!