systemd - Booting into Emergency Mode with Error - Dependency Failed for /home

arch-linux systemd fstab

14,311

Add the -netdev option to the fstab entry for '/home'. It appears the the dependency for mounting is the network management stack.

14,311

dainnen

Updated on September 18, 2022

Comments

dainnen over 1 year

Recently, I updated my Arch Linux install. I try to do that every two weeks. Once I did, I rebooted and received an error that a dependency failed for /home (my home partition.) The boot process immediately went into emergency mode. I found that my home partition was not being mounted at all. The weird thing is, I can go into emergency mode, then exit. After that, the home partition mounts and the system loads just fine.

I should mention that my home partition is btrfs while my root partition is ext4. I do have a HOOK in my /etc/mkinitcpio.conf file that scans for btrfs on boot. It is working, as I can see it outputting that it is scanning for btrfs.

It does tell me to check the output of journalctl -xb but, couldn't see why my home partition is not mounting.

I've read similar posts that say to check the UUID of the home partition in my /etc/fstab file. Everything in my fstab file looked good, but I decided to put in an arch installation USB and regenerate the file. That did not fix the issue.

Here's a portion of my journalctl -xb output:

-- Unit systemd-fsck@dev-disk-by\x2duuid-8244\x2d4C7C.service has begun starting up.
Jul 20 01:46:35 aurora systemd[1]: home.mount: Bound to unit dev-disk-by\x2duuid-9f171b93\x2ddd7d\x2d4353\x2d84f8\x2d79c6f673f47f.device, but unit isn't active.
Jul 20 01:46:35 aurora systemd[1]: Dependency failed for /home.
-- Subject: Unit home.mount has failed
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit home.mount has failed.
--
-- The result is RESULT.
Jul 20 01:46:35 aurora systemd[1]: Dependency failed for Local File Systems.
-- Subject: Unit local-fs.target has failed
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit local-fs.target has failed.
--
-- The result is RESULT.
Jul 20 01:46:35 aurora systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'.
Jul 20 01:46:35 aurora systemd[1]: local-fs.target: Triggering OnFailure= dependencies.
Jul 20 01:46:35 aurora systemd[1]: home.mount: Job home.mount/start failed with result 'dependency'.
Jul 20 01:46:35 aurora systemd[1]: Reached target Network.
-- Subject: Unit network.target has finished start-up
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit network.target has finished starting up.
--
-- The start-up result is RESULT.
Jul 20 01:46:35 aurora systemd[1]: Reached target Sockets.
-- Subject: Unit sockets.target has finished start-up
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit sockets.target has finished starting up.
--
-- The start-up result is RESULT.
Jul 20 01:46:35 aurora systemd[1]: Starting Create Volatile Files and Directories...
-- Subject: Unit systemd-tmpfiles-setup.service has begun start-up
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit systemd-tmpfiles-setup.service has begun starting up.
Jul 20 01:46:35 aurora systemd[1]: Started Emergency Shell.
-- Subject: Unit emergency.service has finished start-up
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit emergency.service has finished starting up.
--
-- The start-up result is RESULT.
Jul 20 01:46:35 aurora systemd[1]: Reached target Emergency Mode.
-- Subject: Unit emergency.target has finished start-up
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit emergency.target has finished starting up.
--
-- The start-up result is RESULT.
Jul 20 01:46:35 aurora systemd[1]: Reached target Timers.
-- Subject: Unit timers.target has finished start-up
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit timers.target has finished starting up.
--
-- The start-up result is RESULT.
Jul 20 01:46:35 aurora systemd[1]: Mounting /home...
-- Subject: Unit home.mount has begun start-up
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit home.mount has begun starting up.
Jul 20 01:46:35 aurora kernel: BTRFS info (device sdb3): disk space caching is enabled
Jul 20 01:46:35 aurora kernel: BTRFS info (device sdb3): has skinny extents
Jul 20 01:46:35 aurora systemd-fsck[305]: fsck.fat 4.1 (2017-01-24)
Jul 20 01:46:35 aurora systemd-fsck[305]: /dev/sda6: 349 files, 3156/127746 clusters
Jul 20 01:46:35 aurora systemd[1]: Started File System Check on /dev/disk/by-uuid/8244-4C7C.
-- Subject: Unit systemd-fsck@dev-disk-by\x2duuid-8244\x2d4C7C.service has finished start-up
-- Defined-By: systemd
-- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit systemd-fsck@dev-disk-by\x2duuid-8244\x2d4C7C.service has finished starting up.

Here's my /etc/fstab file (I've intentionally erased my other UUIDs for security reasons):

# /dev/sda5
UUID=XXXX-XXXX-XX       /               ext4            rw,relatime,data=ordered        0 1

# /dev/sda6
UUID=XXXX-XXXX-XX          /boot/efi       vfat            rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro       0 2

# /dev/sdb3
UUID=9f171b93-dd7d-4353-84f8-79c6f673f47f       /home           btrfs           rw,relatime,space_cache,subvolid=5,subvol=/     0 0

# /dev/sda7
UUID=XXXX-XXXX-XX       none            swap            defaults,pri=-2 0 0

Here's the output of lsblk -f (again, I've intentionally erased my other UUIDs for security reasons)

sda
├─sda1 vfat         XXXX-XXXX-XX
├─sda2
├─sda3 ntfs         XXXX-XXXX-XX
├─sda4 ntfs         XXXX-XXXX-XX
├─sda5 ext4         XXXX-XXXX-XX    /
├─sda6 vfat         XXXX-XXXX-XX    /boot/efi
└─sda7 swap         XXXX-XXXX-XX    [SWAP]
sdb
├─sdb1
├─sdb2 ntfs   Games XXXX-XXXX-XX
└─sdb3 btrfs        9f171b93-dd7d-4353-84f8-79c6f673f47f /home

As suggested below, I checked dmesg here's a portion of that:

[    2.055164] Btrfs loaded, crc32c=crc32c-intel
[    2.055509] BTRFS: device fsid 9f171b93-dd7d-4353-84f8-79c6f673f47f devid 1 transid 83839 /de                                                                                                                                             v/sdb3
[    2.086828] PM: Image not found (code -22)
[    2.340606] EXT4-fs (sda5): mounted filesystem with ordered data mode. Opts: (null)
[    2.457135] systemd[1]: systemd 239 running in system mode. (+PAM -AUDIT -SELINUX -IMA -APPAR                                                                                                                                             MOR +SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTIL                                                                                                                                             S +KMOD -IDN2 +IDN +PCRE2 default-hierarchy=hybrid)
[    2.473611] systemd[1]: Detected architecture x86-64.
[    2.478035] systemd[1]: Set hostname to <aurora>.
[    2.557974] systemd[1]: local-fs-pre.target: Wants dependency dropin /etc/systemd/system/loca                                                                                                                                             l-fs-pre.target.wants/btrfs-dev-scan.service is not a symlink, ignoring.
[    2.567559] random: systemd: uninitialized urandom read (16 bytes read)
[    2.567629] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[    2.567665] random: systemd: uninitialized urandom read (16 bytes read)
[    2.569028] systemd[1]: Created slice system-systemd\x2dfsck.slice.
[    2.569041] random: systemd: uninitialized urandom read (16 bytes read)
[    2.569246] systemd[1]: Created slice system-getty.slice.
[    2.569352] systemd[1]: Listening on udev Control Socket.
[    2.569422] systemd[1]: Listening on udev Kernel Socket.
[    2.569518] systemd[1]: Listening on Journal Socket (/dev/log).
[    2.583473] EXT4-fs (sda5): re-mounted. Opts: data=ordered
[    2.643252] systemd-journald[236]: Received request to flush runtime journal from PID 1
[    2.729364] systemd-journald[236]: File /var/log/journal/cabc2f34b30c47f09c201422dfa880bd/sys                                                                                                                                             tem.journal corrupted or uncleanly shut down, renaming and replacing.
[    3.001682] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    3.027930] rtc_cmos 00:02: RTC can wake from S4
[    3.029174] rtc_cmos 00:02: registered as rtc0
[    3.029191] rtc_cmos 00:02: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
[    3.029493] i801_smbus 0000:00:1f.3: enabling device (0001 -> 0003)
[    3.029623] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt
[    3.066003] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    3.066005] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    3.066898] e1000e 0000:00:19.0: Interrupt Throttling Rate (ints/sec) set to dynamic conserva                                                                                                                                             tive mode
[    3.084156] mousedev: PS/2 mouse device common for all mice
[    3.086094] input: PC Speaker as /devices/platform/pcspkr/input/input7
[    3.092504] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 163840 ms ovfl timer
[    3.092506] RAPL PMU: hw unit of domain pp0-core 2^-16 Joules
[    3.092506] RAPL PMU: hw unit of domain package 2^-16 Joules
[    3.092507] RAPL PMU: hw unit of domain pp1-gpu 2^-16 Joules
[    3.094356] ipmi message handler version 39.2
[    3.097517] ipmi device interface
[    3.102120] Linux agpgart interface v0.103
[    3.145910] iTCO_vendor_support: vendor-support=0
[    3.150461] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11
[    3.150494] gpio_ich: GPIO from 436 to 511 on gpio_ich
[    3.150597] iTCO_wdt: Found a Cougar Point TCO device (Version=2, TCOBASE=0x0460)
[    3.153467] iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
[    3.189732] Adding 3288640k swap on /dev/sda7.  Priority:-2 extents:1 across:3288640k SSFS
[    3.232407] e1000e 0000:00:19.0 0000:00:19.0 (uninitialized): registered PHC clock
[    3.235465]  sdb: sdb1 sdb2 sdb3
[    3.301961] BTRFS info (device sdb3): disk space caching is enabled
[    3.301964] BTRFS info (device sdb3): has skinny extents

I guess the permanent solution would be to copy everything from the BTRFS partition, reformat it to EXT4, and put the files back on it. I don't necessarily want to do that because it would take a lot of time, but that's probably what needs to be done.

Any help would be appreciated, I feel like I've tried everything I know to do at this point.

Let me know if you need any other outputs or logs.

user2948306 almost 6 years

special! For some reason systemd thinks the block device got removed (or failed somehow?) at that point. When the device unit is removed, systemd automatically unmounts the filesystem (but in your boot process it hadn't even mounted it yet)... Obviously the device is there when you look, and when you retry local-fs.target the second time it doesn't have this problem. But I'm not sure what to suggest looking at :-(.
user2948306 almost 6 years

There are suggestions on how to enable systemd debugging at the end of freedesktop.org/wiki/Software/systemd/Debugging , the messages should show exactly when the device unit disappears or give you some such pointer about it. I guess you should first just look in dmesg and see if there's some really silly kernel log messages, showing the block device was added twice (and hence removed before the second add).
dainnen almost 6 years

Thank you for the suggestions! I don't see anything on dmesg that points to anything weird. It mentions mounting the btrfs partition, but that's it. I will edit my post and post the output. However, after reading the article on the link you posted, I'm going to turn on some more debugging and see if that leads to any other errors. I'm currently backing up my home partition because I have a feeling this is going to end with me reformatting it to ext4.

dainnen almost 6 years

That fixed it! Thank you! I added it to my fstab. For anyone else that's having this issue, my fstab entry looks like: # /dev/sdb3 UUID=9f171b93-dd7d-4353-84f8-79c6f673f47f /home btrfs rw,relatime,space_cache,subvolid=5,_netdev,subvol=/ 0 0