How to view all boot messages in Linux after booting?

Question

Relevant questions are:

Where Linux places the messages of boot?

Name of log file where boot process is logged

However, these do not answer this question. This question is concerned with how all the boot messages can be viewed.

This is for Gentoo, OpenRC, modern kernels, 4.9.6 if you want to get specific. A generic solution that works for all distributions would however be preferable.

The problem is that sometimes an error or warning will scroll by so quickly that it cannot be seen. Nor is it always possible to simply scroll up for two reasons (even with --noclear in inittab): when switching to the framebuffer scrolling up to the point before the switch too place is no longer possible, and second, that after X starts, switching to console and trying to scroll up doesn't allow scrolling at all until new text is added to the buffer. Sometimes, certain messages are simply not found in dmesg nor /var/log/messages.

How can I view all the messages?

I see someone here https://www.linuxquestions.org/questions/linux-newbie-8/please-how-to-pause-scrolling-messages-at-boot-323772/ suggesting that pressing scroll lock might pause it. However, this is a not very elegant solution at best -- some messages will scroll by too quickly, systems can suddenly churn out a lot of text these days when booting.

This is what I ideally want:

• A dmesg | less type solution, if possible, or some other way of single stepping through the boot process.
• A way to ensure that everything printed on the screen also gets logged.

Is there a straightforward way to achieve either of these?

I know of one solution:

CONFIG_BOOT_PRINTK_DELAY: Delay each boot printk message by N milliseconds

Strangely I don't seem allowed to even select BOOT_PRINTK_DELAY in my menuconfig, I can find it when searching for it, but under Kernel hacking -> printk and dmesg options ->, I only have "Show timing information on printks" and "Default message log level". Where is the printk delay option? Do I need to enable something else first to make it visible? What? It would be nice to have this as part of an answer, if anyone knows.

But anyway this requires a kernel recompilation which makes this an ugly and invasive hack for a seemingly trivial task. A proper way of doing this would be very much welcome.

Solution 1

So your console has two types of messages:

• generated by the kernel (via printk);
• generated by userspace (usually your init system).

Kernel messages are always stored in the kmsg buffer, visible via dmesg. They're also often copied to your syslog. (This also applies to userspace messages written to /dev/kmsg, but those are fairly rare.)

Meanwhile, when userspace writes its fancy boot status text to /dev/console or /dev/tty1, it's not stored anywhere at all. It just goes to the screen and that's it. So I think pretty much any solution – except for Rowan's serial console suggestion – will end up being either very distro-specific (due to each init system performing logging differently) or an "invasive hack" that involves strace or kernel hacking or such.

In the best case, your init system will itself log all important events to the syslog (/var/log/messages or such). For example:

systemd[1]: Starting BIRD routing daemon...
bird[478296]: /etc/bird.conf, line 2: syntax error
systemd[1]: bird.service: Control process exited, code=exited status=1
systemd[1]: Failed to start BIRD routing daemon.

(systemd and upstart also log the services' stdout/stderr as well; many other init systems just redirect it to the console or nowhere.)

Solution 2

One suggestion is to have another laptop film the boot screen with high resolution and frame rates then slow play the resulting (MOV - MP4 - AVI) - maybe not the best solution but simply to deploy and its for debugging anyway right ? Just an Idea ...

Solution 3

Grawity's answer is must better described than I would have been able to accomplish for that stage in the boot.

On the physical redirection method, your BIOS has to support it. Server geared boards usually do. The Intel, IBM, and SuperMicro systems typically have a Console Redirection option under the main heading of the BIOS. It cannot be used if your motherboard does not have a physical serial port, e.g. only USB. You may need to attach a physical port to pins on the motherboard if it has it.

Console Redirection can cause wackiness when you get to the point where the operating system is trying to use the serial port for something else. I would say it's primarily used for debugging in a particular case rather than constant use.

To use, you would need to set matching port settings on your receiver com port which could be usb based, then you need to link the two devices with a null modem cable which exchanges the transmit and receive pins between the devices.

The receiving device needs to be fully powered on and watching for the communication during the boot of the source system. The receiving system will get all text based output.

If during boot your OS starts displaying text via a grapical construct rather than as text, the receiving system will get gibberish.

Comments

[quixotic]

what distribution? sysvinit or systemd (or other)? kernel version?

[AttributedTensorField]

Edited the question.

[Rowan Hawkins]

Well dmesg is the location of that file. If it doesn't exist on boot it will be created so if you want make your startup include a line in the init.d which moves it to another file. I'm sure you could reuse the command set that does the standard log rotation for that file then you could keep the last N iterations. The only messages you won't see would cause grub to fail to boot and if that's the case you won't have a huge pile of text scrolling onto the screen.

[Rowan Hawkins]

If instead you were trying to capture the text prior to the BIOS passing control to the hard drive boot sector you would have to have some sort of screen capture utility over a Serial management port or ipmi. Both of which require server-class hardware.

[Rowan Hawkins]

I'm adding this is a comment because it doesn't directly apply to the answer. Serial 9 pin ports are not technically part of the serial spec. The original serial spec only included the 25 pin port and an almost never used 15 pin port. The 9 pin that we see everywhere today is a hack of the 15 pin connector into a smaller form factor.

[user1686]

I need to add a mention of Linux "netconsole" but I haven't researched it yet.

[WinEunuuchs2Unix]

As grub boots before the kernel perhaps it can leave a listener on /dev/tty1? Some sort of low level screen recorder.

[Community]

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.