Time it takes to reboot a Linux server

Question

Is there a way to work out how long it would take to reboot a Linux server? To clarify, the time from the reboot command starts to when the server is back up and running (ie all services started, users can log on etc).

I tried looking at syslog but that seems to get rotated too quickly.

To the nearest minute would suffice.

OS = CentOS & Ubuntu

Update: if there isn't a simple way - perhaps what would be a way to capture this data for future use.

Solution

I'm going to assume you're on CentOS 7+ or Ubuntu 15.04+ which both come with systemd. Systemd has some great tools for figuring out how long your system took to boot along with some visualizations to see why.

For the most basic output just run systemd-analyze and you'll get a nice summary like so

Startup finished in 853ms (kernel) + 3min 50.610s (initrd) + 10.345s (userspace) = 4min 1.809s

That can tell you how much time your last boot took once systemd was started. That doesn't take into account BIOS/hardware initialization or GRUB timeouts but should be accurate for actual OS boot time.

If you want to figure out why the OS is taking so long try systemd-analyze blame which will give you a chart of services from longest running to shortest. eg from my system

3min 49.219s systemd-cryptsetup@luks\x2d62611c1c\x2d74ab\x2d4be9\x2d8990\x2d41c0fd863b5a.service
      5.315s plymouth-quit-wait.service
      3.084s systemd-udev-settle.service
      2.275s plymouth-start.service
      2.256s docker.service
      1.819s powertop.service
       778ms firewalld.service
       676ms dev-mapper-fedora\x2droot.device
       621ms abrtd.service
       493ms lvm2-monitor.service

Looks like 3 of the 4 minutes it takes to my boot my laptop is because I have an encrypted drive.

Finally You can see a chain of events that are considered "critical" to get the system booted. From the man page critical means "time-critical chain of units". This is because systemd parallelizes a lot of services. This will list the units that have to wait for another unit and how long it took to start.

The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.

graphical.target @10.336s
└─multi-user.target @10.323s
  └─docker.service @4.900s +2.256s
    └─network.target @4.868s
      └─wpa_supplicant.service @4.828s +14ms
        └─dbus.service @3.753s
          └─basic.target @3.749s
            └─sockets.target @3.749s
              └─docker.socket @3.741s +6ms
                └─sysinit.target @3.737s
                  └─systemd-update-utmp.service @3.726s +10ms
                    └─auditd.service @3.713s +9ms
                      └─systemd-tmpfiles-setup.service @3.617s +82ms
                        └─fedora-import-state.service @3.568s +36ms
                          └─local-fs.target @3.560s
                            └─run-user-42.mount @5.753s
                              └─local-fs-pre.target @383ms
                                └─systemd-tmpfiles-setup-dev.service @301ms +80ms
                                  └─kmod-static-nodes.service @268ms +10ms
                                    └─system.slice
                                      └─-.slice

You can also do some cool stuff by exporting your boot tree to a picture to email it or plot it with svg. See the man page for more details or this related question for a few more details.

Comments

[Admin]

what logs are you looking at, and how many days/weeks after a reboot? Does your CentOS/Ubuntu versions save /var/log/dmesg* files?

[Admin]

from the dmesg file, you can at least see how long the actual boot of linux took, but it is not possible to see how long it took from shutdown until the bootloader was complete.

[Admin]

IMHO, there is no reliable way to measure the time for reboot to complete. there are many factors which add to the unpredictability, for example, there could be a filesystem check (fsck) forced during the reboot where the delay is unpredictable due to number and size of disks/ partitions involved. if we ignore such abnormal events, the other factors which need to be accounted for - whats the default runlevel? are your users logging in via ssh/ remote X11/ vnc etc ?

[Admin]

@nagu - most of those things stay the same or are manually changed (ie you can factor in the additional time). You can tell when a fsck will be forced and again factor in additional time. Generally a server will take the same amount of time to reboot each time - I was looking for a guide not an exact measure.

[Admin]

+1 - thanks! I'm using CentOS 7 but ... Ubuntu 14.04 ie upstart.

[telcoM]

@Prvt_Yadav Not with only \x but with \x2d - and the - character is encoded in ASCII as 45, or 2d in hexadecimal. The name of a systemd mount unit should reflect the mount point of the filesystem with slashes in the pathname converted to minus signs - and if the pathname already contains minus signs, those existing minus signs need to be converted to something else so that the pathname of the mount point can be recovered without any ambiguity.