Run script on start up after everything else

Question

OK, so there's a service (nagios) running on a Debian box that runs just fine, except for one very specific issue that only occurs after the box has been re-started. The issue is easily fixed by re-starting the service by hand. The problem itself is so specific and so confoundingly esoteric that I don't have the time to run it down (a single check out of 500+ comes back with a bug, but only when it's being run by nagios).

Next best thing then, would be to have it re-start the service itself on startup, so no one has to do it by hand every time. So far I've tried to accomplish this by the following:

Adding in "/etc/init.d/nagios restart" to the /etc/rc.local, this does run from looking at the logs, but doesn't fix the issue (must still be done by hand)

Moving the timing for starting nagios to the very end (update-rc.d nagios defaults 99 10)

Went back to the rc.local fix, this time adding in a "sleep 20" line, this does nothing but delay starting the box by 20 seconds.

Anything else I can try/look at?

Solution 1

Check if the init script you want to delay have a comment block like this:

### BEGIN INIT INFO
# Provides:          scriptname
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start daemon at boot time
# Description:       Enable service provided by daemon.
### END INIT INFO

This block, as stated in Debian's LSBInitScripts wiki tells the init subsystem that some other facilities should be Required to be started before your nagios script runs.

Then, all you need to do is to point out why your nagios init script needs to be restarted, it's because of network not up yet? It's because webserver still starting? Or, nfs not yet in sync?

It's up to you to find out why you need to re-start nagios every time, but this approach is the most elegant solution to your case.

Solution 2

If the restart must be done by hand then your check might depend on some variables which are set when you log in.

Check if running by hand service nagios restart also fix your problem or it has to be /etc/init.d/nagios restart.

# Required-Start:    $all

would be the way to ensure that all other services are started before starting your script. But I think that rc.local is running after all services are started anyway so it will not help.

Comments

[terdon]

Have you tried adding the restart command to /etc/crontab? Something like @reboot /etc/init.d/nagios restart. Just a shot in the dark but it might work.

[terdon]

No idea, that should print to your log file as far as I can see, you're not using any relative paths or variables. I assume you checked after a reboot? Try writing to a log file that is not on /tmp in case crond runs before tmpfs is set up.

[Will Bates]

OK, just tried sending the output to a new directory /mytemp I created in the root directory (chmoded to 777), and just for a sanity check I also added in a @reboot echo "hello puny human" > /mytemp/rebootSpike 2>&1. Restart the box, and no files were created in the /mytemp...

[dentex]

Same problem that I have with transmission-daemon in Raspbian Jessie...

[user]

In Debian Wheezy, the rc.local initscript has the Required-Start: $all directive.