Do background processes get a SIGHUP when logging off?

Question

This is a followup to this question.

I've run some more tests; looks like it really doesn't matter if this is done at the physical console or via SSH, neither does this happen only with SCP; I also tested it with cat /dev/zero > /dev/null. The behaviour is exactly the same:

• Start a process in the background using & (or put it in background after it's started using CTRL-Z and bg); this is done without using nohup.
• Log off.
• Log on again.
• The process is still there, running happily, and is now a direct child of init.

I can confirm both SCP and CAT quits immediately if sent a SIGHUP; I tested this using kill -HUP.

So, it really looks like SIGHUP is not sent upon logoff, at least to background processes (can't test with a foreground one for obvious reasons).

This happened to me initially with the service console of VMware ESX 3.5 (which is based on RedHat), but I was able to replicate it exactly on CentOS 5.4.

The question is, again: shouldn't a SIGHUP be sent to processes, even if they're running in background, upon logging off? Why is this not happening?

---

Edit

I checked with strace, as per Kyle's answer.
As I was expecting, the process doesn't get any signal when logging off from the shell where it was launched. This happens both when using the server's console and via SSH.

Solution 1

Answer found.

For BASH, this depends on the huponexit shell option, which can be viewed and/or set using the built-in shopt command.

Looks like this options is off by default, at least on RedHat-based systems.

More info on the BASH man page:

The shell exits by default upon receipt of a SIGHUP. Before exiting, an interactive shell resends the SIGHUP to all jobs, running or stopped. Stopped jobs are sent SIGCONT to ensure that they receive the SIGHUP. To prevent the shell from sending the signal to a particular job, it should be removed from the jobs table with the disown builtin (see SHELL BUILTIN COMMANDS below) or marked to not receive SIGHUP using disown -h.

If the huponexit shell option has been set with shopt, bash sends a SIGHUP to all jobs when an interactive login shell exits.

Solution 2

It will be sent SIGHUP in my tests:

Shell1:

[kbrandt@kbrandt-opadmin: ~] ssh localhost
[kbrandt@kbrandt-opadmin: ~] perl -e sleep & 
[1] 1121
[kbrandt@kbrandt-opadmin: ~] ps
  PID TTY          TIME CMD
 1034 pts/46   00:00:00 zsh
 1121 pts/46   00:00:00 perl
 1123 pts/46   00:00:00 ps

Shell2:

strace -e trace=signal -p1121

Shell1 Again:

[kbrandt@kbrandt-opadmin: ~] exit
zsh: you have running jobs.
[kbrandt@kbrandt-opadmin: ~] exit
zsh: warning: 1 jobs SIGHUPed
Connection to localhost closed.

Shell2 Again:

strace -e trace=signal -p1121
Process 1121 attached - interrupt to quit
pause()                                 = ? ERESTARTNOHAND (To be restarted)
--- SIGHUP (Hangup) @ 0 (0) ---
Process 1121 detached

Why does it still run?:
Advanced Programing in the Unix Environment by Stevens covers this under section 9.10: Orphaned Process Groups. The most relevant section being:

Since the process group is orphaned when the parent terminates, POSIX.1 requires that every process in the newly orphaned process group that is stopped (as our child is) be sent the hang-up signal (SIGHUP) followed by the continue signal (SIGCONT).

This causes the child to be continued, after processing the hang-up signal. The default action for the hang-up signal is to terminate the process, so we have to provide a signal handler to catch the signal. We therefore expect the printf in the sig_hup function to appear before the printf in the pr_ids function.

Solution 3

I ran some tests using CentOS 7.1 and bash. Note this means huponexit is off by default, and was off for the majority of my tests.

You need nohup when you start a job in a terminal, because if you close that terminal without exiting the shell cleanly, the terminal sends bash the SIGHUP signal to the shell, which then sends it to all children. If you exit the shell cleanly- meaning the job must already be in the background so you can type exit or hit Control-D at the command prompt- no signals of any sort are sent to the background job from bash.

Test:

Terminal 1

$ echo $$
16779

Terminal 2

$ strace -e signal -p16779
Process 16779 attached

(close terminal 1, seen in terminal 2):

--- SIGHUP {si_signo=SIGHUP, si_code=SI_USER, si_pid=16777, si_uid=3000090} ---
rt_sigprocmask(SIG_BLOCK, [CHLD TSTP TTIN TTOU], [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_SETMASK, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_SETMASK, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], NULL, 8) = 0
rt_sigaction(SIGHUP, {SIG_DFL, [], SA_RESTORER, 0x7f7ace3d9a00}, {0x456880, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], SA_RESTORER, 0x7f7ace3d9a00}, 8) = 0
kill(16779, SIGHUP)                     = 0
rt_sigreturn()                          = -1 EINTR (Interrupted system call)
--- SIGHUP {si_signo=SIGHUP, si_code=SI_USER, si_pid=16779, si_uid=3000090} ---
+++ killed by SIGHUP +++

Job doit.sh:

#!/bin/bash

imhupped() {
        echo "HUP" >> /tmp/outfile
}

trap imhupped SIGHUP

for i in $(seq 1 6); do echo out $i >> /tmp/outfile; sleep 5; done

Start it in the background in Terminal 1:

Terminal 1

$ ./doit.sh &
[1] 22954

Strace it in Terminal 2; close Terminal 1 after a couple of loops:

Terminal 2

$ strace -e signal -p22954
Process 22954 attached
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=22980, si_status=0, si_utime=0, si_stime=0} ---
rt_sigreturn()                          = 0
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f7a5d547a00}, {0x43e4b0, [], SA_RESTORER, 0x7f7a5d547a00}, 8) = 0
...
--- SIGHUP {si_signo=SIGHUP, si_code=SI_USER, si_pid=21685, si_uid=3000090} ---
rt_sigreturn()                          = -1 EINTR (Interrupted system call)
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=23017, si_status=SIGHUP, si_utime=0, si_stime=0} ---
rt_sigreturn()                          = 0
...

Output in Terminal 3:

Terminal 3

out 1
out 2
out 3
HUP
out 4
out 5
out 6

However, if you exit bash, it simply exits without sending any signal to the child at all. The terminal will exit because it no longer has a child, but of course there is no one to HUP because the child shell is already gone. The SIGINT, SIG_BLOCK and SIG_SETMASK you see below are due to the sleep in the shell.

Terminal 1

$ ./doit.sh &
26275

Terminal 2

$ strace -e signal -p26275
Process 26275 attached
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=26280, si_status=0, si_utime=0, si_stime=0} ---
rt_sigreturn()                          = 0
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f5edd3a5a00}, {0x43e4b0, [], SA_RESTORER, 0x7f5edd3a5a00}, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x43e4b0, [], SA_RESTORER, 0x7f5edd3a5a00}, {SIG_DFL, [], SA_RESTORER, 0x7f5edd3a5a00}, 8) = 0


(..."exit" is typed in bash, notice no new signals sent...)


rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=26303, si_status=0, si_utime=0, si_stime=0} ---
rt_sigreturn()                          = 0
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f5edd3a5a00}, {0x43e4b0, [], SA_RESTORER, 0x7f5edd3a5a00}, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x43e4b0, [], SA_RESTORER, 0x7f5edd3a5a00}, {SIG_DFL, [], SA_RESTORER, 0x7f5edd3a5a00}, 8) = 0

Terminal 3, output

out 1
out 2
out 3
out 4
out 5
out 6

Interestingly, I set huponexit to be on with shopt -s huponexit; shopt (the latter shopt to review), then performed the last test, and again bash sent no signal to the background process. Even MORE interstingly, as we have seen bash did send the signal to the background process after it received it from a terminal that closed in its face. It seems as though huponexit had no bearing one way or the other.

I hope this removes any mystery or confusion regarding at least bash's huppiness, about when and how the HUP signal is sent. At least my tests were completely reproducible, for me. I would be interested to know if there are any other settings that may be affecting bash's behavior.

And, as always, YSMV (Your Shell May Vary).

Addendum 1

When I run a shell as exec /bin/sh, then run the script as /bin/sh ./doit.sh &, then exit the shell cleanly, no signals are sent to the background job and it continues to run to completion.

Addendum 2

When I run a shell as exec /bin/csh, then run the script as /bin/sh ./doit.sh &, then exit the shell cleanly, no signals are sent to the background job and it continues to run to completion.

Comments

[Mike S]

Using Bash on CentOS 7.1, a simple shell script loop will get a SIGHUP if it's left in foreground, but the terminal killed; strace from another terminal shows: --- SIGHUP {si_signo=SIGHUP, si_code=SI_USER, si_pid=10676, si_uid=3000090} --- rt_sigreturn() = -1 EINTR (Interrupted system call) rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0

[Mike S]

So will a background script. Note that the terminal is closed while the loop is waiting on a sleep. The shell is NOT exited: --- SIGHUP {si_signo=SIGHUP, si_code=SI_USER, si_pid=13944, si_uid=3000090} --- +++ killed by SIGHUP +++

[Mike S]

See my answer for tests. Interestingly, I saw no change in behavior due to huponexit.

[Massimo]

WTF I just realized I asked this question 11 YEARS AGO...

[Mike S]

Welcome back, old timer!

[Massimo]

But you explicitly sent a SIGHUP to it here; I was talking about what happens when you log off from the shell where you started the process.

[Kyle Brandt]

Same results when I type exit, although I get a warning about jobs, but then type exit again. I tested this with ZSH.

[Massimo]

I'm using BASH, and this probably depends on the shell. But BASH should send SIGHUP to child processes when logging off...

[Kyle Brandt]

Bash sends SIGCONT apparently if the job is stopped, but I confirm it doesn't send anything if the job was not stopped.

[CarpeNoctem]

Verified. When I performed an "exit", "logout", or CTL-D the child proc (job) would not receive a sighup (both root and reg user). However when I did "kill -HUP $$" to kill the current instance of bash the child processes DID receive a sighup. I then set huponexit and the child process did receive SIGHUP upon exit.

[Mike S]

Using Bash on CentOS 7.1, I get a SIGTERM sent to my process stopped in another window: 1.) Start simple shell script (loop with an echo and a sleep), 2.) Control-Z it, 3) strace the process in another window, 4) exit the original terminal. It complains that I have running jobs, then after exiting my strace shows: $ strace -e signal -p1705 Process 1705 attached --- stopped by SIGTSTP --- --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=791, si_uid=3000090} --- +++ killed by SIGTERM +++ Odd, definitely not in accord with the section quoted from Stevens.