Why does bash show 'Terminated' after killing a process?

bash process kill

18,642

Solution 1

Short answer

In bash (and dash) the various "job status" messages are not displayed from signal handlers, but require an explicit check. This check is performed only before a new prompt is provided, probably not to disturb the user while he/she is typing a new command.

The message is not shown just before the prompt after the kill is displayed probably because the process is not dead yet - this is particularly probable condition since kill is an internal command of the shell, so it's very fast to execute and doesn't need forking.

Doing the same experiment with killall, instead, usually yields the "killed" message immediately, sign that the time/context switches/whatever required to execute an external command cause a delay long enough for the process to be killed before the control returns to the shell.

matteo@teokubuntu:~$ dash
$ sleep 60 &
$ ps
  PID TTY          TIME CMD
 4540 pts/3    00:00:00 bash
 4811 pts/3    00:00:00 sh
 4812 pts/3    00:00:00 sleep
 4813 pts/3    00:00:00 ps
$ kill -9 4812
$ 
[1] + Killed                     sleep 60
$ sleep 60 &
$ killall sleep
[1] + Terminated                 sleep 60
$

Long answer

`dash`

First of all, I had a look at the dash sources, since dash exhibits the same behavior and the code is surely simpler than bash.

As said above, the point seems to be that job status messages are not emitted from a signal handler (which can interrupt the "normal" shell control flow), but they are the consequence of an explicit check (a showjobs(out2, SHOW_CHANGED) call in dash) that is performed only before requesting new input from the user, in the REPL loop.

Thus, if the shell is blocked waiting for user input no such message is emitted.

Now, why doesn't the check performed just after the kill show that the process was actually terminated? As explained above, probably because it's too fast. kill is an internal command of the shell, so it's very fast to execute and doesn't need forking, thus, when immediately after the kill the check is performed, the process is still alive (or, at least, is still being killed).

`bash`

As expected, bash, being a much more complex shell, was trickier and required some gdb-fu.

The backtrace for when that message is emitted is something like

(gdb) bt
#0  pretty_print_job (job_index=job_index@entry=0, format=format@entry=0, stream=0x7ffff7bd01a0 <_IO_2_1_stderr_>) at jobs.c:1630
#1  0x000000000044030a in notify_of_job_status () at jobs.c:3561
#2  notify_of_job_status () at jobs.c:3461
#3  0x0000000000441e97 in notify_and_cleanup () at jobs.c:2664
#4  0x00000000004205e1 in shell_getc (remove_quoted_newline=1) at /Users/chet/src/bash/src/parse.y:2213
#5  shell_getc (remove_quoted_newline=1) at /Users/chet/src/bash/src/parse.y:2159
#6  0x0000000000423316 in read_token (command=<optimized out>) at /Users/chet/src/bash/src/parse.y:2908
#7  read_token (command=0) at /Users/chet/src/bash/src/parse.y:2859
#8  0x00000000004268e4 in yylex () at /Users/chet/src/bash/src/parse.y:2517
#9  yyparse () at y.tab.c:2014
#10 0x000000000041df6a in parse_command () at eval.c:228
#11 0x000000000041e036 in read_command () at eval.c:272
#12 0x000000000041e27f in reader_loop () at eval.c:137
#13 0x000000000041c6fd in main (argc=1, argv=0x7fffffffdf48, env=0x7fffffffdf58) at shell.c:749

The call that checks for dead jobs & co. is notify_of_job_status (it's more or less the equivalent of showjobs(..., SHOW_CHANGED) in dash); #0-#1 are related to its inner working; 6-8 is the yacc-generated parser code; 10-12 is the REPL loop.

The interesting place here is #4, i.e. from where the notify_and_cleanup call comes. It seems that bash, unlike dash, may check for terminated jobs at each character read from the command line, but here's what I found:

      /* If the shell is interatctive, but not currently printing a prompt
         (interactive_shell && interactive == 0), we don't want to print
         notifies or cleanup the jobs -- we want to defer it until we do
         print the next prompt. */
      if (interactive_shell == 0 || SHOULD_PROMPT())
    {
#if defined (JOB_CONTROL)
      /* This can cause a problem when reading a command as the result
     of a trap, when the trap is called from flush_child.  This call
     had better not cause jobs to disappear from the job table in
     that case, or we will have big trouble. */
      notify_and_cleanup ();
#else /* !JOB_CONTROL */
      cleanup_dead_jobs ();
#endif /* !JOB_CONTROL */
    }

So, in interactive mode it's intentional to delay the check until a new prompt is provided, probably not to disturb the user entering commands. As for why the check doesn't spot the dead process when displaying the new prompt immediately after the kill, the previous explanation holds (the process is not dead yet).

Solution 2

To avoid any job termination messages (on the command line as well as in ps output) you can put the command to be backgrounded into a sh -c 'cmd &' construct.

{
ps
echo
pid="$(sh -c 'sleep 60 1>&-  & echo ${!}')"
#pid="$(sh -c 'sleep 60 1>/dev/null  & echo ${!}')"
#pid="$(sh -c 'sleep 60 & echo ${!}' | head -1)"
ps
kill $pid
echo
ps
}

By the way, it is possible to get immediate job termination notifications in bash by using the shell options set -b or set -o notify respectively.

In this case "bash receives a SIGCHLD signal, and its signal handler displays the notification message immediately - even if bash is currently in the middle of waiting for a foreground process to complete" (see next reference below).

To get a third mode of job control notification inbetween set +b (the default mode) and set -b (so that you get immediate job termination notifications without corrupting what you have already typed on your current command line - similar to ctrl-x ctrl-v) requires a patch to bash by Simon Tatham (for the patch itself and further information please see: Sensible asynchronous job notification in bash(1)).

So just let's repeat Matteo Italia's gdb-fu for a bash shell that has been set to notify of job termination immediately with set -b.

# 2 Terminal.app windows

# terminal window 1
# start Bash compiled with -g flag
~/Downloads/bash-4.2/bash -il
set -bm
echo $$ > bash.pid

# terminal window 2
gdb -n -q
(gdb) set print pretty on
(gdb) set history save on
(gdb) set history filename ~/.gdb_history
(gdb) set step-mode off
(gdb) set verbose on
(gdb) set height 0
(gdb) set width 0
(gdb) set pagination off
(gdb) set follow-fork-mode child
(gdb) thread apply all bt full
(gdb) shell cat bash.pid
(gdb) attach <bash.pid>
(gdb) break pretty_print_job

# terminal window 1
# cut & paste
# (input will be invisible on the command line)
sleep 600 &   

# terminal window 2
(gdb) continue
(gdb) ctrl-c

# terminal window 1
# cut & paste
kill $!

# terminal window 2
(gdb) continue
(gdb) bt

Reading in symbols for input.c...done.
Reading in symbols for readline.c...done.
Reading in symbols for y.tab.c...done.
Reading in symbols for eval.c...done.
Reading in symbols for shell.c...done.
#0  pretty_print_job (job_index=0, format=0, stream=0x7fff70bb9250) at jobs.c:1630
#1  0x0000000100032ae3 in notify_of_job_status () at jobs.c:3561
#2  0x0000000100031e21 in waitchld (wpid=-1, block=0) at jobs.c:3202
#3  0x0000000100031a1a in sigchld_handler (sig=20) at jobs.c:3049
#4  <signal handler called>
#5  0x00007fff85a9f464 in read ()
#6  0x00000001000b39a9 in rl_getc (stream=0x7fff70bb9120) at input.c:471
#7  0x00000001000b3940 in rl_read_key () at input.c:448
#8  0x0000000100097c88 in readline_internal_char () at readline.c:517
#9  0x0000000100097dba in readline_internal_charloop () at readline.c:579
#10 0x0000000100097de6 in readline_internal () at readline.c:593
#11 0x0000000100097842 in readline (prompt=0x100205f80 "noname:~ <yourname>$ ") at readline.c:342
#12 0x0000000100007ab7 in yy_readline_get () at parse.y:1443
#13 0x0000000100007bbe in yy_readline_get () at parse.y:1474
#14 0x00000001000079d1 in yy_getc () at parse.y:1376
#15 0x000000010000888d in shell_getc (remove_quoted_newline=1) at parse.y:2231
#16 0x0000000100009a22 in read_token (command=0) at parse.y:2908
#17 0x00000001000090c1 in yylex () at parse.y:2517
#18 0x000000010000466a in yyparse () at y.tab.c:2014
#19 0x00000001000042fb in parse_command () at eval.c:228
#20 0x00000001000043ef in read_command () at eval.c:272
#21 0x0000000100004088 in reader_loop () at eval.c:137
#22 0x0000000100001e4d in main (argc=2, argv=0x7fff5fbff528, env=0x7fff5fbff540) at shell.c:749

(gdb) detach
(gdb) quit

18,642

syntagma

Updated on September 18, 2022

Comments

syntagma over 1 year

Here is the behaviour I want to understand:

$ ps
  PID TTY           TIME CMD
  392 ttys000    0:00.20 -bash
 4268 ttys000    0:00.00 xargs
$ kill 4268
$ ps
  PID TTY           TIME CMD
  392 ttys000    0:00.20 -bash
[1]+  Terminated: 15          xargs
$ ps
  PID TTY           TIME CMD
  392 ttys000    0:00.21 -bash

Why does it show the [1]+ Terminated: 15 xargs after I kill a process, instead of just not showing it as it was just killed?

I'm using bash on Mac OS X 10.7.5.

Aquarius Power almost 10 years

cool! but do you believe there could have some other way? I am trying this: pid="$(sh -c 'cat "$fileName" |less & echo ${!}')" but less wont show up