How to get the Job ID?

Question

As we know, the shell enables the user to run background processes using & at the command line's end. Each background process is identified by a job ID and, of course, by it's PID.

When I'm executing a new job, the output is something like [1] 1234 (the second number is the process ID). Trying to invoke commands like stop 1 and stop %1 causes a failure message: stop: Unknown job: 1

Understanding that the stop command causes a job to be suspended, I was wondering how to get the job ID and do it right. If the only way to kill a job is by it's process ID, what is the purpose of the job ID?

Solution 1

After a process is sent to the background with &, its PID can be retrieved from the variable $!. The job IDs can be displayed using the jobs command, the -l switch displays the PID as well.

 $ sleep 42 &
 [1] 5260
 $ echo $!
 5260
 $ jobs -l
 [1]  - 5260 running    sleep 42

Some kill implementations allow killing by job ID instead of PID. But a more sensible use of the job ID is to selectively foreground a particular process. If you start five processes in the background and want to foreground the third one, you can run the jobs command to see what processes you launched and then fg %3 to foreground the one with the job ID three.

Solution 2

Job control is a feature that was added to BSD systems in the early 80s. csh being the BSD shell, it comes as no surprise that the feature was introduced in that shell first. You had to wait several years before job control was added to non-BSD Unices and to other shells (starting with the Korn shell).

It should be noted that job control is not the ability to start a process asynchronously (that is with the shell not waiting for its termination before issuing the next prompt; the Bourne shell had & from the start, but didn't support job control until 10 years later), but about the ability to interact with several tasks from the same terminal.

We're talking of jobs, not processes. To implement job control, process groups were introduced. A job is a process group (job from the user point of view, process group from the OS implementation point of view).

It's typically interactive shells that create and manage process groups, and again that's all about interaction with the terminal.

When you run a command, all the processes and their descendants are placed in a new process group.

In a session on a given terminal, one process group is granted privileged access to the terminal. That's the foreground process group of the terminal. That process group is the one whose members receive a SIGINT/SIGTSTP/SIGQUIT when you press CTRL-{C,Z,\}. Processes in other process groups are suspended when they try to read from (and sometime write to) the terminal. etc.

When you start a job in foreground, the interactive shell makes its process group the foreground process group of the terminal, and not for the ones started in background.

That way, a user can do several things at a time in a terminal, he can have a list of jobs, which are groups of processes which can be in foreground (with privileged access to the terminal), background, running or suspended. And the job can be moved from/to background/foreground and suspended/running.

Nowadays, all systems and all shells support job control (though in the Bourne shell, you have to explicitly ask for it with set -m IIRC).

All commands started at the prompt are given a job number. From the system point of view those are mapped to process group ids.

 $ ps -fj | cat
UID        PID  PPID  PGID   SID  C STIME TTY          TIME CMD
chazelas  8237  7315  8237  8237  0 15:54 pts/7    00:00:00 /bin/zsh
chazelas 10720  8237 10720  8237  0 21:25 pts/7    00:00:00 ps -fj
chazelas 10721  8237 10720  8237  0 21:25 pts/7    00:00:00 cat

Above, I started that ps -j | cat job in foreground. It was assigned the 10720 process group (which also happens to be the process id of ps). If you'd queried the foreground process group id of the terminal at the time those where running (with tcgetpgrp()), you'd have gotten 10720.

$ (sleep 100;:) | sleep 101 &
[1] 10787 10789
$ ps -fj
UID        PID  PPID  PGID   SID  C STIME TTY          TIME CMD
chazelas  8237  7315  8237  8237  0 15:54 pts/7    00:00:00 /bin/zsh
chazelas 10787  8237 10787  8237  0 21:33 pts/7    00:00:00 /bin/zsh
chazelas 10788 10787 10787  8237  0 21:33 pts/7    00:00:00 sleep 100
chazelas 10789  8237 10787  8237  0 21:33 pts/7    00:00:00 sleep 101
chazelas 10790  8237 10790  8237  0 21:33 pts/7    00:00:00 ps -fj

That time, I started a pipeline in background. The shell gave me a job number (1) and 2 process ids ((t)csh, pdksh and zsh gives all the process ids, the behaviour varies in other shells), one for a subshell, one for sleep 101. Incidentally, that subshell spawned a 3rd process (sleep 100). The shell didn't report the pid of that one because it doesn't know anything about it, but it's still part of the same job (process group 10787 above).

Now, you've got a few commands to manipulate those jobs: fg, bg, kill, wait and jobs (some shells also have a disown and aliases to kill to send a different default signal).

Those take job ids in the format of %n where n is the job number. There's %foo to refer to the job whose command starts with foo, and more (see your shell's manual).

kill %1

sends the SIGTERM signal to all the processes of the job 1. kill -s STOP %1 sometime aliased to stop sends the SIGSTOP signal instead.

fg %1

puts it in foreground (makes it the foreground process group of the terminal, send the SIGCONT signal and tells the shell to wait for its completion).

bg %1

resumes (sends a SIGCONT) a stopped background job.

wait %1

does not put the job in foreground, but waits for its completion (or until it's suspended).

jobs

lists the current jobs (jobs -l with pgids and/or pids).

%% and %+ refer to the current job. It should be noted that the current job is not necessarily the most recently started job. It's the job marked with a + in jobs' output. It's also the job that fg or bg without argument deal with. It's the most recently suspended job if there are suspended ones, or the most recently started one if they are all running.

So, beware kill %% is not going to kill the job you just started in background if there are suspended jobs. It's always a good idea to run jobs to see what jobs are currently running before killing any.

It should be noted that you should not use with process ids when dealing with jobs.

$! returns the process id of that last command executed in background. In the case of a pipeline, that's the pid of the rightmost command.

$ sleep 100 | sleep 101 & ps -fj; echo "$!"
[1] 11044 11045
UID        PID  PPID  PGID   SID  C STIME TTY          TIME CMD
chazelas  8237  7315  8237  8237  0 15:54 pts/7    00:00:00 /bin/zsh
chazelas 11044  8237 11044  8237  0 22:08 pts/7    00:00:00 sleep 100
chazelas 11045  8237 11044  8237  0 22:08 pts/7    00:00:00 sleep 101
chazelas 11046  8237 11046  8237  0 22:08 pts/7    00:00:00 ps -fj
11045

Above $! is 11045, but job 1's process group is 11044. If I kill "$1", I'm going to kill only sleep 101, not sleep 100. If I kill -- "-$!" (that is kill all the processes in the process group 11045), that will fail because there's no such process group.

There, you want to use kill %1 (or kill %sleep or possibly kill %%) to kill that job (which will do a kill -- -11044).

Solution 3

It looks like when you typed stop, you were actually running the command that comes with upstart, which is unrelated to the csh or ksh builtin stop commands that you're familiar with. So its error message about an "Unknown job" doesn't apply to what you were trying to do.

ksh implements stop as an alias for kill -s STOP, which understands the %n notation for jobs or process group IDs. This alias will work in other shells, too, for example if your shell is bash:

$ alias stop='kill -s STOP'
$ sleep 30&
[3] 9820
$ stop %3
[3]+  Stopped     sleep 30

Comments

[Admin]

What is stop here? Do you mean the one that comes with upstart? What does that have to do with backgrounded jobs? Aren't you looking for this?

[Admin]

Sorry, this is not a duplicate of unix.stackexchange.com/questions/110911/how-to-get-the-job-i‌​d . This question is specifically asking to get the job ID, it's not asking how to resume a stopped process interactively on the command-line.

[Admin]

@Flimm any idea of how one can actually programmatically get the job ID of the last backgrounded job?

[Alen Milakovic]

So $! shows the most recent job that was backgrounded?

[Stéphane Chazelas]

Note that when you kill by job id (supported by all POSIX shells and (t)csh where the feature comes from), you kill a process group, not a process. kill %% is like kill -- "-$!". Jobs refer to process groups (think backgrounded pipelines or compound commands).

[Daniel Schneller]

Thanks for the nice explanation of the process groups and how a single job can represent more than one process :)

[rexford]

IIRC == if i remember correctly?