File descriptors across exec

Question

By default file descriptors remain open across the exec functions. The benefit is perhaps understandable for descriptors 0-2. But is there a practical use case for keeping other descriptors open? Are there any real applications that rely on this fact?

Solution 1

There's a flag you can set on a file descriptor (upon open(): O_CLOEXEC or later with fcntl(): FD_CLOEXEC) if you don't want that fd to be passed to executed commands.

That's what you should do for your internal file descriptors if you're going to execute commands.

In shells, that's what ksh93 does when you do exec 3< some-file for instance. For other shells or fds opened with { cmd1; cmd2; } 3< file, you'd need to close by hand if you don't want cmd1 or cmd2 to access that fd: {cmd1 3<&-; cmd2; } 3< file. That's good practice but not always followed as it's usually not critical if you don't do it.

Now, whether the feature is useful. Yes, several commands rely on it.

A few commands take a file descriptor as argument that is meant to have been opened by a caller. A few examples that come to mind:

• xterm with its -S option
• qemu for various things
• flock (to lock a file on the caller's fd)
• the test command aka [ for it's -t option (OK the test utility is built in most Bourne-like shells nowadays, but there still is a test command that can be executed).
• dialog needs file descriptors for input from the user, output and error to the user and input and output to the caller, so you can use extra fds for that.
• gpg or openssl to which you can specify a file descriptor to communicate the passphrase or other information.

There are a number of helper utilities (for instance, the need to execute could be to run a part of a command as a different user or group using a setuid/setgid executable) that rely on that.

Process substitution relies on it:

In, diff <(cmd1) <(cmd2), 2 file descriptors (to pipes) are passed to diff and diff accesses them by opening them via the special /dev/fd/n passed as argument.

For shells that don't have process substitution, you do the same by hand with things like:

cm1 | { cmd2 | diff /dev/fd/3 -; } 3<&0

Solution 2

TinyMUSH and probably many of its sibling and child codebases use this functionality of exec to great effect. One can issue a command to restart the server, possibly upgrading to an entirely new binary, while leaving users connected.

This is done by writing a small db of information about each connected user, including their file descriptor. The newly exec'd copy of TinyMUSH reads the restart db to restore its knowledge of connected users and pick up where it left off.

End result: new features be released with only a brief pause visible to users.

Nginx does something somewhat similar to do binary upgrades without losing connections.

Solution 3

Connected sockets can be passed to a child process this way, so for example a network server that accepts incoming connections can pass handling them to another process entirely.

See the source code for inetd, for a ubiquitous example.

Comments

[thrig]

Quite the contrary, OpenSSH in particular takes pains to close descriptors >= 3 by default (via the closefrom(2) call).

[Andrew Henle]

@thrig That doesn't mean it's useless to pass an open file descriptor for another purpose. In fact, the great pains OpenSSH goes through to make certain all other file descriptors are closed should indicate just how useful it can be to pass open file descriptors this way.

[Kaz]

Even some shell scripts set up descriptors > 2 and pass them into command pipes for useful effect.

[Stéphane Chazelas]

The question is about executed commands, not child processes. Note that for inetd, the socket is on fds 0,1,2.

[Andrew Henle]

@StéphaneChazelas The question is about executed commands, not child processes. Note that for inetd, the socket is on fds 0,1,2 No. The questions are But is there a practical use case for keeping other descriptors open? and Are there any real applications that rely on this fact? My answer is an example of a "practical use case" with a "real application" - passing an already-open socket to a child process. The fact that inetd uses file descriptors 0, 1, and 2 for the purpose of passing the open socket is a convention that lines up with keeping those open for other reasons.

[r.v]

This is nice to know there are so many applications. Frankly, I had expected just a few arcane use cases.

[Stéphane Chazelas]

Or the caller can pass that FD as argument or env var to the callee.

[JdeBP]

There are in fact plenty of "real generic" programs that do this. There are Bernstein tools that expect to pass/receive file descriptors 3, 6, and 7 as part of well-defined protocols such as UCSPI and Bernstein TTY handling. systemd's "socket activation" has a protocol that involves file descriptors 3 and upwards. And so on.

[dave_thompson_085]

Yes, inetd is a practical case of passing 0,1,2. The very source you link to carefully closes everything other than 0,1,2 before exec'ing the program; see main and run_service. It is not an example of passing fds other than 0,1,2, which was the question.

[penguin359]

@r.v there's actually a lot more. Look up SystemD Socket Activation for system services, for example.