Sharing file descriptors

Question

If file descriptors are specific to each process (i.e. two processes may use the same file descriptor id to refer to different open files) then how is it possible to share transfer file descriptors (e.g. for shared mmaps) over sockets etc?

Does it rely on the kernel being mapped to the same numerical address range under each process?

Solution

When you share a file descriptor over a socket, the kernel mediates. You need to prepare data using the cmsg(3) macros, send it using sendmsg(2) and receive it using recvmsg(2). The kernel is involved in the latter two operations, and it handles the conversion from a file descriptor to whatever data it needs to transmit the file descriptor, and making the file descriptor available in the receiving process.

How can same fd in different processes point to the same file? provides useful background. The sending process sends a file descriptor which means something in relation to its (private) file descriptor table; the kernel knows what that maps to in the system-wide open file table, and creates a new entry as necessary in the receiving process’ file descriptor table.

Comments

[Kusalananda]

Related: stackoverflow.com/questions/28003921/…

[Kusalananda]

... apart from not talking about sharing of fds over sockets, yes.

[benjimin]

Surprising as the sendmsg(2) manual makes no mention of this behaviour (although recvmsg does).

[benjimin]

Similar explanation here: keithp.com/blogs/fd-passing The infrastructure for different processes possessing shared file descriptors was necessary for POSIX forking; the mystic bit is Linux's intercept-and-translate mechanism for communicating them on other occasions.

[benjimin]

Is there any way to usefully transfer a file descriptor (or a physical memory address) between processes without kernel mediation? Also, is there any way to perform this mediation process without a socket connection? (I'm interested in thread-safe ways to conveniently communicate an anonymous shared mmap to a process pool in python.)

[Stephen Kitt]

Physical memory addresses aren’t accessible from userspace, so no. Same goes for file descriptors: the full meaning of a file descriptor is only available using information in the kernel, so you can’t transfer one without going through the kernel. To transfer a file descriptor without a socket connection, you can fork...