Sharing memory between processes through the use of mmap()

Question

I'm in Linux 2.6. I have an environment where 2 processes simulate (using shared memory) the exchange of data through a simple implementation of the message passing mode.

I have a client process (forked from the parent, which is the server) which writes a struct(message) to a memory mapped region created (after the fork) with:

message *m = mmap(NULL, sizeof(message), PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0)

This pointer is then written to a queue (in form of a linked list) into another shared memory area which is common to server and client process (because if was created prior to fork with the same code above). This area is then read by the server which obtains the pointer to the message and processes it.

The problem is that *m is created after the fork() and when the server process tries to access the pointed memory location, i get a segmentation error. Is it possible to attach that region of memory to the server POST forking, after the client creates it?

NOTE: I don't want to mmap the pointer to message before forking (and then sharing it prior with the server) because I typically don't know how many messages the client wants to send to the server, and also there may be more than 1 client process, so I'd like to create a new block of shared memory only when a client needs to send a message, and unmap it after the server has received that message.

NOTE: This is for academic purpose: I know this is not the best way to solve this problem, but I just need to follow this path.

Thanks in advance!

Solution 1

Is it possible to attach that region of memory to the server POST forking, after the client creates it?

MAP_ANONYMOUS|MAP_SHARED mapped memory can only be accessed by the process which does that mmap() call or its child processes. There is no way for another process to map the same memory because that memory can not be referred to from elsewhere since it is anonymous.

Using shm_open() call it is possible to create named shared memory which can be referred to and mapped by unrelated processes.

Solution 2

Just for anyone reading this question in 2018 and later. The solution is now to use memfd_create to create an anonymous file and use a unix socket to pass this file handle to the other process.

memfd_create is a linux only syscall

Solution 3

That's not going to work.

If you create a mapping after the fork(), it won't be the same in the other related process(es).

You can't assume the sharing of pointers in this way.

If you really want to do it this way (I would not recommend it!), you should mmap a big area before the fork(), then allocate somehow, buffers of a suitable size (without race conditions with other processes, of course!) and pass those pointers.

Two related processes which call mmap() after a fork, may get the same pointer back, pointing at different memory. In fact this is extremely likely.

Comments

[Andrea Sprega]

Thanks! I used shm_open() and then mmap() with the given fd, it solved the problem partially. Is it possible to rename (using rename()) the "tag" string I supply to shm_open() after the fd is mmapped? I tried doing this but then, when I try to use shm_open() from another process using the renamed tag i get a runtime "Bus Error". NOTE: I noticed that in /sys/shm/ the renamed file exists, so the renaming seems to succeed.

[Maxim Egorushkin]

shm_open("name") on Linux normally translates to open("/dev/shm/name"). You should be able to rename it just like a regular file. Note that "/dev/shm/" is the default mount point of tmpfs on Linux. It can be mapped elsewhere. See google.com/codesearch/p?hl=en#xy1xtVWIKOQ/pub/glibc/releases‌​/…

[Andrea Sprega]

Thanks for the note. Do you have any clue why the renaming could cause a Bus Error if I try, then, to access the shared memory zone?

[Maxim Egorushkin]

Does shm_open() result in SIGBUS?

[Andrea Sprega]

Yes. The client process does shm_open(), mmap() and then rename() the "file". Then the server uses shm_open() again (with the renamed tag) and I get a Bus Error in the terminal, which I think corresponds to a SIGBUS.

[Maxim Egorushkin]

Try running under a debugger and see what causes SIGBUS, I don't think it is shm_open(). SIGBUS is often a result of accessing mapped memory beyond the end of file (the file has been truncated while being mapped).

[jclin]

/dev/shm or shm_open has default quota limitation of half physical memory size. You can shm_open and mmap a larger memory space initially, but when you write beyond the limitation or someone else also uses /dev/shm or shm_open so that the remain space is zero, your program directly gets SIGBUS.

[Alex Shpilkin]

... And it’s not a Linux-only syscall anymore since 2019, because FreeBSD now has it too, though you had access to equivalent functionality there even before that because FreeBSD explicitly allows you to ftruncate(2) and read(2) descriptors you got from shm_open(2). Linux actually had it since 3.17 in 2014, not 2018, though Glibc did in fact only deign to expose it in 2.27 in 2018. On OpenBSD ≥ 5.4 (2013), you can use shm_mkstemp(2) instead. Another SO answer has the details.

[Lothar]

@AlexShpilkin Nice, now lets hope we get it in macOS too. Not sure how good Apple is in syncing Darwin with FreeBSD nowadays