Why does the memory consumption grows when using more threads on Linux? (C++)

Question

Program:

I created a C++ calculation program for high volume of data, which could be run on 1 or more threads. (via config file)

The program environment is the following: C++, OpenMp, Redhat-x64, malloc/free

Results on Linux:

• When I run it on 1 thread, process size is 177 MB.
• When I run it on 2 threads, process size is 317 MB.
• When I run it on 4 threads, process size is 600 MB.

Results on Windows:

• Process size still the same regardless the number of threads used, 110MB.

Question:

Why does the memory consumption grows when using more threads on Linux?

Solution 1

A fair guess is that delete in Linux will cache released memory for future new requests, just as it does on Windows, but Linux has a cache per thread whereas Windows has a cache per process.

(More accurately, it would be the C++ runtime library that determines how such a cache works).

Solution 2

Use /proc/<PID>/maps, or better pmap(1), to figure out where the pages go.

Solution 3

Every thread you start needs a stack allocated to run it, and memory for any thread-local variables. I'm not entirely sure that 150Mb per thread looks sensible, but a thread local array could cause something like that.

Comments

[Axel Borja]

I do not find any information about this cache, is there a name for it?

[MSalters]

It's considered an implementation detail, without an explicit name. It's also somewhat of an abstraction. I believe that some implementations will have multiple pools or caches, to keep freed blocks of the same size grouped together.