C++ memory allocation mechanism performance comparison (tcmalloc vs. jemalloc)

Question

I have an application which allocates lots of memory and I am considering using a better memory allocation mechanism than malloc.

My main options are: jemalloc and tcmalloc. Is there any benefits in using any of them over the other?

There is a good comparison between some mechanisms (including the author's proprietary mechanism -- lockless) in http://locklessinc.com/benchmarks.shtml and it mentions some pros and cons of each of them.

Given that both of the mechanisms are active and constantly improving. Does anyone have any insight or experience about the relative performance of these two?

Solution 1

If I remember correctly, the main difference was with multi-threaded projects.

Both libraries try to de-contention memory acquire by having threads pick the memory from different caches, but they have different strategies:

• jemalloc (used by Facebook) maintains a cache per thread
• tcmalloc (from Google) maintains a pool of caches, and threads develop a "natural" affinity for a cache, but may change

This led, once again if I remember correctly, to an important difference in term of thread management.

• jemalloc is faster if threads are static, for example using pools
• tcmalloc is faster when threads are created/destructed

There is also the problem that since jemalloc spin new caches to accommodate new thread ids, having a sudden spike of threads will leave you with (mostly) empty caches in the subsequent calm phase.

As a result, I would recommend tcmalloc in the general case, and reserve jemalloc for very specific usages (low variation on the number of threads during the lifetime of the application).

Solution 2

I have recently considered tcmalloc for a project at work. This is what I observed:

• Greatly improved performance for heavy usage of malloc in a multithreaded setting. I used it with a tool at work and the performance improved almost twofold. The reason is that in this tool there were a few threads performing allocations of small objects in a critical loop. Using glibc, the performance suffers because of, I think, lock contentions between malloc/free calls in different threads.

• Unfortunately, tcmalloc increases the memory footprint. The tool I mentioned above would consume two or three times more memory (as measured by the maximum resident set size). The increased footprint is a no go for us since we are actually looking for ways to reduce memory footprint.

In the end I have decided not to use tcmalloc and instead optimize the application code directly: this means removing the allocations from the inner loops to avoid the malloc/free lock contentions. (For the curious, using a form of compression rather than using memory pools.)

The lesson for you would be that you should carefully measure your application with typical workloads. If you can afford the additional memory usage, tcmalloc could be great for you. If not, tcmalloc is still useful to see what you would gain by avoiding the frequent calls to memory allocation across threads.

Solution 3

Be aware that according to the 'nedmalloc' homepage, modern OS's allocators are actually pretty fast now:

"Windows 7, Linux 3.x, FreeBSD 8, Mac OS X 10.6 all contain state-of-the-art allocators and no third party allocator is likely to significantly improve on them in real world results"

http://www.nedprod.com/programs/portable/nedmalloc

So you might be able to get away with just recommending your users upgrade or something like it :)

Solution 4

There's a pretty good discussion about allocators here:

http://www.reddit.com/r/programming/comments/7o8d9/tcmalloca_faster_malloc_than_glibcs_open_sourced/

Solution 5

Your post do not mention threading, but before considering mixing C and C++ allocation methods, I would investigate the concept of memory pool.BOOST has a good one.

View more solutions

Comments

[John Dibling]

why are you using malloc in C++?

[Shayan Pooya]

@JohnDibling Performance

[John Dibling]

I guess the next natural question is, why are you using C++?

[Shayan Pooya]

There is a discussion about malloc vs. new here: stackoverflow.com/questions/184537/…. I am using malloc just for allocating blobs of data. There is no benefit in using new. (See the comments of the best answer)

[Matthieu M.]

@JohnDibling: I would note that common implementations of new rely on malloc to get memory anyway...

[John Dibling]

@Matthieu: I understand that.

[edA-qa mort-ora-y]

You can also get improved performance by simply not allocating as much. Object pools are helpful here. Can get a bit trickier to program, but if the allocation scheme is causing a performance problem then you're at the point where this should be considered.

[Bastienm]

about TC and PT malloc ou have some graph here goog-perftools.sourceforge.net/doc/tcmalloc.html but not example of program.

[Shayan Pooya]

Thanks. I'll read it. But as I said, I think it is outdated by now.

[Shayan Pooya]

Thanks. First, that seems good. I'll look at it. Second, I am in profile and optimize phase, and optimizing the bottlenecks (here memory allocation). Third, is there any problem in mixing C/C++ allocation methods? (other than making the code dirty/non-standard).

[v.oddou]

that's also my observation. and the observation of the CRT devs because the standard malloc today is just a wrapper over VirtualAlloc in win32 and mmap in linux.

[v.oddou]

I would speculate that migrating to a garbage collector is not as simple as just changing malloc to gc_malloc. You also need to change the types of your pointers to somekind of opaque handle type

[Basile Starynkevitch]

Not with Boehm's GC. Its gc_malloc is designed to be a replacement for malloc, and also gives a void*. It is a conservative GC.

[Scott Stensland]

yet another thread reddit.com/r/programming/comments/7o8d9/…

[Yakov Galka]

@v.oddou: all malloc implementations end up calling mmap at some point -- as that's the only way to get memory from the system (almost). The problem with mmap is that it is slow no matter what (it's a system call that involves a context switch) and it can allocate memory only with a page granularity (4K or more). The goal of malloc implementations is to get big chunks of memory from the system using mmap, and then allocate smaller objects in those memory regions from within the user process.

[Yakov Galka]

@rogerdpack: note that jemalloc is the FreeBSD's allocator.

[v.oddou]

@YakovGalka Without explicit research on my end to be sure, I'll emit a disclaimer of uncertainty. But by my current knowledge your information is wrong or outdated. Some old malloc used srbk to get blocks from the system. And mmap was a fallback for large independent blocks. Today it is called directly, not "at some point", malloc implementations are empty nowadays. they literally wrap mmap with no decorum. Also system calls have fast traps.

[Yakov Galka]

@v.oddou I sincerely recommend you to look at the sources of any of those "empty modern malloc implementations" and figure out why wrapping "mmap" involves thousands of lines of code.

[v.oddou]

@YakovGalka Yeah well, it's how I said. github.com/Chuyu-Team/VC-LTL/blob/master/src/ucrt/heap/… Just a wrapper to HeapAlloc and free is a wrapper to HeapFree. There is no thousands of lines, just minimalist boilerplate for errors and comments.

[Yakov Galka]

@v.oddou: ... except that it's not what you said. You said that "standard malloc today is just a wrapper over VirtualAlloc in win32 and mmap in linux". But as you just showed, that's not the case with MSVC CRT where it calls HeapAlloc instead. Now, you may want to learn the difference between MSVC CRT, WinAPI, and Windows NT kernel API. While VirtualAlloc is more or less a direct syscall and is WinAPI's mmap analogue, HeapAlloc is not a kernel API at all.

[Yakov Galka]

It is a user-space heap allocator, just like glibc malloc, jemalloc, tcmalloc, and dozen others are. And guess what? They all have thousands of lines of code sitting there between your malloc call and the kernel API that acquires memory (VirtualAlloc/mmap).