How to profile memory usage?

Question

I am aware of Valgrind, but it just detects memory management issues. What I am searching is a tool that gives me an overview, which parts of my program do consume how much memory. A graphical representation with e.g. a tree map (as KCachegrind does for Callgrind) would be cool.

I am working on a Linux machine, so windows tools will not help me very much.

Solution 1

Use massif, which is part of the Valgrind tools. massif-visualizer can help you graph the data or you can just use the ms_print command.

Solution 2

Try out the heap profiler delivered with gperftools, by Google. I've always built it from sources, but it's available as a precompiled package under several Linux distros.

It's as simple to use as linking a dynamic library to your executables and running the program. It collects information about every dynamic memory allocation (as far as I've seen) and save to disk a memory dump every time one of the following happens:

• HEAP_PROFILE_ALLOCATION_INTERVAL bytes have been allocated by the program (default: 1Gb)
• the high-water memory usage mark increases by HEAP_PROFILE_INUSE_INTERVAL bytes (default: 100Mb)
• HEAP_PROFILE_TIME_INTERVAL seconds have elapsed (default: inactive)
• You explicitly call HeapProfilerDump() from your code

The last one, in my experience, is the most useful because you can control exactly when to have a snapshot of the heap usage and then compare two different snapshots and see what's wrong.

Eventually, there are several possible output formats, like textual or graphical (in the form of a directed graph):

Using this tool I've been able to spot incorrect memory usages that I couldn't find using Massif.

Solution 3

A "newer" option is HeapTrack. Contrary to massif, it is an instrumented version of malloc/free that stores all the calls and dumps a log.

The GUI is nice (but requires Qt5 IIRC) and the results timings (because you may want to track time as well) are less biased than valgrind (as they are not emulated).

Comments

[math]

Thanks, the screenshots looking very great..., I think this is what I always wanted.

[Lightness Races in Orbit]

Building massif-visualizer on CentOS 7 is a bit of a pain, but it is worth it. Instructions

[Guy Avraham]

As far as I understand, I think this answer should be noted as well (if the application is statically linked the "Valgrind" family tools won't work): stackoverflow.com/a/13591746/1971003

[lahjaton_j]

Also available as dpkg: sudo apt install massif-visualizer

[pretzlstyle]

This doesn't seem to be supported for MPI programs. Is that correct?

[Vasiliy Soshnikov]

Hello, callgrind is nice tool, but it should be used for other purposes. Probably you are meaning massif (valgrind.org/docs/manual/ms-manual.html)?