Malloc header contents

Question

So in most implementations malloc stores an header before the allocated memory to keep track of the allocated memory size (so that it can do free and recalloc). What are the header contents?

I wrote a naive code to find it but it doesn't make any sense

int * ptr;
ptr = malloc(12*sizeof(int));
printf("Header = %d\n",*(ptr-1));

It returns

Header = 57

What is happening here?

Solution 1

I'm guessing you want to learn and see how the memory is allocated. I would ignore the Undefined Behaviour answers. They are right (of course) when you talk about portability and such, but that is not your question. I think it is a really good idea to try and figure out how the allocation is done.

First I would encourage you to start looking at the malloc implementation for your platform. If that code is not available, you are out of luck and the only think you can do is google for clues how the allocation is done.

If you run linux, you can look at the malloc implementation of glibc or uclibc. Here a link to the uclibc implementation: http://git.uclibc.org/uClibc/tree/libc/stdlib/malloc/malloc.c The code has lot of comments, but can be overwhelming.

For your question, look at http://git.uclibc.org/uClibc/tree/libc/stdlib/malloc/malloc.h on line 104. which is the part you are talking about. You see the layout depends on MALLOC_HEADER_SIZE which can be different for different systems. By reading the code you can learn which types to use, and on which offset the memory size is stored (in this specific implementation)

Of course, above is just an example implementation from uclibc to get you started...

Solution 2

Nobody really answered where the number "57" came from, so here's my understanding of it.

The header that is set when using malloc or calloc, at least on the architectures that I've used, is the total size of chunk of memory on the runtime heap, plus a few "boolean flags".

You requested 12 ints, with each int (presumably) being 4 byte. 12x4 = 48. Another 4 byte, for the header block itself (the number 57), is added to this count, leaving us at 52.
So why are you getting 57?

Well, Malloc and Calloc only request memory in 8-bit chunks, to avoid bus errors. The next higher multiple of 8 is 56.

Now, recall that any number that is divisible by 8 has a binary representation that will always end in three 0's. Being the memory-conserving language that C is, compilers take advantage of this fact and use the last three 0's as boolean flags.

In this specific case, the last boolean flag is set, adding 1 to 56, resulting in the number 57 when read as an int.

Solution 3

None of this is any of your business (it's an implementation detail, opaque to the user), and what you do is undefined behaviour.

That's as far as the standard goes.

Now, if you want to be naughty and poke around memory, beware that pointer arithmetic operates in units of the type size (e.g. 4 for int). So you should always cast your pointers to char* (or an unsigned version) for such shenanigans:

struct Foo * f = malloc(sizeof(Foo) * 7);
const unsigned char * const i_know_what_im_doing = f;

printf("%02X\n", *(i_know_what_im_doing - 1));

Comments

[Joe]

+1 because "shenanigans" is the only way to describe what the OP is trying to do. I would have to agree with cnicutar and say that it makes since that this information would be stored elsewhere in a table.

[Kerrek SB]

Joe: Easiest to find out is to look up the implementation of malloc. If it's dlmalloc or pt2malloc you can probably find the paper somewhere that describes it, since those are fairly famous.

[Bruce]

@Kerrek: So I *(i_know_what_im_doing - 4) gives me the size of the allocated heap. But the MALLOC_HEADER size increases as I allocate more memory. Is this normal?

[Kerrek SB]

@Bruce: I have no idea, as all of this is totally platform and implementation dependent. Your best bet is to look at the implementation of your own malloc() and find out yourself what's "normal". You can also print a couple of more bytes backwards if you like and see what's there.

[Hassan Syed]

I don't get why people are criticising curiosity......... these questions are what creates a systems programmer ....

[Kerrek SB]

@HassanSyed: The curious programmer will be digging through their implementation code anyway. The danger is the casual C learner who stumbles across a page like this and starts thinking they can make assumptions. I'd rather protect the world from a hundred baseless assumptions than provide marginal utility to the genuinely curious.