Malloc -> how much memory has been allocated?

Question

# include <stdio.h>
# include <stdbool.h>
# include <string.h>
# include <stdlib.h>

int main ()
{
  char * buffer;
  buffer = malloc (2);

  if (buffer == NULL){
    printf("big errors");
  }
  strcpy(buffer, "hello");
  printf("buffer is %s\n", buffer);
  free(buffer);
  return 0;
}

I allocated 2 bytes of memory to the pointer/char buffer yet if I assign the C-style string hello to it, it still prints the entire string, without giving me any errors. Why doesn't the compiler give me an error telling me there isn't enough memory allocated? I read a couple of questions that ask how to check how much memory malloc actually allocates but I didn't find a concrete answer. Shouldn't the free function have to know exactly how much memory is allocated to buffer?

Solution 1

The compiler doesn't know. This is the joy and terror of C. malloc belongs to the runtime. All the compilers knows is that you have told it that it returns a void*, it has no idea how much, or how much strcpy is going to copy.

Tools like valgrind detect some of these errors. Other programming languages make it harder to shoot yourself in the foot. Not C.

Solution 2

No production malloc() implementation should prevent you from trying to write past what you allocated. It is assumed that if you allocate 123 bytes, you will use all or less than what you allocated. malloc(), for efficiency sake, has to assume that a programmer is going to keep track of their pointers.

Using memory that you didn't explicitly and successfully ask malloc() to give you is undefined behavior. You might have asked for n bytes but got n + x, due to the malloc() implementation optimizing for byte alignment. Or you could be writing to a black hole. You never can know, that's why it's undefined behavior.

That being said ...

There are malloc() implementations that give you built in statistics and debugging, however these need to be used in lieu of the standard malloc() facility just like you would if you were using a garbage collected variety.

I've also seen variants designed strictly for LD_PRELOAD that expose a function to allow you to define a callback with at least one void pointer as an argument. That argument expects a structure that contains the statistical data. Other tools like electric fence will simply halt your program on the exact instruction that resulted in an overrun or access to invalid blocks. As @R.. points out in comments, that is great for debugging but horribly inefficient.

In all honesty or (as they say) 'at the end of the day' - it's much easier to use a heap profiler such as Valgrind and its associated tools (massif) in this case which will give you quite a bit of information. In this particular case, Valgrind would have pointed out the obvious - you wrote past the allocated boundary. In most cases, however when this is not intentional, a good profiler / error detector is priceless.

Using a profiler isn't always possible due to:

• Timing issues while running under a profiler (but those are common any time calls to malloc() are intercepted).
• Profiler is not available for your platform / arch
• The debug data (from a logging malloc()) must be an integral part of the program

We used a variant of the library that I linked in HelenOS (I'm not sure if they're still using it) for quite a while, as debugging at the VMM was known to cause insanity.

Still, think hard about future ramifications when considering a drop in replacement, when it comes to the malloc() facility you almost always want to use what the system ships.

Solution 3

You've written past the end of the buffer you allocated. The result is undefined behavior. Some run time libraries with the right options have at least some ability to diagnose problems like this, but not all do, and even those that do only do so at run-time, and usually only when compiled with the correct options.

Solution 4

How much malloc internally allocates is implementation-dependent and OS-dependent (e.g. multiples of 8 bytes or more). Your writing into the un-allocated bytes may lead to overwriting other variable's values even if your compiler and run-time dont detect the error. The free-function remembers the number of bytes allocated separate from the allocated region, for example in a free-list.

Solution 5

Why doesnt the compiler give me an error telling me there isnt enough memory allocated ?

C does not block you from using memory you should not. You can use that memory, but it is bad and result in Undefined Behaviour. You are writing in a place you should not. This program might appear as running correctly, but might later crash. This is UB. you do not know what might happen.

This is what is happening with your strcpy(). You write in place you do not own, but the language does not protect you from that. So you should make sure you always know what and where you are writing, or make sure you stop when you are about to exceed valid memory bounds.

I read a couple of questions that ask how to check how much memory malloc actually allocates but I didn't find a concrete answer. Shouldn't the 'free' function have to know how much memory is exactly allocated to 'buffer' ?

malloc() might allocate more memory than you request cause of bit padding.

More : http://en.wikipedia.org/wiki/Data_structure_alignment

free() free-s the exact same amount you allocated with malloc(), but it is not as smart as you think. Eg:

int main()
{
     char * ptr = malloc(10);
     if(ptr)
     {
         ++ptr; // Now on ptr+1
         free(ptr); // Undefined Behaviour
     }
}

You should always free() a pointer which points to the first block. Doing a free(0) is safe.

View more solutions

Comments

[holex]

you were lucky you did not override any important fragments of the memory by the 4 extra bytes which you don't own; you won't be such lucky every time, and that will lead crash.

[Fusion]

@holex You mean OS crash?

[holex]

nowadays, it is unreal to fear/hope that you could override a memory segment of the OS or other app, due to memory protection, but you can mess up easily your own app and could make it crashing.

[Admin]

@bmaurgulies, yes sorry. fixed.

[fouronnes]

But in other languages, when you do shoot yourself in the foot, it blows your whole leg off !

[R.. GitHub STOP HELPING ICE]

Actually a malloc implementation can protect you from doing this, if it arranges for the the N bytes to the last N bytes of a page, and puts a guard page after it... :-) This is really inefficient but it's a good debugging implementation.

[Tim Post]

@R.. Updated my answer for clarity, I was speaking in the context of 'why doesn't the system malloc() stop me from doing this?'