Under what circumstances can malloc return NULL?

Question

It has never happened to me, and I've programming for years now.

Can someone give me an example of a non-trivial program in which malloc will actually not work?

I'm not talking about memory exhaustion: I'm looking for the simple case when you are allocating just one memory block in a bound size given by the user, lets say an integer, causes malloc to fail.

Solution 1

Yes.

Just try to malloc more memory than your system can provide (either by exhausting your address space, or virtual memory - whichever is smaller).

malloc(SIZE_MAX)

will probably do it. If not, repeat a few times until you run out.

Solution 2

You need to do some work in embedded systems, you'll frequently get NULL returned there :-)

It's much harder to run out of memory in modern massive-address-space-and-backing-store systems but still quite possible in applcations where you process large amounts of data, such as GIS or in-memory databases, or in places where your buggy code results in a memory leak.

But it really doesn't matter whether you've never experienced it before - the standard says it can happen so you should cater for it. I haven't been hit by a car in the last few decades either but that doesn't mean I wander across roads without looking first.

And re your edit:

I'm not talking about memory exhaustion, ...

the very definition of memory exhaustion is malloc not giving you the desired space. It's irrelevant whether that's caused by allocating all available memory, or heap fragmentation meaning you cannot get a contiguous block even though the aggregate of all free blocks in the memory arena is higher, or artificially limiting your address space usage such using the standards-compliant function:

void *malloc (size_t sz) { return NULL; }

The C standard doesn't distinguish between modes of failure, only that it succeeds or fails.

Solution 3

Any program at all written in c that needs to dynamically allocate more memory than the OS currently allows.

For fun, if you are using ubuntu type in

 ulimit -v 5000

Any program you run will most likely crash (due to a malloc failure) as you've limited the amount of available memory to any one process to a pithy amount.

Solution 4

Unless your memory is already completely reserved (or heavily fragmented), the only way to have malloc() return a NULL-pointer is to request space of size zero:

char *foo = malloc(0);

Citing from the C99 standard, §7.20.3, subsection 1:

If the size of the space requested is zero, the behavior is implementationdefined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object.

In other words, malloc(0) may return a NULL-pointer or a valid pointer to zero allocated bytes.

Solution 5

Pick any platform, though embedded is probably easier. malloc (or new) a ton of RAM (or leak RAM over time or even fragment it by using naive algorithms). Boom. malloc does return NULL for me on occasion when "bad" things are happening.

In response to your edit. Yes again. Memory fragmentation over time can make it so that even a single allocation of an int can fail. Also keep in mind that malloc doesn't just allocate 4 bytes for an int, but can grab as much space as it wants. It has its own book-keeping stuff and quite often will grab 32-64 bytes minimum.

View more solutions

Comments

[Ferruccio]

My first computer was an 8-bit system with 56KB of RAM. malloc() returned NULL all too often.

[RanZilber]

thanks that buts thats pretty trivial. I'm talking about the simple case we you allocate just a bouded piece of memory. Can it still fail? see my edit

[Useless]

I'm sorry, clearly I should have posted a non-trivial program allocating hundreds of millions of sensible-sized objects for a good reason, which would obviously have exactly the same result. It's unlike to be both readable, concise and non-trivial though!

[Admin]

This describes the contract but, when can/does it happen? ("...never happened to me ... non-trivial program in which malloc will actually not work? ... when you are allocating just one memory block in a bound size given by the user ... is it still possible that malloc will fail?")

[Admin]

Although, since the data is not being made "dirty" ... perhaps it's not actually realized in [physical] memory? Not saying that there are infinite resources ...

[visual_learner]

@pst - Possibly, but if I add that the code will be too readable.

[Michael Dorgan]

C-64 with the basic memory mapped to malloc?

[Admin]

I like this answer because it talks about memory fragmentation and book-keeping. Just because [physical] memory is "there" doesn't mean it is always "available". That is a, malloc can even fail with less data allocated than total memory. (I like the complementing answer about virtual memory and over-committing as well.)

[R.. GitHub STOP HELPING ICE]

It's disabled at run time not compile time, and most competent admins disable it unless they have a specific reason not to.

[R.. GitHub STOP HELPING ICE]

The data will be dirty immediately because 999 is smaller than a page. Thus each page will get touched at least once for the bookkeeping structures between allocations.

[visual_learner]

@R.. - This may be something that merits its own question, if not delving more deeply into the subject of memory management systems, but why don't page-sized allocations need bookkeeping structures?

[R.. GitHub STOP HELPING ICE]

@Chris: If the allocation is larger than a page and contains one or more whole pages within the region returned for use by the application, then there's no reason to expect those have been touched. But if you just keep allocating 999 bytes, successive allocations' bookkeeping information will be separated by less than a page and thus you'll end up touching every page that gets allocated.

[Brian Bulkowski]

This answer is the case where memory is exhausted. The poster asked for a case where malloc() returns 0 and memory is NOT exhausted.

[Brian Bulkowski]

The poster asked if there is a case where malloc() returns 0 and memory is NOT exhausted. The case of fragmentation gets a lot more amusing, but it'll take non-trivial code (tuned to a particular allocator) to show the pattern.

[Brian Bulkowski]

This looks like memory exhaustion to me, didn't the poster ask for a case that wasn't memory exhaustion?

[paxdiablo]

Brian, I think that's covered in my answer. Memory exhaustion is defined as malloc not being able to give you what you want. Whether you're truly out of memory totally or whether you asked for 60 bytes and the allocator has one billion 30-bytes chunks that can't be coalesced, doesn't really matter. It's exhaustion in both those cases. In any case, a malloc that returns NULL every tenth time you call it regardless of how much memory it has (written by a true sadist) still complies with the standard.

[Useless]

That was edited into the question after the answer was written. However, your VM running out of pages, or hitting an overcommit limit, or your process running out of contiguous addresses are different forms of exhaustion: I have no idea which of them "memory exhaustion" is supposed to indicate.

[Koray Tugay]

Are you sure? Can you please see: stackoverflow.com/questions/29613162/…

[Koray Tugay]

This is not even an answer.

[tbodt]

On my computer (macOS sierra) malloc never returns NULL. Instead, at around 50 GiB of virtual memory allocated, the program gets a SIGKILL.

[chux - Reinstate Monica]

Not quite. The -128 stored in bufferSize is converted to a size_t, resulting in a large number. The malloc() resulted in NULL because that large amount was not available. OTOH, it might have worked. When "the malloc param is ... 0" is another story.

[Soucup Bogdan]

indeed that was my fault, but its is still a mistake, I think for this specific case in normal condition this will alwais return 0

[chux - Reinstate Monica]

Agree it is a mistake, but the allocation, being SIZE_MAX-128+1 (huge) could return non-NULL. example