Is there a need to check for NULL after allocating memory, when kernel uses overcommit memory

Question

It is general practice to check for NULL (whether memory is successfully allocated) after a malloc(), some thing like

void *ptr = malloc(10);    
if (ptr != NULL) {  
  // do some thing usefull  
} else {  
 // no memory. safely return/throw ...  
}  

with memory overcommit enabled in kernel, is there a chance of getting NULL? Should I follow the practice of religiously checking NULL for each allocation? Will malloc return NULL inspite of aggresive overcommit mechanism (I guess value 1)?

As a matter of fact Android kernel uses memory overcommit (not sure about the value, would love to know it(overcommit value) and its significance). Some of the framework source(C/C++) code in Android (might be 3rd party) doesn't check for NULL nor catch bad_alloc after allocations. Am I missing something?

There are some threads in SO regarding overcommit memory, but none of them resolved my confusion.

EDIT: If aggressive overcommit is being employed NULL wont be returned(assumption 1). When there is no physical memory available and up on trying to access the allocated memory (write in to the memory allocated), OOM will kill some process and allocates memory for the application till it gets killed in turn(assumption 2). In either case i dont see any need for cheking NULL (memory getting allocated or process getting killed). am i right in my assumptions?
Portability is not a concern for this question.

Solution 1

Yes, you should still check for failures returned by malloc. In an environment that overcommits memory you will not be able to detect and recover from failures due to the environment running out of physical storage required when you write to parts of the address space that have been allocated to your program by a previous call to malloc.

However, this is not the only problem that would cause a malloc to fail in a traditional environment. A request for a particularly large block of memory when the address space of your program has become fragmented may fail even if there is potentially enough total physical memory to satisfy the request. Because there is no contiguous range of free address space malloc must fail. This type of failure must be signaled by malloc returning NULL, whether or not the environment is overcommitting memory.

Solution 2

You must check return value for NULL every time. Any library function can fail. Even fclose() do (on disconnected NFS share, and error from fclose of NFS file means, that data was not saved).

The most of software is badly written and doesn't contain all checks.

malloc can't return something other than NULL or pointer. All-or-nothing. You can't get 1 byte from malloc if you ask for 10.

Solution 3

It would be advisable to check for NULL religiously across all functions calls that may return NULL, regardless of whether the kernel has over-committable memory or not.

This following code segment below shows how to check if the call to malloc worked or not...

void *ptr = malloc(10);
if (ptr != NULL){
   /* Do something here with ptr */
}else{
   /* Do something here if it fails */
}

File operations, memory operations to name but a few will return a NULL upon failure.

Hope this helps, Best regards, Tom.

Solution 4

well...on Linux since memory is not page backed (initially) and only creates page backing after first read/write, the OS will always succeed to give you memory (unless you exhausted the address space, something not possible in 64 bit systems). So if it runs out of memory and cannot give you the promised memory, OOM killer will just kill your application or some other application to give you the page backing you need. So whether you do the NULL check or not, the out come is the same, a crash.......

View more solutions

Comments

[FL4SOF]

my question is not with implementation of malloc, but usage of malloc. with memory overcommit enabled , there is very less chance of getting NULL(theoretically) and it is evident in some of the android c/c++ framework sources - which doesn't do this check after allocations.

[osgx]

With overcommit mallic will fail farther. opsmonkey.blogspot.com/2007/01/linux-memory-overcommit.html Generally, malloc will not fail, when memory is available and ulimit is not reached

[FL4SOF]

'code segment is a dangerous assumption' can you explain? 'Android middle ware' sorry for being so generic, 'some of the frame work source(C/C++) code in android' i should say. I will edit my question.

[FL4SOF]

"With overcommit malloc will fail farther." I believe this is wrong assumption, please look into the link you mentioned. It says demo1: malloc memory and do not use it: Allocated 1.4TB, killed by OOM killer demo2: malloc memory and use it right away: Allocated 7.8GB, killed by OOM killer (process killed by OOM killer and not a case of malloc fail).

[osgx]

demo1 - allocates memory and dont use this allocated memory - so overcommit will work, pages will not really allocated. demo2 - allocates memory and memset it to 0 (this will remap this memory from write-protected zeropage filled with 0 to real memory, so it DISABLES overcommiting) It there swap enabled on typical Android box?

[kay]

Your answer is an unrelated rant. I removed the half of it. Also your answer is bad, so I voted it down.

[Danger Saints]

I don't know why this site allow you to edit my post. But with OOM-Killer and Memory Overcommit, check null pointer after malloc is no longer necessary and become misleading.

[kay]

Please see stackoverflow.com/faq#editing. You can undo my changes just as well … To the original question: it is true that modern OSes allocate RAM for process lazily and overcommit. And when the memory is actually accessed, it will be allocated. And maybe someone has to get OOM'd just then. But that is not the question. malloc calls brk to increase its heap eagerly. The OS my deny the request. In this case malloc would return NULL. E.g. ulimit could limit the maximal virtual memory size of the process in question.

[Skrymsli]

Maybe this is off topic, but I'm curious what action do you take in the /* Do something here if it fails */ section? Presumably, whatever you do will not be able to allocate memory if you are truly unable to allocate 10 bytes of memory. You may get into an infinite loop of checking for out of memory and handling it.. Do you just exit? Is that better than crashing on null-dereference somehow?

[Skrymsli]

If you try to allocate 1M of memory and malloc returns NULL due to fragmentation how do you proceed to handle this case? What if you attempt to allocate a lot of memory due to a bug, wouldn't you rather crash and get a core dump than cleanly exit or otherwise work around the bug?

[CB Bailey]

@Skrymsli: It depends on the application and the environment. If "crashing" immediately is required then I'd prefer an explicit abort rather than carrying on running an indeterminate amount of time until some use of the null pointer causes a crash as a side-effect. Some applications might be able to purge lots of allocated memory and recover in some way. The premise of the question is, though, should I check for null because, surely, with over-commit it will never happen which I claim is not true.

[Danger Saints]

Best practice about memory management is to 'prevent' evil malloc before it really happens. If evil malloc happens, nothing helps.