How does catching an OutOfMemoryException work?

Question

I am a little bit confused about the fact that we can just catch an OutOfMemoryException using a try/catch block.

Given the following code:

Console.WriteLine("Starting");

for (int i = 0; i < 10; i++)
{
    try
    {
        OutOfMemory();
    }
    catch (Exception exception)
    {
        Console.WriteLine(exception.ToString());
    } 
}

try
{
    StackOverflow();
}
catch (Exception exception)
{
    Console.WriteLine(exception.ToString());
}

Console.WriteLine("Done");

The methods I used to create the OutOfMemory + StackOverflowException:

public static void OutOfMemory()
{
    List<byte[]> data = new List<byte[]>(1500);

    while (true)
    {
        byte[] buffer = new byte[int.MaxValue / 2];

        for (int i = 0; i < buffer.Length; i++)
        {
            buffer[i] = 255;
        }

        data.Add(buffer);
    }
}

static void StackOverflow()
{
    StackOverflow();
}

It prints out the OutOfMemoryException 10 times and then terminates due to the StackOverflowException, which it can't handle.

The RAM graph looks like that while executing the program:

My question now it why are we able to catch the OutOfMemoryException? After catching it we can just go on execute any code we want. As proven by the RAM graph, there is memory released. How does the runtime know which objects it can GC and which are still required for further execution?

Solution 1

The GC makes an analysis on the references that are used in the program, and can throw away any object that isn't used anywhere.

An OutOfMemoryException doesn't mean that the memory is completely depleted, it just means that a memory allocation failed. If you tried to allocate a large memory area at once, there may still be plenty of free memory left.

When there isn't enough free memory for an allocation, the system does a garbage collection to try to free up memory. If there still isn't enough memory for the allocation, it will throw the exception.

A StackOverflowException is not possible to handle, because it means that the stack is full, and it's not possible to remove anything from it as it is with the heap. You would need more stack space to continue running the code that would handle the exception, but there is no more.

Solution 2

The OutOfMemoryException is quite possibly thrown because you are running a 32 bit program, with the memory graph you have not indicated how much ram the system has, so perhaps try building it as a 64 bit, and maybe use MemoryFailPoint to prevent this occurring anyway.

You could also let us know what is in the OutOfMemory() function for a clearer picture.

P.S. StackOverFlow is the only error which cannot be handled.

Edit: as mentioned above, and I thought it only logical and hence didn't mention it earlier, if you for example try to allocate more memory than you have 'spare' then it is not possible to do so and an exception occurs. As you are allocating large arrays with your data.Add() it falls over before the final 'illegal' add occurs, hence there is still free memory.

So I would assume that it is at this point data.Add(buffer); the issue occurs during the building of the array when you trip the 2GB process limit by adding a 400MB byte array to 'data', e.g. an array of around 1 billion objects at 4 bytes a piece I would expect to be around 400MB.

P.S. Up until .net 4.5 max process memory allocation is 2GB, after 4.5 larger are available.

Comments

[Matthew Watson]

Show us your OutOfMemory() method. I bet you're not keeping a reference to the memory it allocates, so the GC can release it after it returns.

[Matthew Watson]

Yep, so as soon as OutOfMemory() throws, all the memory it allocated can be freed up by the GC.

[Paul Zahra]

Ah so it does go out of memory, which leads me to believe you are hitting application limits and not necessarily system limits.

[GameScripting]

I am not interested in how to fix it, there's no actal problem, it's all about research and understanding how thr CLR works

[Øyvind Bråthen]

P.S. StackOverflow is not the only one you cannot catch. You have AccessViolationException also.

[Paul Zahra]

Interesting read stackoverflow.com/questions/3469368/…