C# not releasing memory after task complete

Question

The following code is a simplified example of an issue I am seeing. This application consumes approx 4GB of memory before throwing an exception as the dictionary is too big.

 class Program
 {
    static void Main(string[] args)
    {
        Program program = new Program();

        while(true)
        {
            program.Method();
            Console.ReadLine();
        }
    }

    public void Method()
    {
        WasteOfMemory memory = new WasteOfMemory();
        Task tast = new Task(memory.WasteMemory);
        tast.Start();
    }


}

public class WasteOfMemory
{
     public void WasteMemory()
     {
         Dictionary<string, string> aMassiveList = new Dictionary<string, string>();

         try
         {
             long i = 0;
             while (true)
             {
                 aMassiveList.Add(i.ToString(), "I am a line of text designed to waste space.... I am exceptionally useful........");
                 i++;
             }

         }
         catch(Exception e)
         {
             Console.WriteLine("I have broken myself");
         }
     }
}

This is all as expected, although what we cannot currently work out is when this memory should be released from the CLR.

We have let the task complete and then simulated a memory overload situation, but the memory consumed by the dictionary is not released. As the OS is running out of memory, is it not putting pressure on the CLR to release the memory?

However and even more confusing, if we wait until the task has completed, then hit enter to run the task again the memory is released, so obviously the previous dictionary has been garbage collected (hasn't it?).

So, why is the memory not being released? And how can we get the CLR to release the memory?

Any explanations or solutions would be greatly appreciated.

EDIT: Following replies, particularly Beska's, it is obvious my description of the issue is not the the clearest, so I will try to clarify.

The code may not be the best example, sorry! It was a quick crude piece of code to try to replicate the issue.

The dictionary is used here to replicate the fact we have a large custom data object, which fills a large chunk of our memory and it is not then released after the task has completed.

In the example, the dictionary fills up to the limit of the dictionary and then throws an exception, it does NOT keep filling forever! This is well before our memory is full, and it does not cause an OutOfMemoryException. Hence the result is a large object in memory, and then the task completes.

At this point we would expect the dictionary to be out of scope, as both the task and the method 'Method' have completed. Hence, we would expect the dictionary to be garbage collected and the memory reclaimed. In reality, the memory is not freed until 'Method' is called again, creating a new WasteOfMemory instance and starting a new task.

Hopefully that will clarify the issue a bit

Solution 1

Okay, I've been following this...I think there are a couple issues, some of which people have touched on, but I think not answering the real question (which, admittedly, took me a while to recognize, and I'm not sure I'm answering what you want even now.)

This is all as expected, although what we cannot currently work out is when this memory should be released from the CLR.

As others have said, while the task is running, the dictionary will not be released. It's being used. It gets bigger until you run out of memory. I'm pretty sure you understand this.

We have let the task complete and then simulated a memory overload situation, but the memory consumed by the dictionary is not released. As the OS is running out of memory, is it not putting pressure on the CLR to release the memory?

Here, I think, is the real question.

If I understand you correctly, you're saying you set this up to fill up memory. And then, after it crashes (but before you hit return to start a new task) you're trying other things outside of this program, such as running other programs in Windows to try to get the GC to collect the memory, right? Hoping that the OS would talk to the GC, and start pressuring it to do it's thing.

However and even more confusing, if we wait until the task has completed, then hit enter to run the task again the memory is released, so obviously the previous dictionary has been garbage collected (hasn't it?).

I think you answered your own question...it has not been necessarily been released until you hit return to start a new task. The new task needs memory, so it goes to the GC, and the GC happily collects the memory from the previous task, which has now ended (after throwing from full memory).

So, why is the memory not being released? And how can we get the CLR to release the memory?

I don't know that you can force the GC to release memory. Generally speaking, it does it when it wants (though some hacker types might know some slick way to force its hand.) Of course, .NET decides when to run the GC, and since nothing is happening while the program is just sitting there, it may well be deciding that it doesn't need to. As to whether the OS can pressure the GC to run, it seems from your tests the answer is "no". A bit counter-intuitive perhaps.

Is that what you were trying to get at?

Solution 2

The garbage collector only frees locations in memory that are no longer in use that are objects which have no pointer pointing to them.

(1) your program runs infinitely without termination and

(2) you never change the pointer to your dictionary, so the GC has certainly no reason to touch the dictionary.

So for me your program is doing exactly what it is supposed to do.

Solution 3

The memory is not being released because the scope aMassiveList is never finished. When a function returns, it releases all non-referenced resources created inside it.

In your case, aMassiveList never leaves context. If you want your function never to return you have to find a way to 'process' your info and release it instead of storing all of them forever.

If you create a function that increasingly allocates resources and never release it you will end up consuming all the memory.

Solution 4

GC will only release unreferenced objects, so as the dictionary is being referenced by your program it can't be released by the GC

View more solutions

Comments

[Daniel A. White]

where did the handle come from?

[aroon65]

You're also disposing WasteOfMemory before the task has completed. The task is running asynchronously, so the correct way to dispose (given this design) would be with a continuation.

[DanGordon]

I think you are on the right track, this is more what I was trying to get to the bottom of and it seems a bit like there may be nothing we can do. I will edit the question above to make it clearer and reassess based on the replies so far.

[DanGordon]

Chosen as answer as whilst it did not get to the root of the issue elegantly explained what is happening and confirmed my suspicion that the GC seems to just collect when it wants to. It was possible to "force" the GC into a clean up by clearing the dictionary, setting it to null and calling GC collect as seen in this question: stackoverflow.com/questions/823661/…