C# : Out of Memory exception

Question

Today my application threw an OutOfMemoryException. To me this was always almost impossible since I have 4GB RAM and a lot of virtual memory too. The error happened when I tried to add an existing collection to a new list.

List<Vehicle> vList = new List<Vehicle>(selectedVehicles);  

To my understanding there isn't much memory allocated here since the vehicles my new list should contain already exist inside the memory. I have to admit Vehicle is a very complex class and I tried to add about 50.000 items to the new list at once. But since all Vehicles in the application come from a database that is only 200MB in size, I have no idea what may cause an OutOfMemoryException at this point.

Solution 1

Two points:

1. If you are running a 32 bit Windows, you won't have all the 4GB accessible, only 2GB.
2. Don't forget that the underlying implementation of List is an array. If your memory is heavily fragmented, there may not be enough contiguous space to allocate your List, even though in total you have plenty of free memory.

Solution 2

3 years old topic, but I found another working solution. If you're sure you have enough free memory, running 64 bit OS and still getting exceptions, go to Project properties -> Build tab and be sure to set x64 as a Platform target.

Solution 3

.Net4.5 does not have a 2GB limitation for objects any more. Add this lines to App.config

<runtime>
    <gcAllowVeryLargeObjects enabled="true" />    
</runtime>

and it will be possible to create very large objects without getting OutOfMemoryException

Please note it will work only on x64 OS's!

Solution 4

Data stored in database compared to memory in your application is very different.

There isn't a way to get the exact size of your object but you could do this:

GC.GetTotalMemory() 

After a certain amount of objects have been loaded and see how much your memory is changing as you load the list.

If it is the list that is causing the excessive memory usage then we can look at ways to minimize it. Such as why do you want 50,000 objects loaded into memory all at once in the first place. Wouldn't it be best to call the DB as you require them?

If you take a look here: http://www.dotnetperls.com/array-memory you will also see that objects in .NET are greater than their actual data. A generic list is even more of a memory hog than an array. If you have a generic list inside your object then it will grow even faster.

Solution 5

OutOfMemoryException (on 32-bit machines) is just as often about Fragmentation as actual hard limits on memory - you'll find lots about this, but here's my first google hit briefly discussing it: http://blogs.msdn.com/b/joshwil/archive/2005/08/10/450202.aspx. (@Anthony Pegram is referring to the same problem in his comment above).

That said, there is one other possibility that comes to mind for your code above: As you're using the "IEnumerable" constructor to the List, you may not giving the object any hints as to the size of the collection you're passing to the List constructor. If the object you are passing is is not a collection (does not implement the ICollection interface), then behind-the-scenes the List implementation is going to need to grow several (or many) times, each time leaving behind a too-small array that needs to be garbage collected. The garbage collector probably won't get to those discarded arrays fast enough, and you'll get your error.

The simplest fix for this would be to use the List(int capacity) constructor to tell the framework what backing array size to allocate (even if you're estimating and just guessing "50000" for example), and then use the AddRange(IEnumerable collection) method to actually populate your list.

So, simplest "Fix" if I'm right: replace

List<Vehicle> vList = new List<Vehicle>(selectedVehicles);

with

List<Vehicle> vList = new List<Vehicle>(50000);  
vList.AddRange(selectedVehicles);

All the other comments and answers still apply in terms of overall design decisions - but this might be a quick fix.

Note (as @Alex commented below), this is only an issue if selectedVehicles is not an ICollection.

View more solutions

Comments

[Cody Gray]

You only have 2 GB available in 64-bit Windows, too. This is a limitation of the .NET Framework, not just of the 32-bit address space.

[Henk Holterman]

@CodyGray That would be 2GB per object (array), not 2 GB total.

[Alex]

If selecyedVehicles is a collection, the constructor will allocate the correct array size. No need to go through AddRange.

[evolvedmicrobe]

@CodyGray, I think the 2GB array limit is not in the new .NET version.

[Michael Blankenship]

Nicely done. This worked for me, noting had to change the Build target to x64.

[Kiquenet]

applies too to ASP.NET 4.5 ? Using local Report, My App v1 (asp.net 3.5 - clr 2.0 - classic ) works OK, but my App v2 (asp.net 4.5, clr 4.0, classic) generate OutOfMemoryException error, in the same IIS server

[RBT]

One can overcome it using gcAllowVeryLargeObjects configuration setting. More details here - msdn.microsoft.com/en-us/library/…

[M.kazem Akhgary]

I can't see the image, what's the option?

[a'']

Project properties -> Build tab -> change Platform target to x64

[yu yang Jian]

I got Exception when memory go to 4.6G though setting this

[Wolfgang Grinfeld]

True enough, it is not intuitive what changing this flag can have as impact. But, the "prefer 32-bit" flag existed way before .Net 4.7.2.