Java memory clean up

Question

We have created the custom test-sutomation framework for our project. This mimics the interaction (mainly json two way data flow in GET and POST request) between browser and server using HTTP client.

Here is hot it happens, We have some 1000 test cases automated. Test case execution involves creation of tonnes of messages and hence tonnes of strings; reading large files repeatedly and verifying the contents and off-course throwing everything after getting verification result. It goes for all the test cases, takes anywhere between 15 to 20 hours.

The problem is that, we are stuck with OutOfMemory Error. Any Idea?; on cleaning up the memory which is consumed after reading and parsing from large files 100MB - 250MB. There are lot of splits, replace, substring operations going on.

Please, Can anybody provide tips,suggestions?

-Thanks in advance

Solution 1

This error is thrown by the Java Virtual Machine (JVM) when an object cannot be allocated due to lack of memory space and also, the garbage collector cannot free some space. The OutOfMemoryError objects are created by the JVM when suppression is disabled and/ir the stack trace is not writable.

Solution for OutOfMemoryError: 1. The most obvious solution to this error is to increase the available memory size for the Java Virtual Machine. If your application requires more memory then, you shall grant it to your application.

2. Verify that your application does not store unnecessary information. Store and maintain only those pieces of information required for the proper execution of your Java application.

3. You can use the availabe memory analyzer tools, in order to carefully observe the portions of memory occupied by your application. Examples of such tools are the Eclipse Memory Analyzer(http://www.eclipse.org/mat/) and Java Heap Analysis Tool (jhat)(http://docs.oracle.com/javase/6/docs/technotes/tools/share/jhat.html).

4. Try to configure your JVM to use more memory as shown before (-Xms750m -Xmx2048m -XX:MaxPermSize=1024m ).

5. Enable Garbage Collection logging (-Xloggc:/var/log/YOUR_APP/YOUR_APP-gc.log) and see how it behaves, how heap is growing. Probably you have a memory leak.

6. If so, take a HeapDump, use YourKit to open it and look for objects that use the largest amount of memory. Try to figure out why and fix it.

7. You can call Garbage collector using:

System.gc();

When calling the Garbage collector method suggests that the Java Virtual Machine expend effort toward recycling unused objects in order to make the memory they currently occupy available for quick reuse. When control returns from the method call, the Java Virtual Machine has made a best effort to reclaim space from all discarded objects.

The JVM decides when to execute it. In general if the JVM is about to throw an OutOfMemoryError, calling System.gc() won't prevent it. Better investigate why you're leaking so much memory and clean it up along the way. But this does not mean that it'll be executed immediately.

Read more about JVM Garbage Collection Tuning SE 6 HotSpot for details on how to tune garbage collection with Java SE 6.

Hope this help you.

Solution 2

In general, you should make sure you maximize the objects available for garbage collection. One way to do this is declaring variables in the narrowest possible scope. (Eg: inside a try block, use method local variables)

You may also have to increase Java heap size.