Java unit testing: how to measure memory footprint for method call

Question

Assuming I have a class that does some heavy processing, operating with several collections. What I want to do is to make sure that such operation can't lead to out-of-memory or even better I want to set a threshold of how much memory it can use.

class MyClass()
{
   public void myMethod()
   {
      for(int i=0; i<10000000; i++)
      {
         // Allocate some memory, may be several collections
      }
   }
}

class MyClassTest
{
   @Test
   public void myMethod_makeSureMemoryFootprintIsNotBiggerThanMax()
   {
      new MyClass().myMethod(); 
      // How do I measure amount of memory it may try to allocate?
   }
}

What is the right approach to do this? Or this is not possible/not feasible?

Solution 1

I can think of several options:

• Finding out how much memory your method requires via a microbenchmark (i.e. jmh).
• Building allocation strategies based on heuristic estimation. There are several open source solutions implementing class size estimation i.e. ClassSize. A much easier way could be utilizing a cache which frees rarely used objects (i.e. Guava's Cache). As mentioned by @EnnoShioji, Guava's cache has memory-based eviction policies.

You can also write your own benchmark test which counts memory. The idea is to

1. Have a single thread running.
2. Create a new array to store your objects to allocate. So these objects won't be collected during GC run.
3. System.gc(), memoryBefore = runtime.totalMemory() - runtime.freeMemory()
4. Allocate your objects. Put them into the array.
5. System.gc(), memoryAfter = runtime.totalMemory() - runtime.freeMemory()

This is a technique I used in my lightweight micro-benchmark tool which is capable of measuring memory allocation with byte-precision.

Solution 2

You can use profiler (for ex. JProfiler) for view memory usage by classes. Or , how mentioned Areo, just print memory usage:

    Runtime runtime = Runtime.getRuntime();
    long usedMemoryBefore = runtime.totalMemory() - runtime.freeMemory();
    System.out.println("Used Memory before" + usedMemoryBefore);
        // working code here
    long usedMemoryAfter = runtime.totalMemory() - runtime.freeMemory();
    System.out.println("Memory increased:" + (usedMemoryAfter-usedMemoryBefore));

Solution 3

To measure current memory usage use :

Runtime.getRuntime().freeMemory() , Runtime.getRuntime().totalMemory()

Here is a good example: get OS-level system information

But this measurement is not precise but it can give you much information. Another problem is with GC which is unpredictable.

Solution 4

Here is an example from Netty which does something similar: MemoryAwareThreadPoolExecutor. Guava's cache class has also a size based eviction. You could look at these sources and copy what they are doing. In particular, Here is how Netty is estimating object sizes. In essence, you'd estimate the size of the objects you generate in the method and keep a count.

Getting overall memory information (like how much heap is available/used) will help you decide how much memory usage to allocate to the method, but not to track how much memory was used by the individual method calls.

Having said that, it's very rare that you legitimately need this. In most cases, capping the memory usage by limiting how many objects can be there at a given point (e.g. by using a bounded queue) is good enough and is much, much simpler to implement.

Solution 5

This question is a bit tricky, due to the way in which Java can allocate a lot of short-lived objects during processing, which will subsequently be collected during garbage collection. In the accepted answer, we cannot say with any certainty that garbage collection has been run at any given time. Even if we introduce a loop structure, with multiple System.gc() calls, garbage collection might run in between our method calls.

A better way is to instead use some variation of what is suggested in https://cruftex.net/2017/03/28/The-6-Memory-Metrics-You-Should-Track-in-Your-Java-Benchmarks.html, where System.gc() is triggered but we also wait for the reported GC count to increase:

long getGcCount() {
    long sum = 0;
    for (GarbageCollectorMXBean b : ManagementFactory.getGarbageCollectorMXBeans()) {
        long count = b.getCollectionCount();
        if (count != -1) { sum += count; }
    }
    return sum;
}

long getReallyUsedMemory() {
    long before = getGcCount();
    System.gc();
    while (getGcCount() == before);
    return getCurrentlyAllocatedMemory();
}

long getCurrentlyAllocatedMemory() {
    final Runtime runtime = Runtime.getRuntime();
    return (runtime.totalMemory() - runtime.freeMemory()) / (1024 * 1024);
}

This still gives only an approximation of the memory actually allocated by your code at a given time, but the value is typically much closer to what one would usually be interested in.

View more solutions

Comments

[Enno Shioji]

@Steve P.: Getting overall memory usage won't allow you to tell for what the memory was used for.

[Sergey Makarov]

Yep, but I can set the requirement like "this algorithm must not consume more than 100KB of RAM, should not depend on size of data to process". Idea is to enforce this by creating explicit unit test.

[Gimby]

But why would you set such a requirement? Resources are cheap, Java is designed towards that fact. You shouldn't be worrying about memory consumption until you have an actual problem. Other than that, even if you find a way to measure memory, you still don't have a way to measure that you are interpreting the results correctly and you created a realistic working environment to produce realistic results. You'll just get "a number" and not be any wiser really.

[prash]

@Gimby What is the harm if some curious dev is trying to measure two approaches - does it have to be in realistic working env before making any sense - I guess not. Some approaches in general take far less memory/CPU or both in any circumstance than other. Such analysis can come in handy to atleast get a rough idea.

[Alex R]

@Gimby "resources are cheap" is a rather presumptuous comment when you don't know the infrastructure and staffing budget for the solution

[Marc_Alx]

Small tip for further readers : values are given in bytes, divide by 1000000 to get values in MB.

[fabien]

the custom way is still an approximation though. even with running the GC before and after, there is a possibility for the young GC to be invoked in between those two calls (especially if the tested call is memory intensive), even if you allocate all the RAM as heap. also, there are different JVMs and different GCs (G1 is very different than mark & sweep for example) so I don't know how reliable results would be... I like the idea of a custom GC.If JMH can do it though, it's better to use it (as you say).

[Christian Grün]

In practice, it can be helpful to call System.gc() multiple times before allocating memory. Quite obviously, the exact behavior will also depend on the JVM and the GC.

[95_96]

@pasha701 small problem with using this approach, runtime.totalMemory() gives memory assigned to jvm at that instance. This may increased overtime after execution of working code. In that case (usedMemoryAfter-usedMemoryBefore) may even give out negative values in some scenarios.

[R.A]

This is not very accurate, as System.gc() is only a "request" and does not happen on the spot. You never know when gc really will run.

[luke]

@R.A Yes, you are right. I wrote about it in the answer. This method is simple and it doesn't require any extra tools, but unfortunately It might be not accurate at all. I had a case where It looked like It works, but It was a very simple application and my findings were based purely on my observations without having proof that it works accurately.

[MD. Mohiuddin Ahmed]

@Marc_Alx: would preceisely be 1024*1024 == 1048576