Estimating maximum safe JVM heap size in 64-bit Java

Question

In the course of profiling a 64-bit Java app that's having some issues, I notice that the profiler itself (YourKit) is using truly colossal amounts of memory. What I've got in the YourKit launch script is:

JAVA_HEAP_LIMIT="-Xmx3072m -XX:PermSize=256m -XX:MaxPermSize=768m"

Naively, assuming some overhead, this would lead me to guess that YourKit is going to use a max of something maybe a bit over four GB. However, what I actually see in PS is:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
dmoles   31379  4.4 68.2 14440032 8321396 ?    Sl   11:47  10:42 java -Xmx3072m -XX:PermSize=256m -XX:MaxPermSize=768m -XX:+HeapDumpOnOutOfMemoryError -Dyjp.probe.table.length.limit=20000 -Xbootclasspath/a:/home/dmoles/Applications/yjp-9.5.6/bin/../lib/tools.jar -jar /home/dmoles/Applications/yjp-9.5.6/bin/../lib/yjp.jar

That's a virtual size of nearly 14 GB and a resident size of nearly 8 GB -- nearly 3x the Java heap.

Now, I've got enough memory on my dev box to run this, but going back to the original memory problem I'm trying to diagnose: How do I know how much Java heap I have to play with?

Clearly, if the customer has, say, 16 GB physical RAM, it's not a great idea for me to tell them to set -Xmx to 16 GB.

So what is a reasonable number? 12 GB? 8 GB?

And how do I estimate it?

Solution

Clearly, if the customer has, say, 16 GB physical RAM, it's not a great idea for me to tell them to set -Xmx to 16 GB.

If the customer was running nothing else significant on his/her machine, then setting the heap size to 16G isn't necessarily a bad idea. It depends on what the application is doing.

So what is a reasonable number? 12 GB? 8 GB?

The ideal number would be to have "JVM max heap + JVM non-heap overheads + OS + other active applications' working sets + buffer cache working set" add up to the amount of physical memory. But the problem is that none of those components (apart from the max heap size) can be pinned down without detailed measurements on the customer's machine ... while the application is running on the actual problem.

And how do I estimate it?

The bottom line is that you can't. The best you can do is to guess ... and be conservative.

An alternative approach is to estimate how much heap the application actually needs for the problem it is trying to solve. Then add an extra 50 or 100 percent to give the GC room to work efficiently. (And then tune ...)

Comments

[Adam Mihalcin]

+1 for saying that it's a good idea to estimate how much heap the application needs and go from there. The "how do I estimate it" question reminds me of Eric Lippert's post about the tragedy of thread happiness.

[prashant]

@AdamMihalcin It's good advice, but there's two sides to the coin. Heads (estimating how much the application needs) is, "here's how many users we expect; how much server should we buy?" Tails is, "here's how much server we have; how many users should we expect to be able to handle before it runs out of memory?"

[prashant]

So Stephen, when you say "add an extra 50 or 100 percent to give the GC room to work efficiently", do you mean add it to the JVM heap size, or do you mean have that much free RAM? I guess it's the "JVM non-heap overheads" that I need to understand here.

[Stephen C]

@DavidMoles - 1) Estimate / work out how much memory the application needs to run. 2) Multiply that number by 1.5 or 2.0 and use that as the heap size. 3) Tune. And 4) Acknowledge openly to everyone that this is all guesswork ... because that is what it is.