Memory allocation for Java Webstart Application

Question

I'm using a Java webstart application called BLAST2GO I'm running into problems with memory allocation. It needs to load a large data set into memory, so I've been trying to allocate 3-6GB to this application. However, no matter what I set as the max heap size, it always uses 455MB (as indicated by the message bar: 'Memory usage: xxMB of 455MB'). I've tried setting it to very reasonable levels (1024 MB), but still only 455MB are allocation.

Any ideas?

BTW, I'm using Ubuntu 10.04 32-bit with 24GB memory, 8 procs.

Update: Here is the JNLP they provide. I get the same result when I edit the values of the initial-heap-size and max-heap-size attributes.

<jnlp spec="1.0+" codebase="http://bioinfo.cipf.es/blast2go/webstart" href="http://bioinfo.cipf.es/blast2go/webstart/makeJnlp.php?mem=3072">
<information>
<title>Blast2GO 3072M</title>
<vendor>CIPF - UPV - IVIA</vendor>
<homepage href="http://www.blast2go.org"/>
<description>Blast2GO 3072M</description>
<description kind="short">Blast2GO 3072M</description>
<icon href="splash.gif"/>
<offline-allowed/>
</information>
<security>
<all-permissions/>
</security>
<resources>
<j2se version="1.5+" initial-heap-size="64m" max-heap-size="3072M"/>
<jar href="ext/blast2go.jar"/>
<jar href="ext/biojava-1.3.1.jar"/>
<jar href="ext/jfreechart-1.0.10.jar"/>
<jar href="ext/mysql-connector-java-3.0.11-stable-bin.jar"/>
<jar href="ext/zvtm.jar"/>
<jar href="ext/jakarta-regexp-1.4.jar"/>
<jar href="ext/jcommon-1.0.13.jar"/>
<jar href="ext/Simpat1.1.jar"/>
<jar href="ext/iText-2.1.1.jar"/>
<jar href="ext/jdom/activation.jar"/>
<jar href="ext/jdom/ant.jar"/>
<jar href="ext/jdom/jaxen-core.jar"/>
<jar href="ext/jdom/jaxen-jdom.jar"/>
<jar href="ext/jdom/jdom.jar"/>
<jar href="ext/jdom/mail.jar"/>
<jar href="ext/jdom/saxpath.jar"/>
<jar href="ext/jdom/xalan.jar"/>
<jar href="ext/jdom/xerces.jar"/>
<jar href="ext/jdom/xml-apis.jar"/>
<jar href="ext/interpro/commons-cli-1.0.jar"/>
<jar href="ext/interpro/commons-cli.jar"/>
<jar href="ext/interpro/freefluo.jar"/>
<jar href="ext/interpro/servlet.jar"/>
<jar href="ext/interpro/WSInterProScan.jar"/>
<jar href="ext/axis_1_4/axis-ant.jar"/>
<jar href="ext/axis_1_4/axis.jar"/>
<jar href="ext/axis_1_4/commons-discovery-0.2.jar"/>
<jar href="ext/axis_1_4/commons-logging-1.0.4.jar"/>
<jar href="ext/axis_1_4/jaxrpc.jar"/>
<jar href="ext/axis_1_4/keggapi.jar"/>
<jar href="ext/axis_1_4/log4j-1.2.8.jar"/>
<jar href="ext/axis_1_4/saaj.jar"/>
<jar href="ext/axis_1_4/wsdl4j-1.5.1.jar"/>
<jar href="ext/axis2/activation-1.1.jar"/>
<jar href="ext/axis2/annogen-0.1.0.jar"/>
<jar href="ext/axis2/axiom-api-1.2.2.jar"/>
<jar href="ext/axis2/axiom-dom-1.2.2.jar"/>
<jar href="ext/axis2/axiom-impl-1.2.2.jar"/>
<jar href="ext/axis2/axis2-adb-1.1.1.jar"/>
<jar href="ext/axis2/axis2-adb-codegen-1.1.1.jar"/>
<jar href="ext/axis2/axis2-codegen-1.1.1.jar"/>
<jar href="ext/axis2/axis2-java2wsdl-1.1.1.jar"/>
<jar href="ext/axis2/axis2-jaxbri-1.1.1.jar"/>
<jar href="ext/axis2/axis2-jibx-1.1.1.jar"/>
<jar href="ext/axis2/axis2-kernel-1.1.1.jar"/>
<jar href="ext/axis2/axis2-saaj-1.1.1.jar"/>
<jar href="ext/axis2/axis2-soapmonitor-1.1.1.jar"/>
<jar href="ext/axis2/axis2-spring-1.1.1.jar"/>
<jar href="ext/axis2/axis2-tools-1.1.1.jar"/>
<jar href="ext/axis2/axis2-xmlbeans-1.1.1.jar"/>
<jar href="ext/axis2/backport-util-concurrent-2.2.jar"/>
<jar href="ext/axis2/commons-codec-1.3.jar"/>
<jar href="ext/axis2/commons-fileupload-1.1.1.jar"/>
<jar href="ext/axis2/commons-httpclient-3.0.1.jar"/>
<jar href="ext/axis2/commons-io-1.2.jar"/>
<jar href="ext/axis2/commons-logging-1.1.jar"/>
<jar href="ext/axis2/geronimo-spec-jms-1.1-rc4.jar"/>
<jar href="ext/axis2/jakarta-httpcore-4.0-alpha2.jar"/>
<jar href="ext/axis2/jaxb-api-2.0.2.jar"/>
<jar href="ext/axis2/jaxb-impl-2.0.2.jar"/>
<jar href="ext/axis2/jaxb-xjc-2.0.2.jar"/>
<jar href="ext/axis2/jaxen-1.1-beta-10.jar"/>
<jar href="ext/axis2/jibx-bind-1.1.3.jar"/>
<jar href="ext/axis2/jibx-run-1.1.3.jar"/>
<jar href="ext/axis2/mail-1.4.jar"/>
<jar href="ext/axis2/neethi-2.0.jar"/>
<jar href="ext/axis2/servletapi-2.3.jar"/>
<jar href="ext/axis2/stax-api-1.0.1.jar"/>
<jar href="ext/axis2/woden-1.0.0M6.jar"/>
<jar href="ext/axis2/wsdl4j-1.6.2.jar"/>
<jar href="ext/axis2/wstx-asl-3.2.0.jar"/>
<jar href="ext/axis2/xalan-2.7.0.jar"/>
<jar href="ext/axis2/xbean-2.2.0.jar"/>
<jar href="ext/axis2/xercesImpl-2.8.1.jar"/>
<jar href="ext/axis2/xml-apis-1.3.03.jar"/>
<jar href="ext/axis2/XmlSchema-1.2.jar"/>
<jar href="ext/batik/batik-anim.jar"/>
<jar href="ext/batik/batik-awt-util.jar"/>
<jar href="ext/batik/batik-bridge.jar"/>
<jar href="ext/batik/batik-codec.jar"/>
<jar href="ext/batik/batik-css.jar"/>
<jar href="ext/batik/batik-dom.jar"/>
<jar href="ext/batik/batik-extension.jar"/>
<jar href="ext/batik/batik-ext.jar"/>
<jar href="ext/batik/batik-gui-util.jar"/>
<jar href="ext/batik/batik-gvt.jar"/>
<jar href="ext/batik/batik-parser.jar"/>
<jar href="ext/batik/batik-script.jar"/>
<jar href="ext/batik/batik-svg-dom.jar"/>
<jar href="ext/batik/batik-svggen.jar"/>
<jar href="ext/batik/batik-swing.jar"/>
<jar href="ext/batik/batik-transcoder.jar"/>
<jar href="ext/batik/batik-util.jar"/>
<jar href="ext/batik/batik-xml.jar"/>
<jar href="ext/batik/js.jar"/>
<jar href="ext/batik/pdf-transcoder.jar"/>
<jar href="ext/batik/xalan-2.6.0.jar"/>
<jar href="ext/batik/xerces_2_5_0.jar"/>
<jar href="ext/batik/xml-apis-ext.jar"/>
<jar href="ext/batik/xml-apis.jar"/>
</resources>
<application-desc main-class="es.blast2go.prog.MainProg"/>
</jnlp>

Update: I tried changing the href attribute in the root jnlp tag and I got a very strange message on the terminal. I wonder if Java is re-downloading the JNLP from this URL each time and ignoring my attempts to edit the JNLP. Is there a way I can prevent this?

> javaws blast2go3GB.jnlp 
Error occurred during initialization of VM
CCoouulldd  nnoott  crreesaetrev et heen oJuagvha  svpiarcteu aflo rm aocbhjiencet. 
heap

Final Update: Turns out this was a problem with the Java Version. I got the expected behavior when I downloaded and used Sun Java.

Solution 1

If you are running a 32-bit Linux, you can only run a 32-bit JVM which is limited to about 4 Gb memory per instance. I believe this is also the case even if you have more visible to the kernel.

If you really, really need a lot of memory in a single JVM you need to go 64-bit.

---

EDIT: Java WebStart was not open sourced for OpenJDK so an alternative implementation is used. Try to download and use Sun Java for this instead. It should be enough to download and unzip the JDK, and run bin/javaws foo.jnlp.

Solution 2

You can specify the heap size in the j2se tag like this:

    <resources>
    .........
            <j2se version="1.6.0+" href="" initial-heap-size="1024m" max-heap-size="3000m" />
            <j2se version="1.5.0_05" href="" initial-heap-size="1024m" max-heap-size="3000m" />
            <j2se version="1.4.2_04" href="" initial-heap-size="1024m" max-heap-size="3000m" />        
    .........
    </resources>

And specify every jre version you would use.

Solution 3

You can change it simply via command line option, e.g.

javaws -J-Xmx16384m -J-Xms16384m <jnlp url>

Solution 4

Try to set it by using a manual command line parameter. Something like this:

<j2se version="1.6+" java-vm-args="-Xms3072m"/>

Solution 5

Did you try the solution from the blast2go start page?

"Blast2GO needs more memory? Use the dynamic memory JNLP URL:"

http://bioinfo.cipf.es/blast2go/webstart/makeJnlp.php?mem=<put here the amount of memory you need>

It is not clear what is going on here, but I think it is likely that blast2go is doing some manual memory management tricks. (The fact that it claims to report how much memory is used is indicative of this.) If that is the case, then there may be application specific arguments or properties that need to be set if you tweak the JNLP file.

View more solutions

Comments

[matbrgz]

How do you indicate how much memory you want?

[Daniel Standage]

I use the link they provide and specify how much I want.

[matbrgz]

Just did the same: THe line is <j2se version="1.5+" initial-heap-size="64m" max-heap-size="2048M"/> for 2048 entered.

[matbrgz]

On my Mac (with 5 Gb memory and 64-bit JVM) the line is "Memory usage: 18MB of 2043MB". I would try on a 64-bit machine.

[Daniel Standage]

Yes, I've edited that line. Whatever I put for the initial heap size and max heap size, it's always 455MB.

[Enrique]

Im running Windows Vista 64-bits with 4GB RAM, a 32 bit JVM. I selected the 1500MB link and the status bar is "Memory usage: 29MB of 1450MB"

[Daniel Standage]

As far as I'm aware, I only have one jre on the system. The .jnlp file they created had only one j2se tag and it had version 1.5+. I'm running 1.6.0_18 on my system, so that should be covered.

[Daniel Standage]

That's great, but why is only 455MB allocated when I ask for, say, 1024MB?

[matbrgz]

You may do it wrong, and it is the default behaviour. Or it just gives you what it can. Hard to say - show some JNLP.

[Enrique]

@Daniel Have you tried to run your app without jnlp(from command line java -jar myapp.jar -Xms1024m -Xmx3000m) Do you have the same issue when running from command line?

[matbrgz]

Ok, JNLP comes from linked page. See comments to question.

[Daniel Standage]

The result is the same whether I edit the JNLP they provide or not. I'll post it though.

[matbrgz]

They just put the number in the right location of the JNLP. Nothing fancy.

[Daniel Standage]

That's what I've been using all along. No matter what I put for the amount of memory, I only get 455MB.

[Daniel Standage]

@Thorbjorn It was definitely the javaws version. I didn't know Ubuntu was not using Sun Java. I downloaded/unpacked the latest JDK. It gives me the memory I ask for when it's available and a helpful error messages when I try to allocate too much. Thanks!

[matbrgz]

@Daniel, you are welcome. What is the maximum amount of memory you can get?

[Nowaker]

Nice. Wouldn't think you use OpenJDK. :) BTW, you can enable a "partners" repo in Ubuntu and get sun-java from it. Don't need to download and unpack anything manually.