java.lang.Error: "Not enough storage is available to process this command" when generating images

Question

I am running a web application on BEA Weblogic 9.2. Until recently, we were using JDK 1.5.0_04, with JAI 1.1.2_01 and Image IO 1.1. In some circumstances (we never figured out exactly why), when we were processing large images (but not that large - a few MB), the JVM would crash without any error message or stack trace or anything. This didn't happen much in production, but enough to be a nuisance and eventually we were able to reproduce it.

We decided to switch to JRockit90 1.5.0_04 and we were no longer able to reproduce the problem in our test environment, so we thought we had it licked. Now, however, after the application server has been up for a while, we start getting the error message, "Not enough storage is available to process this command" during image operations. For example:

java.lang.Error: Error starting thread: Not enough storage is available to process this command.
at java.lang.Thread.start()V(Unknown Source)
at sun.awt.image.ImageFetcher$1.run(ImageFetcher.java:279)
at sun.awt.image.ImageFetcher.createFetchers(ImageFetcher.java:272)
at sun.awt.image.ImageFetcher.add(ImageFetcher.java:55)
at sun.awt.image.InputStreamImageSource.startProduction(InputStreamImageSource.java:149)
at sun.awt.image.InputStreamImageSource.addConsumer(InputStreamImageSource.java:106)
at sun.awt.image.InputStreamImageSource.startProduction(InputStreamImageSource.java:144)
at sun.awt.image.ImageRepresentation.startProduction(ImageRepresentation.java:647)
at sun.awt.image.ImageRepresentation.prepare(ImageRepresentation.java:684)
at sun.awt.SunToolkit.prepareImage(SunToolkit.java:734)
at java.awt.Component.prepareImage(Component.java:3073)
at java.awt.ImageMediaEntry.startLoad(MediaTracker.java:906)
at java.awt.MediaEntry.getStatus(MediaTracker.java:851)
at java.awt.ImageMediaEntry.getStatus(MediaTracker.java:902)
at java.awt.MediaTracker.statusAll(MediaTracker.java:454)
at java.awt.MediaTracker.waitForAll(MediaTracker.java:405)
at java.awt.MediaTracker.waitForAll(MediaTracker.java:375)
at SfxNET.System.Drawing.ImageLoader.loadImage(Ljava.awt.Image;)Ljava.awt.image.BufferedImage;(Unknown Source)
at SfxNET.System.Drawing.ImageLoader.loadImage(Ljava.net.URL;)Ljava.awt.image.BufferedImage;(Unknown Source)
at Resources.Tools.Commands.W$zw(Ljava.lang.ClassLoader;)V(Unknown Source)
at Resources.Tools.Commands.getContents()[[Ljava.lang.Object;(Unknown Source)
at SfxNET.sfxUtils.SfxResourceBundle.handleGetObject(Ljava.lang.String;)Ljava.lang.Object;(Unknown Source)
at java.util.ResourceBundle.getObject(ResourceBundle.java:320)
at SoftwareFX.internal.ChartFX.wxvw.yxWW(Ljava.lang.String;Z)Ljava.lang.Object;(Unknown Source)
at SoftwareFX.internal.ChartFX.wxvw.vxWW(Ljava.lang.String;)Ljava.lang.Object;(Unknown Source)
at SoftwareFX.internal.ChartFX.CommandBar.YWww(LSoftwareFX.internal.ChartFX.wxvw;IIII)V(Unknown Source)
at SoftwareFX.internal.ChartFX.Internet.Server.xxvw.YzzW(LSoftwareFX.internal.ChartFX.Internet.Server.ChartCore;Z)LSoftwareFX.internal.ChartFX.CommandBar;(Unknown Source)
at SoftwareFX.internal.ChartFX.Internet.Server.xxvw.XzzW(LSoftwareFX.internal.ChartFX.Internet.Server.ChartCore;)V(Unknown Source)
at SoftwareFX.internal.ChartFX.Internet.Server.ChartCore.OnDeserialization(Ljava.lang.Object;)V(Unknown Source)
at SoftwareFX.internal.ChartFX.Internet.Server.ChartCore.Zvvz(LSoftwareFX.internal.ChartFX.Base.wzzy;)V(Unknown Source)

Has anyone seen something like this before? Any clue what might be happening?

Solution

Marcus Adams should get the credit for this, but his little advice was in the form of a comment not an answer, so I can't just check it. He pointed me to this example from another answer and that did the trick.

Looking at the registry setting for \System\CurrentControlSet\Control\Session Manager\SubSystem and found this: SharedSection=1024,3072,512. Since this was a service (headless) we changed the third number. The new value was SharedSection=1024,3072,1024. The problem has not recurred since making this change a few weeks ago.

Comments

[Marcus Adams]

Also see stackoverflow.com/questions/507853/… and stackoverflow.com/questions/548971/…

[jhericks]

Interesting. It is running on 32-bit Windows Server 2003 R2 (Standard). The machine has 3.83 GB or RAM installed. Looking at the registry setting for \System\CurrentControlSet\Control\Session Manager\SubSystem and found this: SharedSection=1024,3072,512. The question you referenced suggested changing the middle number. Is that still the correct thing to change for a headless application running as a service?

[jhericks]

We still can't find the problem, but we're going to be adjusting the 3rd number (512) to 1024 and see if that helps the situation.

[jhericks]

Is that bug likely to exist in JRockit as well? If so, will the fix delivered in tiger-b05 likely apply to an updated JRockit?

[Jeff Storey]

Not sure. But if this is your issue, how many threads are you creating? Can you scale that back?

[jhericks]

We are not really creating the threads ourselves - WebLogic is. We used to tune the thread count, but since 9.0, WebLogic "auto-tunes" the thread pool, but I'm sure there's a way to scale that down.

[Jeff Storey]

It would be interesting to monitor thread counts using a tool like VisualVM and see how many are live to see if it is thread related.

[jhericks]

I'll look into it. VisualVM only works with JDK 6. Is there a similar (free) tool that you know of that will work with JDK 5?

[Ravinder Reddy]

oh great ... headless service needed a memory boost.