SocketException (0x80004005): An existing connection was forcibly closed by the remote host

Question

Hoping somebody can help with this exception. Our client app can spawn up to 3 server apps (new processes) and communicates with each through TCP. These in turn spawn a jvm when prompted (server app essentially a wrapper for communicating to jvm through shared memory). When I set the memory on each jvm to be about 25gb (I have 8gb ram) it will not run any of the 3 server applications and throws the above exception. When I set it be 10gb it will run one of them but throw that exception when I try start the second jvm. The exception appears to happen when I try and start the jvm (prints logging statement just before it makes first java call) however this exception is only captured on the client side. I'm wondering if this is a stackoverflow on the server side?

12:58:33 : Exception : System.Net.Sockets.SocketException (0x80004005): An existing connection was forcibly closed by the remote host
Server stack trace: 
at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at System.Runtime.Remoting.Channels.SocketStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Runtime.Remoting.Channels.SocketHandler.ReadFromSocket(Byte[] buffer, Int32 offset, Int32 count)
at System.Runtime.Remoting.Channels.SocketHandler.Read(Byte[] buffer, Int32 offset, Int32 count)
at System.Runtime.Remoting.Channels.SocketHandler.ReadAndMatchFourBytes(Byte[] buffer)
at System.Runtime.Remoting.Channels.Tcp.TcpSocketHandler.ReadVersionAndOperation(UInt16& operation)
at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]: 
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Isl.RemotableObjects.ServerMgmtIF.initialiseApp(String licenseSource, OceanPluginTypeEnum type)
at Isl.OceanShared.Server.initialise(String licenseSource)
12:58:33 : Stacktrace: 
Server stack trace: 
at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size,SocketFlags socketFlags)
at System.Runtime.Remoting.Channels.SocketStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Runtime.Remoting.Channels.SocketHandler.ReadFromSocket(Byte[] buffer, Int32 offset, Int32 count)
at System.Runtime.Remoting.Channels.SocketHandler.Read(Byte[] buffer, Int32 offset, Int32 count)
at System.Runtime.Remoting.Channels.SocketHandler.ReadAndMatchFourBytes(Byte[] buffer)
at System.Runtime.Remoting.Channels.Tcp.TcpSocketHandler.ReadVersionAndOperation(UInt16& operation)
at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]: 
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Isl.RemotableObjects.ServerMgmtIF.initialiseApp(String licenseSource, OceanPluginTypeEnum type)
at Isl.OceanShared.Server.initialise(String licenseSource)
12:58:33 : Message   : StackTrace: 
Server stack trace: 
at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at System.Runtime.Remoting.Channels.SocketStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Runtime.Remoting.Channels.SocketHandler.ReadFromSocket(Byte[] buffer, Int32 offset, Int32 count)
at System.Runtime.Remoting.Channels.SocketHandler.Read(Byte[] buffer, Int32 offset, Int32 count)
at System.Runtime.Remoting.Channels.SocketHandler.ReadAndMatchFourBytes(Byte[] buffer)
at System.Runtime.Remoting.Channels.Tcp.TcpSocketHandler.ReadVersionAndOperation(UInt16& operation)
at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]: 
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Isl.RemotableObjects.ServerMgmtIF.initialiseApp(String licenseSource, OceanPluginTypeEnum type)
at Isl.OceanShared.Server.initialise(String licenseSource)

Solution

This will happen when, as the exception implies, the remote host (server) closes the connection unexpectedly. This can be due to a multitude of reasons, but typically is due to an unhandled exception on the server side. Investigate the server code and go from there.

Comments

[user1584120]

I've put a try catch around the point where the server side is failing (determined by trace statements before and after failing line of code) but it doesn't catch anything so that's why I've assumed a stackoverflow. I'm not quite sure what happens when the jvm allocates its memory, whether it partitions it off when the jvm starts up or just uses it when needed? Judging by this scenario its partitioning it and throwing some kind of uncatchable exception. Would something like wireshark pick this up and give more information?

[Haney]

Java is different than C# of course, and I'm approaching this from the C# side with little knowledge of the java runtime... Last time I used it for work was 2008-2009... Anyway, I'd imagine it would log such crashes to the Event Viewer, or an internal app server log (I recall that GlassFish used to have a server log for example).