Communication between local JVMs

Question

My question: What approach could/should I take to communicate between two or more JVM instances that are running locally?

Some description of the problem:
I am developing a system for a project that requires separate JVM instances to isolate certain tasks from each other entirely.

In it's running, the 'parent' JVM will create 'child' JVMs that it will expect to execute and then return results to it (in the format of relatively simple POJO classes, or perhaps structured XML data). These results should not be transferred using the SysErr/SysOut/SysIn pipes as the child may already use these as part of its running.

If a child JVM does not respond with results within a certain time, the parent JVM should be able to signal to the child to cease processing, or to kill the child process. Otherwise, the child JVM should exit normally at the end of completing its task.

Research so far:
I am aware there are a number of technologies that may be of use e.g....

• Using Java's RMI library
• Using sockets to transfer objects
• Using distribution libraries such as Cajo, Hessian

...but am interested in hearing what approaches others may consider before pursuing one of these options, or any others.

Thanks for any help or advice on this!

Edits:
Quantity of data to transfer- relatively small, it will mostly be just a handful of POJOs containing strings that will represent the result of the child executing. If any solution would be inefficient on larger amounts of information, this is unlikely to be a problem in my system. The amount being transferred should be pretty static and so this does not have to be scalable.

Latency of transfer- not a critical concern in this case, although if any 'polling' of results is needed this should be able to be fairly frequent without significant overheads, so I can maintain a responsive GUI on top of this at a later time (e.g. progress bar)

Solution 1

I'd use KryoNet with local sockets since it specialises heavily in serialisation and is quite lightweight (you also get Remote Method Invocation! I'm using it right now), but disable the socket disconnection timeout.

RMI basically works on the principle that you have a remote type and that the remote type implements an interface. This interface is shared. On your local machine, you bind the interface via the RMI library to code 'injected' in-memory from the RMI library, the result being that you have something that satisfies the interface but is able to communicate with the remote object.

Solution 2

Not directly an answer to your question, but a suggestion of an alternative. Have you considered OSGI?

It lets you run java projects in complete isolation from each other, within the SAME jvm. The beauty of it is that communication between projects is very easy with services (see Core Specifications PDF page 123). This way there is not "serialization" of any sort being done as the data and calls are all in the same jvm.

Furthermore all your requirements of quality of service (response time etc...) go away - you only have to worry about whether the service is UP or DOWN at the time you want to use it. And for that you have a really nice specification that does that for you called Declarative Services (See Enterprise Spec PDF page 141)

Sorry for the off-topic answer, but I thought some other people might consider this as an alternative.

Update

To answer your question about security, I have never considered such a scenario. I don't believe there is a way to enforce "memory" usage within OSGI.

However there is a way of communicating outside of JVM between different OSGI runtimes. It is called Remote Services (see Enterprise Spec PDF, page 7). They also have nice discussion there of the factors to take into consideration when doing something like that (see 13.1 Fallacies).

Folks at Apache Felix (implementation of OSGI) I think have implementation of this with iPOJO, called Distributed Services with iPOJO (their wrapper to make using services easier). I've never used this - so ignore me if I am wrong.

Solution 3

akka is another option, as well as other java actor frameworks, it provides communication and other goodies derived from the actor model.

Solution 4

If you can't use stdin/stdout, then i'd go with sockets. You need some sort of serialization layer on top of the sockets (as you would with stdin/stdout), and RMI is a very easy to use and pretty effective such layer.

If you used RMI and found the performance wasn't good enough, i'd switch to some more efficient serializer - there are plenty of options.

I wouldn't go anywhere near web services or XML. That seems like a complete waste of time, likely take more effort and deliver less performance than RMI.

Solution 5

It was mentioned above, but i wanted to expand a bit on the JMX suggestion. we actually are doing pretty much exactly what you are planning to do (from what i can glean from your various comments). we landed on using jmx for a variety of reasons, a few of which i'll mention here. for one thing, jmx is all about management, so in general it is a perfect fit for what you want to do (especially if you already plan on having jmx services for other management tasks). any effort you put into jmx interfaces will do double duty as apis you can call using java management tools like jvisualvm. this leads to my next point, which is the most relevant to what you want. the new Attach API in jdk 6 and above is very sweet. it enables you to dynamically discover and communicate with running jvms. this allows, for example, for your "controller" process to crash and restart and re-find all the existing worker processes. this is the makings of a very robust system. it was mentioned above that jmx basically rmi under the hood, however, unlike using rmi directly, you don't need to manage all the connection details (e.g. dealing with unique ports, discoverability, etc). the attach api is a bit of a hidden gem in the jdk, as it isn't very well documented. when i was poking into this stuff initially, i didn't know the name of the api, so figuring how the "magic" in jvisualvm and jconsole worked was very difficult. finally, i came across an article like this one, which shows how to actually use the attach api dynamically in your own program.

View more solutions

Comments

[Vishy]

Perhaps you could talk about how much data you need to transfer and how latency critical it is? Unless you have significant performance constraints, whatever you consider to be the simplest solution may be best.

[obfuscation]

@Peter Lawrey I have added some extra detail for you, thanks!

[Artem]

He asked about JVM's, not class loaders.

[obfuscation]

thanks for the suggestion but as @bmargulies points out, I do really need separate JVMs due a number of concerns

[obfuscation]

Out of interest, where would your preference lie? I appreciate this might be a little difficult to say when you only have my short description of the project above.

[obfuscation]

That looks very interesting, and may indeed fulfil my needs. Are there are aspects you find it is lacking it at all, even if these don't currently cause you any issues?

[obfuscation]

No need for the apologies, this is an interesting suggestion. One of the problems I face is that part of the code executed by the child JVM is untrusted, and so has to be restricted (e.g. through a security manager). As part of this, I need to restrict the memory usage to ensure the parent JVM has sufficient resources remain available. However, from what I can find I cannot restrict the memory usage within a JVM to prevent out of memory exceptions. Does OSGI offer any protection/regulation of this?

[obfuscation]

Thank you, I believe this is the direction I was leaning towards. As the amount of data being transferred is relatively small, I believe I shouldn't face any real performance issues but I will keep a note of your link in case this does become an issue.

[Chris Dennett]

Some of the issues are to do with registration of networked classes, and race conditions to do with initial registration.. but these can be overcome by making sure that method calls are done in the right order. Also, I've found that it saturates the connection sometimes when connecting (I'm running on a network simulator interface), and some of the errors it gives are confusing and hard to pin down to the actual cause.

[obfuscation]

I've bookmarked this so I can have a more thorough look over it, thanks. Do you have experience in using this, and if so were there any real difficulties that you faced?

[Chris Dennett]

Another thing: it's hard to get the connection from which a method is being invoked, but I'm modding kryonet and adding in that functionality. If I just let clients lie about what connection they're coming from, all havoc might break loose.

[Chris Dennett]

I just want to say how small source code code that enables RMI is here -- it's essentially 600 lines of code. Hardly anything. But it works really well.

[Chris Dennett]

Another thing: it has no support for one-to-many RMI calls (so if you call a remote method, it calls the same method on 100 remote objects).

[Chris Dennett]

A good thing is that Kryo is a serialisation library, so it's hugely flexible. You can write custom serialisers for classes and such.

[Chris Dennett]

I tried Protobuf with sockets, it's incredibly hard to get working properly since you need to be able to redirect the output to the right classes once receiving, but to do that you need to read fields from the buffer. You can use CodedInputStream, but it's not nice. Better to use beans if you can and serialise those.

[obfuscation]

Netty definitely seems to be very highly held, but it seems to me that its speciality is in slightly different problems. Have you ever attempted to use Netty in this way? Also, it seems that it may be slightly heavy-weight for my needs- from your experience, does it take significant investment to learn/use?

[obfuscation]

Thanks very much for all of that expansion- it was much more than I was expecting! Interesting that in general this question has attracted a lot of attention, even though I couldn't find many similar questions when searching prior to posting it! For my purposes, I need to be able to identify the clients to know what the result corresponds to but I can trust the client to report this accurately, so I believe this should be trivial?

[Chris Dennett]

Yes, this will be trivial for you -- just send the address or client ID as a field -- basically just add an extra parameter to the method in class and interface :)

[zengr]

not much, I am also evaluating this for a project I am doing. Not much experience for now. You can mail on their mailing list. More over, its used by jBoss for cluster membership.

[obfuscation]

Exactly what I was thinking of doing. Luckily, whilst I cannot trust some of the code being run by client (and so must protect my main process against issues that seem difficult to otherwise handle such as OutOfMemory errors), I believe I will be able to isolate that from the communication to the parent :)

[mikera]

It does take a bit of setting up - though once you get it working it's rock solid. If you're really looking for something more lightweight I'd probably recommend KryoNet

[Ron]

If your application does not need to be robust and is not going to see significant load, it might be appropriate. I've used JGroups at two jobs-- in both cases it was the status quo and in both cases we migrated to other solutions. I have found it to be a thread hog and a memory hog, and its infinite configurability makes it impossible to sanely configure.

[zengr]

@Ron Can you please elaborate your answer. Also, you might answer one part of this question: stackoverflow.com/questions/4980677/… (i dont have any decent answer yet anyway)

[Ron]

if: (1) gc pauses are infrequent and unmeasurably short; (2) you don't mind the infinitely configurable, infinitesimally understandable jgroups configurations; (3) you don't mind the mess of threads and buffers and caches the jgroups runtime spins up within your jvm, then jgroups is a fine choice.

[Dead Programmer]

Attach api works only with child process id . once you spawn the child process, you need to get the process id.

[jtahlborn]

@SureshSankar - you don't need to know the pid to attach to the process, but it does help.

[Dead Programmer]

if you do not know child process id, how are you gonna control or invoke methods in child process. let me know how to connect to child process without process id.

[jtahlborn]

@SureshSankar - as i said in my answer, the attach api allows you to dynamically discover other java processes. if you have a way of identifying your child processes, you can find them without knowing their pid (this is how our product works).