Java Python Integration

Question

I have a Java app that needs to integrate with a 3rd party library. The library is written in Python, and I don't have any say over that. I'm trying to figure out the best way to integrate with it. I'm trying out JEPP (Java Embedded Python) - has anyone used that before? My other thought is to use JNI to communicate with the C bindings for Python.

Any thoughts on the best way to do this would be appreciated. Thanks.

Solution 1

Why not use Jython? The only downside I can immediately think of is if your library uses CPython native extensions.

EDIT: If you can use Jython now but think you may have problems with a later version of the library, I suggest you try to isolate the library from your app (e.g. some sort of adapter interface). Go with the simplest thing that works for the moment, then consider JNI/CPython/etc if and when you ever need to. There's little to be gained by going the (painful) JNI route unless you really have to.

Solution 2

Frankly most ways to somehow run Python directly from within JVM don't work. They are either not-quite-compatible (new release of your third party library can use python 2.6 features and will not work with Jython 2.5) or hacky (it will break with cryptic JVM stacktrace not really leading to solution).

My preferred way to integrate the two would use RPC. XML RPC is not a bad choice here, if you have moderate amounts of data. It is pretty well supported — Python has it in its standard library. Java libraries are also easy to find. Now depending on your setup either Java or Python part would be a server accepting connection from other language.

A less popular but worth considering alternative way to do RPCs is Google protobuffers, which have 2/3 of support for nice rpc. You just need to provide your transport layer. Not that much work and the convenience of writing is reasonable.

Another option is to write a C wrapper around that pieces of Python functionality that you need to expose to Java and use it via JVM native plugins. You can ease the pain by going with SWIG SWIG.

Essentially in your case it works like that:

1. Create a SWIG interface for all method calls from Java to C++.
2. Create C/C++ code that will receive your calls and internally call python interpreter with right params.
3. Convert response you get from python and send it via swig back to your Java code.

This solution is fairly complex, a bit of an overkill in most cases. Still it is worth doing if you (for some reason) cannot afford RPCs. RPC still would be my preferred choice, though.

Solution 3

Many years later, just to add an option which is more popular these days...

If you need CPython functionality, py4j is a good option. py4j has seen seen frequent updates in 2016 2017 2018 2019 2020 and has gained some popularity, because it is used e.g. by Apache Spark to achieve CPython interoperability.

Solution 4

The best solutions, is to use Python programs throw REST API. You define your services and call them. You perhaps need to learn some new modules. But you will be more flexible for futures changes.

Here a small list of use full modules for this purpose: Python modules

• Flask
• Flask-SQLAlchemy
• Flask-Restful
• SQlite3
• Jsonify

Java modules (for calling rest api) Jersey or Apache CXF

You will need a small Learning curve, but later you will get more productivity and modularity and even elasticity...

Solution 5

My other thought is to use JNI to communicate with the C bindings for Python.

I like very much JNA:

JNA provides Java programs easy access to native shared libraries (DLLs on Windows) without writing anything but Java code—no JNI or native code is required. This functionality is comparable to Windows' Platform/Invoke and Python's ctypes. Access is dynamic at runtime without code generation.

My 0.02$ :)

View more solutions