Java, Classpath, Classloading => Multiple Versions of the same jar/project

Question

I know this may be a silly question for experienced coders. But I have a library (an http client) that some of the other frameworks/jars used in my project require. But all of them require different major versions like:

httpclient-v1.jar => Required by cralwer.jar
httpclient-v2.jar => Required by restapi.jar
httpclient-v3.jar => required by foobar.jar

Is the classloader intelligent enough to seperate them somehow? Most likely not? How does the Classloader handle this, in case a Class is the same in all three jars. Which one is loaded and why?

Does the Classloader only pickup exactly one jar or does it mix classes arbitrarily? So for example if a class is loaded from Version-1.jar, all other classes loaded from the same classloader will all go into the same jar?

How do you handle this problem?

Is there some trick to somehow "incorporate" the jars into the "required.jar" so that the are seen as "one unit/package" by the Classloader, or somehow linked?

Solution 1

Classloader related problems are a quite complex matter. You should in any case keep in mind some facts:

• Classloaders in an application are usually more than a single one. The bootstrap class loader delegates to the appropriate. When you instantiate a new class the more specific classloader is invoked. If it does not find a reference to the class you are trying to load, it delegates to its parent, and so on, until you get to the bootstrap class loader. If none of them find a reference to the class you are trying to load you get a ClassNotFoundException.

• If you have two classes with the same binary name, searchable by the same classloader, and you want to know which one of them you are loading, you can only inspect the way that specific classloader tries to resolve a class name.

• According to the java language specification, there is not a uniqueness constraint for a class binary name, but as far as I can see, it should be unique for each classloader.

I can figure out a way to load two classes with the same binary name, and it involves to have them loaded (and all their dependencies) by two different classloaders overriding default behaviour. A rough example:

    ClassLoader loaderA = new MyClassLoader(libPathOne);
    ClassLoader loaderB = new MyClassLoader(libPathTwo);
    Object1 obj1 = loaderA.loadClass("first.class.binary.name", true)
    Object2 obj2 = loaderB.loadClass("second.class.binary.name", true);

I always found classloader customization a tricky task. I'd rather suggest to avoid multiple incompatible dependencies if possible.

Solution 2

Each classload picks exactly one class. Usually the first one found.

OSGi aims to solve the problem of multiple versions of the same jar. Equinox and Apache Felix are the common open-source implementations for OSGi.

Solution 3

Classloader will load classes from the jar that happened to be in the classpath first. Normally, incompatible versions of library will have difference in packages, But in unlikely case they really incompatible and can't be replaced with one - try jarjar.

Solution 4

Classloaders load class on demand. This means that the class required first by your application and related libraries would be loaded before other classes; the request to load the dependent classes is typically issued during the loading and linking process of a depending class.

You are likely to encounter LinkageErrors stating that duplicate class definitions have been encountered for classloaders typically do not attempt to determine which class should be loaded first (if there are two or more classes of the same name present in the classpath of the loader). Sometimes, the classloader will load the first class occurring in the classpath and ignore the duplicate classes, but this depends on the implementation of the loader.

The recommended practice to resolve such kind of errors is to utilize a separate classloader for each set of libraries that have conflicting dependencies. That way, if a classloader attempts to load classes from a library, the dependent classes would be loaded by the same classloader that does not have access to the other libraries and dependencies.

Solution 5

You can use the URLClassLoader for require to load the classes from a diff-2 version of jars:

URLClassLoader loader1 = new URLClassLoader(new URL[] {new File("httpclient-v1.jar").toURL()}, Thread.currentThread().getContextClassLoader());
URLClassLoader loader2 = new URLClassLoader(new URL[] {new File("httpclient-v2.jar").toURL()}, Thread.currentThread().getContextClassLoader());

Class<?> c1 = loader1.loadClass("com.abc.Hello");

Class<?> c2 = loader2.loadClass("com.abc.Hello");

BaseInterface i1 = (BaseInterface) c1.newInstance();

BaseInterface i2 = (BaseInterface) c2.newInstance();

View more solutions

Comments

[deckingraj]

The bootstrap class loader delegates to the appropriate. When you instantiate a new class the more specific classloader is invoked. If it does not find a reference to the class you are trying to load, it delegates to its parent Please bear with me but it depends on the classloader policy which is by default Parent First. In other words child class will first ask its parent to load the class and will load only if the entire hierarchy failed to load it, no??

[Joe Kearney]

No - typically a classloader delegates to its parent before looking for the class itself. See the class javadoc for Classloader.

[rogerdpack]

I think tomcat does it in the way described here, but "conventional" delegation is to ask the parent first

[Luca Putzu]

@deckingraj: after some googling i found this from oracle docs: "In the delegation design, a class loader delegates classloading to its parent before attempting to load a class itself. [...] If the parent class loader cannot load a class, the class loader attempts to load the class itself. In effect, a class loader is responsible for loading only the classes not available to the parent". I will investigate further. If this will emerge as the default implementation i will update the response accordingly. (docs.oracle.com/cd/E19501-01/819-3659/beadf/index.html)

[Will Hughes]

Just to be clear, a custom classloader can do whatever it likes. It might have the parent check first, or it might do it itself first, or it might sys.print an emoji. If you writting the loader, you're overriding the method, there's no guarantee what happens.