Reflections library not working when used in an Eclipse plug-in

Question

I have developed an application using the Reflections library for querying all the classes having a particular annotation. Everything was working like a charm until I decided to create an Eclipse plug-in from my application. Then Reflections stop working.

Given that my application is working fine when not part of an Eclipse plug-in, I think it should be a class-loader problem. So I added to my Reflections class the classloaders of the plug-in activator class, the context class loader, and all other class loaders I could imagine, without any success. This is a simplified version of my code:

ConfigurationBuilder config = new ConfigurationBuilder();
config.addClassLoaders(thePluginActivatorClassLoader);
config.addClassLoaders(ClasspathHelper.getContextClassLoader());
config.addClassLoaders("all the classloaders I could imagine");
config.filterInputsBy(new FilterBuilder().include("package I want to analyze"));

Reflections reflections = new Reflections(config);
Set<Class<?>> classes = reflections.getTypesAnnotatedWith(MyAnnotation.class); //this Set is empty

I also tried adding URLs of the classes I want to load to the ConfigurationBuilder class, but it did not help.

Could someone tell me if there is a way to make Reflections work as part of an Eclipse plug-in ?, or should I better look for another alternative ?. Thanks a lot, I am really puzzled about it.

Solution 1

I assume you already know how to create bundles (otherwise, check this).

After some debuging and exploration of Reflections API I have realised that the problem is that Reflections simply fails to read OSGi URLs (bundleresource://...) resulting in an exception:

org.reflections.ReflectionsException: could not create Vfs.Dir from url, 
no matching UrlType was found [bundleresource://1009.fwk651584550/]

and this suggestion:

either use fromURL(final URL url, final List<UrlType> urlTypes) 
or use the static setDefaultURLTypes(final List<UrlType> urlTypes) 
or addDefaultURLTypes(UrlType urlType) with your specialized UrlType.

So I believe implementing a UrlType for OSGi (e.g. class BundleUrlType implements UrlType {...}) and registering it like this:

Vfs.addDefaultURLTypes(new BundleUrlType());

should make Reflections API usable from inside a bundle. Reflections dependencies should be added to the Eclipse Plugin project as described here.

This is how my sample MANIFEST.MF looked like after adding needed jars:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: ReflectivePlugin
Bundle-SymbolicName: ReflectivePlugin
Bundle-Version: 1.0.0.qualifier
Bundle-Activator: reflectiveplugin.Activator
Bundle-ActivationPolicy: lazy
Bundle-RequiredExecutionEnvironment: JavaSE-1.6
Import-Package: javax.annotation;version="1.0.0",
 org.osgi.framework;version="1.3.0",
 org.osgi.service.log;version="1.3",
 org.osgi.util.tracker;version="1.3.1"
Bundle-ClassPath: .,
 lib/dom4j-1.6.1.jar,
 lib/guava-r08.jar,
 lib/javassist-3.12.1.GA.jar,
 lib/reflections-0.9.5.jar,
 lib/slf4j-api-1.6.1.jar,
 lib/xml-apis-1.0.b2.jar
Export-Package: reflectiveplugin, 
 reflectiveplugin.data

Note: Used Reflections v. 0.9.5

Here's a sample UrlType implementation:

package reflectiveplugin;

import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.util.Enumeration;
import java.util.Iterator;

import org.osgi.framework.Bundle;
import org.reflections.vfs.Vfs;
import org.reflections.vfs.Vfs.Dir;
import org.reflections.vfs.Vfs.File;
import org.reflections.vfs.Vfs.UrlType;

import com.google.common.collect.AbstractIterator;

public class BundleUrlType implements UrlType {

    public static final String BUNDLE_PROTOCOL = "bundleresource";

    private final Bundle bundle;

    public BundleUrlType(Bundle bundle) {
        this.bundle = bundle;
    }

    @Override
    public boolean matches(URL url) {
        return BUNDLE_PROTOCOL.equals(url.getProtocol());
    }

    @Override
    public Dir createDir(URL url) {
        return new BundleDir(bundle, url);
    }

    public class BundleDir implements Dir {

        private String path;
        private final Bundle bundle;

        public BundleDir(Bundle bundle, URL url) {
            this(bundle, url.getPath());
        }

        public BundleDir(Bundle bundle, String p) {
            this.bundle = bundle;
            this.path = p;
            if (path.startsWith(BUNDLE_PROTOCOL + ":")) { 
                path = path.substring((BUNDLE_PROTOCOL + ":").length()); 
            }
        }

        @Override
        public String getPath() {
            return path;
        }

        @Override
        public Iterable<File> getFiles() {
            return new Iterable<Vfs.File>() {
                public Iterator<Vfs.File> iterator() {
                    return new AbstractIterator<Vfs.File>() {
                        final Enumeration<URL> entries = bundle.findEntries(path, "*.class", true);

                        protected Vfs.File computeNext() {
                            return entries.hasMoreElements() ? new BundleFile(BundleDir.this, entries.nextElement()) : endOfData();
                        }
                    };
                }
            };
        }

        @Override
        public void close() { }
    }

    public class BundleFile implements File {

        private final BundleDir dir;
        private final String name;
        private final URL url;

        public BundleFile(BundleDir dir, URL url) {
            this.dir = dir;
            this.url = url;
            String path = url.getFile();
            this.name = path.substring(path.lastIndexOf("/") + 1);
        }

        @Override
        public String getName() {
            return name;
        }

        @Override
        public String getRelativePath() {
            return getFullPath().substring(dir.getPath().length());
        }

        @Override
        public String getFullPath() {
            return url.getFile();
        }

        @Override
        public InputStream openInputStream() throws IOException {
            return url.openStream();
        }
    }
}

And this is how I create reflections in the Activator class:

private Reflections createReflections(Bundle bundle) {
    Vfs.addDefaultURLTypes(new BundleUrlType(bundle));
    Reflections reflections = new Reflections(new Object[] { "reflectiveplugin.data" });
    return reflections;
}

The last bit is very confusing, but still important: if you run your plugin inside of Eclipse (Run As / OSGi Framework) you have to add also your classes output directory to the Reflections path patterns (i.e. "bin" or "target/classes"). Although, it's not needed for a released plugin (to build a plugin/bundle do "Export"->"Deployable plug-ins and fragments").

Solution 2

Just for the records in case someone else has the same problem. Here a small modification to the answer of Vlad in order to avoid having to add the output directory to the Reflections path patterns. The difference is only in the BundleDir class. It seems to work fine in all my tests:

public class BundleUrlType implements UrlType {

public static final String BUNDLE_PROTOCOL = "bundleresource";

private final Bundle bundle;

public BundleUrlType(Bundle bundle) {
    this.bundle = bundle;
}

@Override
public Dir createDir(URL url) {
    return new BundleDir(bundle, url);
}

@Override
public boolean matches(URL url) {
    return BUNDLE_PROTOCOL.equals(url.getProtocol());
}


public static class BundleDir implements Dir {

    private String path;
    private final Bundle bundle;

    private static String urlPath(Bundle bundle, URL url) {
        try {
            URL resolvedURL = FileLocator.resolve(url);
            String resolvedURLAsfile = resolvedURL.getFile();

            URL bundleRootURL = bundle.getEntry("/");
            URL resolvedBundleRootURL = FileLocator.resolve(bundleRootURL);
            String resolvedBundleRootURLAsfile = resolvedBundleRootURL.getFile();
            return("/"+resolvedURLAsfile.substring(resolvedBundleRootURLAsfile.length()));
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    public BundleDir(Bundle bundle, URL url) {
        //this(bundle, url.getPath());
        this(bundle, urlPath(bundle,url));
    }

    public BundleDir(Bundle bundle, String p) {
        this.bundle = bundle;
        this.path = p;
        if (path.startsWith(BUNDLE_PROTOCOL + ":")) { 
            path = path.substring((BUNDLE_PROTOCOL + ":").length()); 
        }
    }

    @Override
    public String getPath() {
        return path;
    }

    @Override
    public Iterable<File> getFiles() {
        return new Iterable<Vfs.File>() {
            public Iterator<Vfs.File> iterator() {
                return new AbstractIterator<Vfs.File>() {
                    final Enumeration<URL> entries = bundle.findEntries(path, "*.class", true);

                    protected Vfs.File computeNext() {
                        return entries.hasMoreElements() ? new BundleFile(BundleDir.this, entries.nextElement()) : endOfData();
                    }
                };
            }
        };
    }

    @Override
    public void close() { }
}


public static class BundleFile implements File {

    private final BundleDir dir;
    private final String name;
    private final URL url;

    public BundleFile(BundleDir dir, URL url) {
        this.dir = dir;
        this.url = url;
        String path = url.getFile();
        this.name = path.substring(path.lastIndexOf("/") + 1);
    }

    @Override
    public String getName() {
        return name;
    }

    @Override
    public String getRelativePath() {
        return getFullPath().substring(dir.getPath().length());
    }

    @Override
    public String getFullPath() {
        return url.getFile();
    }

    @Override
    public InputStream openInputStream() throws IOException {
        return url.openStream();
    }
}
}

Solution 3

Eclipse is build on top of OSGi and you are up against OSGi class loading... and that is not an easy battle to win.

Have a look at this article of Neil Bartlett: OSGi Readiness — Loading Classes. Also you can google for "OSGi buddy policy".

Comments

[Sergio]

Thanks for your answer @Vlad. No idea how to create such bundles but I will try (I am far from being an expert in OSGi yet...). So I need two bundles: 1) the Reflections library having "DynamicImport-Package: *" at its Plug-in MANIFEST; 2) a bundle with my plugin, where I have to fill in at the "Exported packages" list of the plugin configuration editor, all the packages that should be accessible to the Reflexions Bundle. Then I guess I should compose these two bundles into another one, right ?. Could you please elaborate a bit more your answer? all this seems to be a bit complex!. Thanks

[Sergio]

I see :( . @Tonny, Do you know another library for querying this kind of information -classes with a particular annotation and so on- in an application making part of an Eclipse plug-in?. As far as I know Spring can also answer similar questions, but I have not tried it before, and no idea if it will work fine when making part of an Eclipse Plug-in or if it will present the same problems I currently have. Thanks for your feedback!.

[Tonny Madsen]

You can make the Reflections library into an OSGi bundle and then add the DynamicImport-Package: * heading to manifest.mf. You can make the initial bundle from the existing jar file using the "New Project"->"Plug-in from existing JAR Archive" wizard... Good luck.

[Vlad]

@Sergio, I have updated the answer with my findings (previous answer didn't provide a solution). Unfortunately, it happened to be much less straight forward. Hope, it'll help.

[Sergio]

Thanks a lot for expanding your answer @Vlad. I have tried to implement it without success until now. If I understood correctly from your new answer, I need only the plugin bundle and the Reflexions jar in its bundle classpath (as I originally had it). So I am ignoring the previous answer about putting Reflexions in a separate bundle, is this correct ?. About your manifest file: Your 'Bundle-ClassPath' looks similar to mine, in my 'Export-Package' I do not have a '.data' file as you, and in the 'Import-Package' I do not see the dependencies you have, is that important for Reflexions to work?

[Sergio]

In your test: did you have to use the "ConfigurationBuilder>>addClassLoader()" method to set a special classloader? or manipulating classloaders was not necessary ?. This is the error message I am having in the console: "20 [Thread-1] ERROR org.reflections.Reflections - given scan urls are empty. set urls in the configuration". I am creating a FilterBuilder object with: "FilterBuilder fb = new FilterBuilder();" and after, doing this: "fb.include(FilterBuilder.prefix("mypackageroot"));". As you recommend me I am doing the same for the output directory: "fb.include(FilterBuilder.prefix("bin"));"

[Vlad]

Yes, ignore previous answer. It's important to import all required jars, you need to add them manually as described in the article I provided a link to in the answer (unless you're using maven with eclipse osgi plugins to build your project).

[Vlad]

Re classloaders manipulation it isn't necessary as you can see in the createReflections method, and here's how I had to change it (added "bin" prefix) to make it work when run from Eclipse IDE: Reflections reflections = new Reflections(new Object[] { "reflectiveplugin.data", "bin" });. Though if you're doing it with ConfigurationBuilder you should not forget to register Scanners. Check Reflections(final Object[] urlHints, final Scanner... scanners) constructor, it delegates to the constructor with Configuration argument.

[Sergio]

I managed to make it work. It would have taken me ages to do it without your help here. Thanks. I still will try to find a workaround for not having to hardcode the output folder of my plugin in the Reflexions path, but this is a minor issue in comparison to the original problem!.

[Vlad]

You're welcome. Let us know if you find a solution for the hardcoded output folder (I have a guess it can be related to not having the output folder in Bundle-ClassPath in MANIFEST.MF. Also, if you use maven with pde-maven-plugin to build your MANIFEST.MF, it should automatically construct correct Bundle-ClassPath based on dependencies defined in pom.xml etc.)

[Sergio]

Hi, I tried with the output folder in the Bundle-ClassPath but it did not help :( . In the meanwhile I launched a new thread with that specific question: stackoverflow.com/questions/8407364/…

[Sergio]

UPDATE: apparently the problem of having to write the output folder for the Reflexions filters, occurs not only when executing the plugin with "Run as/Eclipse application", but also when the plug-in is fully deployed in Eclipse (the plugin jar has been located in the plugins folder). I am wondering if a way to avoid this is asking Reflexions to work with the classloader of the plugin activator class. I am doing some experiments in that direction and I will post if I succeed, in case you have a new idea @Vlad also let me know. Thanks!.

[Link19]

I'm today having this exact issue, I have never created bundles before but have tried for this. I don't understand where to get the Bundle Object from to pass in to this constructor, can you help?