Gradle exclude java class from lib replaced by own class to avoid duplicate

Question

In Android Studio, there is a specific file (src/org/luaj/vm2/lib/jse/JavaMethod.java) that I need to overwrite from a package that is pulled in via Gradle (dependencies {compile 'org.luaj:luaj-jse:3.0.1'}).

I copied the file into my source directory with the exact same path and made my changes to it. This was working fine for an individual JUnit test case that was using it. It also looks like it is working for a normal compile of my project (unable to easily confirm at the moment).

However, when I try to run all my tests at once via a configuration of ProjectType="Android Tests", I get Error:Error converting bytecode to dex: Cause: com.android.dex.DexException: Multiple dex files define Lorg/luaj/vm2/lib/jse/JavaMethod$Overload;.

Is there a specific task or command that I need to add to my Gradle file to make sure the project selects the file in my local source directory? I tried the Copy task and the sourceSets->main->java->exclude command, but neither seemed to work (I may have done them wrong). I also tried the "exclude module/group" directive under "compile" from this post.

The non-default settings for the Run/Debug Confirmation:

• Type=Android Tests
• Module=My module
• Test: All in package
• Package: "test"

All my JUnit test cases are in the "test" package.

Any answer that gets this to work is fine. If not Gradle, perhaps something in the android manifest or the local source file itself.

[Edit on 2016-07-24] The error is also happening on a normal compile when my android emulator is running lower APIs. API 16 and 19 error out, but API 23 does not.

Solution 1

This is what I ended up adding after Fabio's suggestion:

//Get LUAJ
buildscript { dependencies { classpath 'de.undercouch:gradle-download-task:3.1.1' }}
apply plugin: 'de.undercouch.download'
task GetLuaJ {
    //Configure
    def JARDownloadURL='http://central.maven.org/maven2/org/luaj/luaj-jse/3.0.1/luaj-jse-3.0.1.jar' //compile 'org.luaj:luaj-jse:3.0.1'
    def BaseDir="$projectDir/luaj"
    def ExtractToDir='class'
    def ConfirmAlreadyDownloadedFile="$BaseDir/$ExtractToDir/lua.class"
    def JarFileName=JARDownloadURL.substring(JARDownloadURL.lastIndexOf('/')+1)
    def ClassesToDeleteDir="$BaseDir/$ExtractToDir/org/luaj/vm2/lib/jse"
    def ClassNamesToDelete=["JavaMethod", "LuajavaLib"]

    //Only run if LuaJ does not already exist
    if (!file(ConfirmAlreadyDownloadedFile).exists()) {
        //Download and extract the source files to /luaj
        println 'Setting up LuaJ' //TODO: For some reason, print statements are not working when the "copy" directive is included below
        mkdir BaseDir
        download {
            src JARDownloadURL
            dest BaseDir
        }
        copy {
            from(zipTree("$BaseDir/$JarFileName"))
            into("$BaseDir/$ExtractToDir")
        }

        //Remove the unneeded class files
        ClassNamesToDelete=ClassNamesToDelete.join("|")
        file(ClassesToDeleteDir).listFiles().each {
            if(it.getPath().replace('\\', '/').matches('^.*?/(?:'+ClassNamesToDelete+')[^/]*\\.class$')) {
                println "Deleting: $it"
                it.delete()
            }
        }
    }
}

I'll upload a version that works directly with the jar later.

Solution 2

issue: when linking your app the linker finds two versions

• org.luaj:luaj-jse:3.0.1:org.luaj.vm2.lib.jse.JavaMethod and
• {localProject}:org.luaj.vm2.lib.jse.JavaMethod

howto fix: tell gradle to exclude org.luaj:luaj-jse:3.0.1:org.luaj.vm2.lib.jse.JavaMethod from building

android {
    packagingOptions {
        exclude '**/JavaMethod.class'
    }
}

I have not tried this with "exclude class" but it works for removing duplicate gpl license files a la "COPYING".

If this "exclude" does not work you can

• download the lib org.luaj:luaj-jse:3.0.1 to the local libs folder,
• open jar/aar with a zip-app and manually remove the duplicate class.
• remove org.luaj:luaj-jse:3.0.1 from dependencies since this is now loaded from lib folder

Solution 3

I am not completely sure I understand your problem; however, it sounds like a classpath ordering issue, not really a file overwrite one.

AFAIK, gradle does not make a 'guarantee' on the ordering from a 'dependencies' section, save for that it will be repeatable. As you are compiling a version of file that you want to customize, to make your test/system use that file, it must come earlier in the classpath than the jar file it is duplicated from.

Fortunately, gradle does allow a fairly easy method of 'prepending' to the classpath:

sourceSets.main.compileClasspath = file("path/to/builddir/named/classes") + sourceSets.main.compileClasspath

I don't know enough about your system to define that better. However, you should be able to easily customize to your needs. That is, you can change the 'compile' to one of the other classpath (runtime, testRuntime, etc) if needed. Also, you can specify the jarfile you build rather than the classes directory if that is better solution. Just remember, it may not be optimal, but it is fairly harmless to have something specified twice in the classpath definition.

Solution 4

This is rather convoluted but it is technically feasible. However it's not a single task as asked by the poster:

1. Exclude said dependency from build.gradle and make sure it's not indirectly included by another jar (hint: use ./gradlew dependencies to check it)
2. create a gradle task that downloads said dependency in a known folder
3. unpack such jar, remove offending .class file
4. include folder as compile dependency

If it's safe to assume that you're using Linux/Mac you can run a simple command line on item 3, it's only using widely available commands:

mkdir newFolder ; cd newFolder ; jar xf $filename ; rm $offendingFilePath

If you don't care about automatic dependency management you can download the jar file with curl, which I believe to be widely available on both linux and mac.

curl http://somehost.com/some.jar -o some.jar

For a more robust implementation you can substitute such simple command lines with groovy/java code. It's interesting to know that gradle can be seen as a superset of groovy, which is arguable a superset of java in most ways. That means you can put java/groovy code pretty much anywhere into a gradle.build file. It's not clean but it's effective, and it's just another option.

For 4 you can have something along either

sourceSets.main.java.srcDirs += ["newFolder/class"]

at the root level of build.gradle, or

dependencies {
. . . 
   compile fileTree(dir: 'newFolder', include: ['*.class'])
. . . 

View more solutions

Comments

[Dakusan]

What you gave is essentially what I am looking for, but it didn't work. I like to keep things clean, so I want to keep luaj as a gradle include, so it doesn't have to get added to the source tree. And I would highly prefer a solution that didn't require others to have to manually edit those gradle jar packages. My temporary solution while waiting for a real answer has been to just open the .jar file and delete the .class files that I duplicated (like you said). I guess I should have included that. So I'll give you a +1 for mentioning that, but it is not the final answer I am looking for.

[Dakusan]

If you can find a way to make your packagingOptions->exclude to work, I would be most appreciative. (I gave up after trying many dozens of combinations). P.S. I was able to exclude all of the conflicting class files by doing an "exclude" within the dependency block, but unfortunately, it removed both the local and packaged instances.

[Dakusan]

I'd be willing to add 50 more to the bounty if you could help me git this figured out :-\

[k3b]

sorry i have not enough knowledge about this topic. Maybe you can find more deeper gradle experts here: discuss.gradle.org . it might be also helpful to change the title of this question. May be "gradle exclude java class from lib replaced by own class to avoid duplicate" can attract more users

[Dakusan]

I tried adding your suggestion in every conceivable manner, but everything gave me errors, mostly about "no such property" for compileClasspath

[Dakusan]

I might be able to accomplish what I am looking for with this. While I would prefer a "proper" solution, having a task run a custom script before the rest of the compile would accomplish my end goal. How would I create the Gradle task to run a custom script, running different ones depending on the OS?

[Fabio]

I wouldn't use a separate script, instead I'd try using gradle exec tasks like stackoverflow.com/questions/15776431/…. I can have a better look on those details later. If you want to go the extra mile you can write a download method in groovy inside build.gradle and call it from there stackoverflow.com/questions/14474973/…. Or you can put them all in a separate gradle script. Oh so many options :)

[Dakusan]

I'm working on the solution now. I've posted the first part

[Fabio]

This looks very cool, and smaller than I expected, there's a lot of gradle coolness that I didn't know in there. The only bit I got confused is a) how you create/check for $ConfirmAlreadyDownloadedFile actual file and b) why you needed to replace \\ for /.

[Dakusan]

The $ConfirmAlreadyDownloadedFile just sees if the operation has previously ran by looking for a file that would have been created. I replace the slash for when running in windows. It might not have had to have been done.

[Peter V. Mørch]

Item 3 can be rewritten as zip -d $filename $offendingFilePath which modifies the jar file in-place. A jar file is just a zip file, after all.