Maven unpack tar.gz artifact of latest version

Question

The goal is to get the latest tar.gz artifact from repository and unpack it to some specific location.

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
            <version>2.5.1</version>
            <executions>
                <execution>
                    <phase>generate-resources</phase>
                    <goals>
                        <goal>unpack</goal>
                    </goals>
                    <configuration>
                        <artifactItems>
                            <artifactItem>
                                <groupId>com.enterprise</groupId>
                                <artifactId>skrillex</artifactId>
                                <version>${product.version}</version>
                                <type>tar.gz</type>
                                <outputDirectory>target/product</outputDirectory>
                            </artifactItem>
                        </artifactItems>
                    </configuration>
                </execution>
            </executions>
        </plugin>

there is also

<dependencies>
    <dependency>
        <groupId>com.enterprise</groupId>
        <artifactId>skrillex</artifactId>
        <version>${product.version}</version>
        <type>tar.gz</type>
    </dependency>
</dependencies>

but we get the error:

[INFO] --- maven-dependency-plugin:2.5.1:unpack (unpack-unix) @ ... ---
[INFO] Configured Artifact: com.enterprise:skrillex:[1.1.70,):tar.gz
Downloading: https://repo/com/enterprise/skrillex/[1.1.70,)/skrillex-[1.1.70,).tar.gz

...

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:2.5.1:unpack (unpack-unix) on project ...: Unable to resolve artifact. Could not transfer artifact com.enterprise:skrillex:tar.gz:[1.1.70,) from/to ext (repo....): IllegalArgumentException

Solution 1

If you don't mind a two step process, use the following pom:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.enterprise</groupId>
<artifactId>skrillex-test</artifactId>
<version>1.0.0-SNAPSHOT</version>
<properties>
    <skrillex.version>[0.0.0,1.0.0)</skrillex.version>
</properties>
<packaging>jar</packaging>
<build>
  <plugins>
    <plugin>
     <groupId>org.codehaus.mojo</groupId>
     <artifactId>versions-maven-plugin</artifactId>
     <version>2.0</version>
     <configuration>
            <includes>
                    <include>com.enterprise:*</include>
            </includes>
     </configuration>
   </plugin>
   <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>2.5.1</version>
        <executions>
            <execution>
                <phase>generate-resources</phase>
                <goals>
                    <goal>unpack</goal>
                </goals>
                <configuration>
                    <artifactItems>
                        <artifactItem>
                            <groupId>com.enterprise</groupId>
                            <artifactId>skrillex</artifactId>
                            <version>${skrillex.version}</version>
                            <type>jar</type>
                            <outputDirectory>target/product</outputDirectory>
                        </artifactItem>
                    </artifactItems>
                </configuration>
            </execution>
        </executions>
    </plugin>
    </plugins>
</build>
<dependencies>
    <dependency>
        <groupId>com.enterprise</groupId>
        <artifactId>skrillex</artifactId>
        <version>${skrillex.version}</version>
    </dependency>
</dependencies>

And run first: mvn versions:resolve-ranges (it updates your pom with the desired version in the property)

followed by the maven goal you want, e.g.: mvn install

Now if you want the original pom back: mvn versions:revert

Solution 2

Note: The following is untested, but should work

OK, the issue here is that <artifactItem> does not resolve version ranges.

What you need to do is switch from dependency:unpack to dependency:unpack-dependencies

e.g.

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>2.5.1</version>
        <executions>
            <execution>
                <phase>generate-resources</phase>
                <goals>
                    <goal>unpack-dependencies</goal>
                </goals>
                <configuration>
                    <includeTypes>tar.gz</includeTypes>
                    <includeArtifactIds>skrillex</includeArtifactIds>
                    <outputDirectory>target/product</outputDirectory>
                </configuration>
            </execution>
        </executions>
    </plugin>

(For anyone else following along, you need to add the dependency ensuring you specify the type, e.g.

<dependencies>
    <dependency>
        <groupId>com.enterprise</groupId>
        <artifactId>skrillex</artifactId>
        <version>${product.version}</version>
        <type>tar.gz</type>
    </dependency>
</dependencies>

)

That should ensure that Maven resolves the range, and as the file type is not classpath compatible, the dependency will not be on the transitive classpath of this artifact anyway.

If you were doing this with a classpath compatible dependency, e.g. a .jar dependency, or a dependency that can be treated as such, e.g. a .zip dependency in a .war then you would want to add either <scope>test</scope> or <optional>true</optional> to the dependency so that the transitive dependency tree is not polluted.

Potential issues

There are some things you need to look out for:

• Maven 2.x does not track side-artifact presence in the remote repository, so if the .tar.gz is not attached to every version (or more critically every -SNAPSHOT version) then you can end up with the artifact not being found

• Maven 3.x does track the side-artifact presence in maven-metadata.xml but IIRC only for -SNAPSHOT versions, the idea being that if you deploy "partial" snapshots you can still resolve all the latest side artifacts (even if the latest was for an older -SNAPSHOT towards the same version

• Using version ranges is a really bad plan. It will put a world of pain on downstream consumers of your project as the range gets resolved in accordance with the update settings of your upstream <repositories>. Please reconsider and use a fixed version.

Solution 3

This is known issue in maven dependency plugin: http://jira.codehaus.org/browse/MDEP-50

In general - nobody like variable dependency versions. And I advise not to use them at all. Your product got own version. Specific version of your product depends on specific version of skrillex library. So set it in stone and live with that version.

Or use solution proposed by Aukjan with double maven invocation. As far as I know there's no way to force maven to reload pom, so you will not be able to do this in single maven invocation. Be aware: in case of changing API in skrillex library you can end up with broken build.

Comments

[gerrytan]

Does it work if you reference exact version number?

[nefo_x]

actually the goal is to break the build if API of Skrillex product would be changed. We're looking for the way to do it in elegant way.

[Roman Romanchuk]

In this case I assume that you want to make this task in CI? In teamcity you can easy configure multi-steps builds so double invocation should not be a problem.

[nefo_x]

in our case it is Jenkins, which works perfectly fine. but I don't remember if we can copy archived workspaces from two jobs.

[Stephen Connolly]

FYI, wearing my Apache Maven Developer hat. If the above does not work with a version range, then you have found a bug. Do not confuse the potential of being a bug with the Maven Developers thinking you should use a version range. Our considered opinion is that version ranges are a seductive siren that we wish we had not listened to.

[Ika]

How do you avoid the transitive dependency issue ?

[Stephen Connolly]

As I said in the answer, set the scope to test. Other, less optimal options are marking the dependency as optional or using scope provided which both will work. But rely on other plugins not misinterpreting (which to date is ok, but may prove not future-proof)

[nish1013]

@Stephen Connolly where should we add <scope>test<scope> , I have got a zip with two jars inside it those needed at compile time

[Stephen Connolly]

The <scope> is a hack. You want to ensure that the dependency is not exported so you lie and say it is test scoped. Then the dependency plugin comes along and grabs the test scoped dependency and puts it where you want it so that it is present at compile time