Managing multi-module dependencies with Maven assembly plugin

Question

I use Maven assembly plugin to create an assembly for my multi-module project. There are two separate applications built from this multi-module project, each having a separate set of dependencies. I made a custom assembly descriptor which assembles two directories (for each application) with module builds and their respective dependencies. It does everything fine but one thing - it puts dependencies for both modules to each other's assembly.

The following is a simplified version of my project, that has exactly the same behavior.

Consider a project consisting of two modules and an assembly module:

APP
  module1
  module2
  assembly

I have added dependencies purely for demonstration:

com.test.app:module1:jar:1.0
\- commons-cli:commons-cli:jar:1.2:compile

com.test.app:module2:jar:1.0
\- commons-daemon:commons-daemon:jar:1.0.8:compile

Here's the parent POM:

<project>
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.test</groupId>
  <artifactId>app</artifactId>
  <version>1.0</version>
  <packaging>pom</packaging>

  <modules>
    <module>module1</module>
    <module>module2</module>
    <module>assembly</module>
  </modules>
</project>

module1 POM:

<project>
  <parent>
    <groupId>com.test</groupId>
    <artifactId>app</artifactId>
    <version>1.0</version>
  </parent>

  <modelVersion>4.0.0</modelVersion>

  <groupId>com.test.app</groupId>
  <artifactId>module1</artifactId>
  <version>1.0</version>
  <packaging>jar</packaging>

  <dependencies>
    <dependency>
      <groupId>commons-cli</groupId>
      <artifactId>commons-cli</artifactId>
      <version>1.2</version>
    </dependency>
  </dependencies>
</project>

module2 POM:

<project>
  <parent>
    <groupId>com.test</groupId>
    <artifactId>app</artifactId>
    <version>1.0</version>
  </parent>

  <modelVersion>4.0.0</modelVersion>

  <groupId>com.test.app</groupId>
  <artifactId>module2</artifactId>
  <version>1.0</version>
  <packaging>jar</packaging>

  <dependencies>
    <dependency>
      <groupId>commons-daemon</groupId>
      <artifactId>commons-daemon</artifactId>
      <version>1.0.8</version>
    </dependency>
  </dependencies>
</project>

assembly POM:

<project>
  <parent>
    <groupId>com.test</groupId>
    <artifactId>app</artifactId>
    <version>1.0</version>
  </parent>

  <modelVersion>4.0.0</modelVersion>

  <groupId>com.test.app</groupId>
  <artifactId>assembly</artifactId>
  <version>1.0</version>
  <packaging>pom</packaging>

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <version>2.2.2</version>

        <executions>
          <execution>
            <id>make-assembly</id>
            <phase>package</phase>

            <goals>
              <goal>single</goal>
            </goals>
          </execution>
        </executions>

        <configuration>
          <appendAssemblyId>false</appendAssemblyId>

          <descriptors>
            <descriptor>src/main/assembly/descriptor.xml</descriptor>
          </descriptors>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

And finally, the assembly descriptor:

<assembly>
  <id>distribution</id>
  <includeBaseDirectory>false</includeBaseDirectory>

  <formats>
    <format>dir</format>
  </formats>

  <moduleSets>
    <moduleSet>
      <useAllReactorProjects>true</useAllReactorProjects>

      <includes>
        <include>com.test.app:module1:jar</include>
      </includes>

      <binaries>
        <outputDirectory>module1</outputDirectory>
        <unpack>false</unpack>

        <dependencySets>
          <dependencySet>
            <unpack>false</unpack>
          </dependencySet>
        </dependencySets>
      </binaries>
    </moduleSet>

    <moduleSet>
      <useAllReactorProjects>true</useAllReactorProjects>

      <includes>
        <include>com.test.app:module2:jar</include>
      </includes>

      <binaries>
        <outputDirectory>module2</outputDirectory>
        <unpack>false</unpack>

        <dependencySets>
          <dependencySet>
            <unpack>false</unpack>
          </dependencySet>
        </dependencySets>
      </binaries>
    </moduleSet>
  </moduleSets>
</assembly>

As you can see, assembly is bind to package phase. So, when I execute

mvn package

from parent directory, I have the following assembly

module1/
  commons-cli-1.2.jar
  commons-daemon-1.0.8.jar
  module1-1.0.jar
module2/
  commons-cli-1.2.jar
  commons-daemon-1.0.8.jar
  module2-1.0.jar

Basically, the problem here is that module1 does not depend on commons-daemon, but the assembly plugin has included the dependence. Similarly, with module2 and commons-cli.

Can someone explain why the assembly plugin behaves this way?

An what would be a solution?

Solution

I've always had similar experiences using the assembly plugin with multi-module projects where the end result wasn't what I expected. I hope someone else can provide a more accurate answer as to why that's happening and how best to use those two concepts in tandem.

That said, a possible work-around would be to have module1 and module2 generate their own assembly artifacts which contain their respective jars and dependencies. Then you can modify the assembly sub-module pom file to have dependencies on the generated distribution artifacts from its sibling modules and then unpack those into a new assembly.

In both Module1 and Module2's pom files you can add an assembly plugin configuration to your package phase much like you did with the assembly sub-module.

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <version>2.2.2</version>

        <executions>
          <execution>
            <id>make-assembly</id>
            <phase>package</phase>
            <goals>
              <goal>single</goal>
            </goals>
          </execution>
        </executions>

        <configuration>
          <descriptors>
            <descriptor>src/main/assembly/descriptor.xml</descriptor>
          </descriptors>
        </configuration>
      </plugin>
    </plugins>
  </build>

Module1 would have a src/main/assembly/descriptor.xml like this

<assembly>
  <id>distribution</id>
  <includeBaseDirectory>false</includeBaseDirectory>

  <formats>
    <format>zip</format>
  </formats>

  <dependencySets>
    <dependencySet>
      <outputDirectory>module1</outputDirectory>
      <unpack>false</unpack>
    </dependencySet>
  </dependencySets>
</assembly>

And Module2 will have a similar src/main/assembly/descriptor.xml

<assembly>
  <id>distribution</id>
  <includeBaseDirectory>false</includeBaseDirectory>

  <formats>
    <format>zip</format>
  </formats>

  <dependencySets>
    <dependencySet>
      <outputDirectory>module2</outputDirectory>
      <unpack>false</unpack>
    </dependencySet>
  </dependencySets>
</assembly>

Then in the assembly/pom.xml you would add the Module 1 and 2 zip artifacts as dependencies

  <dependencies>
    <dependency>
      <groupId>com.test.app</groupId>
      <artifactId>module1</artifactId>
      <version>1.0</version>
      <type>zip</type>
      <classifier>distribution</classifier>
    </dependency>
    <dependency>
      <groupId>com.test.app</groupId>
      <artifactId>module2</artifactId>
      <version>1.0</version>
      <type>zip</type>
      <classifier>distribution</classifier>
    </dependency>
  </dependencies>

...and trim up the assembly/src/main/assembly/descriptor.xml file to look like this

<assembly>
  <id>distribution</id>
  <includeBaseDirectory>false</includeBaseDirectory>

  <formats>
    <format>dir</format>
  </formats>

  <dependencySets>
    <dependencySet>
      <useTransitiveDependencies>false</useTransitiveDependencies>
      <unpack>true</unpack>
    </dependencySet>
  </dependencySets>

</assembly>

Like I said this would be one possible work around and unfortunately adds a significant amount of additional XML configuration to your build process. But it works.

Comments

[sertsy]

Thanks, for a reply, Keith. As far as I understand, in the example you have shown, if I execute mvn package from parent POM, it would first assemble module1, then module2 and packaging assembly module would just put the first two assemblies together. However, this devours the purpose of a separate assembly module. The idea behind a separate module for an assembly is to make sure, that all modules have been built when assembly is created.

[sertsy]

Assume I have another module - module3 which is a dependency for module1. If at the package phase of module1, module3 is not yet built, module1 will not be able to produce an assembly (since assembly is tied to a package phase). That is why assembly module is always placed last - to make sure that other modules have been built already. What you suggest would work in the example I have given, but, I believe, it is not the right way. I hope I have made my thoughts clear.

[Keith]

The order of the modules listed in the parent pom is not relevant. The maven reactor will build all dependencies in correct build order, which it knows how to do because the assembly pom has module1 and module2 listed as dependencies. Even if you add module3 as a dependency for module1, it will build module3 first, then module1, module2, finishing with assembly.

[James Bassett]

Keith you're a legend - I've been trying to get this to work for hours! The <type> and <classifier> in the assembly pom stopped my jars and their dependencies getting unpacked. Thank you :)