Adding a system dependency to Maven

Question

I am working with Apache Spark through Maven, and I am trying to modify the source by including a 3rd party jar and trying to utilize some methods within it.

I get the following error when compiling the Spark project using

mvn -Dhadoop.version=2.2.0 -Dscala-2.11 -DskipTests clean package

not found: object edu
[ERROR] import edu.xxx.cs.aggr._

I modified ResultTask.scala to contain an import statement. So, maven is unable to find the jar I am trying to use and link it with the project.

I have added a dependency to the pom.xml file such as this:

<dependency>
    <groupId>edu.xxx.cs</groupId>
    <artifactId>aggr</artifactId>
    <version>0.99</version>
    <scope>system</scope>
    <systemPath>${basedir}/aggr.jar</systemPath>
</dependency>

The jar file I am trying to link is located in the same directory as the spark pom.xml file. I added this dependency to pom.xml. I inserted it in between 2 existing dependencies within the pom.xml file. I'm not sure whats wrong, but I would just like the jar to get linked for now, so that I can use the methods within it. I'm also not sure if I should be using anything specific for the groupId, artifactId, and version. edu.xxx.cs.aggr is the root package which contains other source files and packages. I would appreciate any help.

UPDATE

I used

mvn install:install-file -Dfile=<path-to-file> -DgroupId=edu.xxx.cs -DartifactId=aggr -Dversion=0.99 -Dpackaging=jar to install the jar to the .m2 repo. I checked the repo to see if it was installed, and it was.

I changed the dependency in pom.xml to

<dependency>
        <groupId>edu.purdue.cs</groupId>
        <artifactId>aggr</artifactId>
        <version>0.99</version>
</dependency>

I still get the same error.

Solution

this is how I add system dependency to my maven pom.xml. with in the project root path, I have created lib directory and there I have placed my jar file.

<dependency>
    <groupId>com.sshx</groupId>
    <artifactId>sshx</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>${basedir}/lib/sshxcute-1.0.jar</systemPath>
</dependency>

if still you face the same issue try adding the dependency manually by issuing the following command from the jar file location

mvn install:install-file -Dfile=sshxcute-1.0.jar -DgroupId=com.sshx -DartifactId=sshx -Dversion=1.0 -Dpackaging=jar

this command will add the jar to your .m2 repository as a dependency and you need to change the pom.xml dependency as follows:

<dependency>
    <groupId>com.sshx</groupId>
    <artifactId>sshx</artifactId>
    <version>1.0</version>
</dependency>

once you are done, issue mvn clean install command from command prompt and build your application.

However, another option is to create a local repository. See at this thread: How to include local jar files in Maven project

Comments

[RagHaven]

I manually added the jar to the local repo. I created the appropriate directory structure <m2repo>/edu/xxx/cs/aggr/0.99/aggr-0.99.jar. I compiled again and I still got the same error

[Sachin Gupta]

don't try to create it manually. Just use command it will create it for you automatically. I think this command will create some 2-3 supporting files with the jar.

[RagHaven]

I manually added the jar file to my .m2 repo. Now, it is under the proper directories and the jar is there. But, I still get this error

[Prasad Khode]

does manually means issuing the mvn install command?? have you updated your dependency in your pom.xml file as mentioned above?

[RagHaven]

Thats what I did. I used mvn install:install-file -Dfile=<path-to-file> -DgroupId=edu.xxx.cs -DartifactId=aggr -Dversion=0.99 -Dpackaging=jar

[RagHaven]

I used mvn install:install-file -Dfile=<path-to-file> -DgroupId=edu.xxx.cs -DartifactId=aggr -Dversion=0.99 -Dpackaging=jar I changed the dependency to <dependency> <groupId>edu.xxx.cs</groupId> <artifactId>aggr</artifactId> <version>0.99</version> </dependency>

[Prasad Khode]

try to refresh your project once from IDE or issue mvn clean install command and try

[Sachin Gupta]

after installing the jar don't forget to refresh the project. or try mvn clean

[RagHaven]

I don't have this setup on an IDE. I am making changes via sublime and building using mvn. So, I should just use mvn clean install from the dir that contains pom.xml?

[Prasad Khode]

yes, you need to use mvn clean install to re-build the project so that your changes take effect

[RagHaven]

when I build the project I already use clean by doing mvn -Dhadoop.version=2.2.0 -Dscala-2.11 -DskipTests clean package. I tried mvn clean before building it, I still get the error.

[RagHaven]

Let us continue this discussion in chat.

[khmarbaise]

Simplest solution is to start using a repository manager and install that artifact there. Afterwards use it as any other dependency without system scope.