How to add hadoop dependency via Maven? I have hadoop installed and present in my IDE project library

Question

• I have hadoop installed on my system(I am using Mac 10.7)
• I am using Intellij IDEA as IDE and my hadoop project has listed hadoop*.jar as dependency
• When I do mvn install, it fails with the following error

(master) $ mvn clean install
[INFO] Scanning for projects...
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project groupId:hadoop:master-SNAPSHOT (/Users/me/code/p/java/hadoop-programs/hadoop-programs/pom.xml)

has 1 error [ERROR] 'dependencies.dependency.systemPath' for org.apache.hadoop:hadoop-core:jar must be omitted. This field may only be specified for a dependency with system scope. @ line 18, column 25 [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException

I make changed to my pom.xml as

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>groupId</groupId>
    <artifactId>hadoop</artifactId>
    <version>master-SNAPSHOT</version>
    <packaging>jar</packaging>

    <dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-core</artifactId>
            <version>1.0.3</version>
            <type>jar</type>
            <systemPath>/usr/local/Cellar/hadoop/1.0.3/libexec/hadoop-core-1.0.3.jar</systemPath>
        </dependency>
    </dependencies>
</project>

But still same error, How do I resolve this in Maven?

Solution 1

use this :

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-core</artifactId>
    <version>0.20.2</version>
</dependency>

Solution 2

A lot of the stuff in hadoop-core have moved to hadoop-client in newer versions, use -

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-hdfs</artifactId>
        <version>2.4.1</version>
    </dependency>