How to import org.apache Java dependencies w/ or w/o Maven

Question

So the quick background is I am creating a java program, that uses many different imports

    import org.apache.hadoop.conf.*;
    import org.apache.hadoop.fs.Path;
    import org.apache.hadoop.io.*;
    import org.apache.hadoop.mapreduce.*;
    import org.apache.hadoop.util.*;
    import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
    import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

I know that Netbeans isn't finding these files because I do not have them on my computer. But is there a way to have Netbeans automatically connect with org.apache and retrieve these files? Or do I just have to go and download them. Someone recommended using Maven, but I am not sure if this is the right solution or how to go about that?

Thanks

Solution 1

Unless you use a Maven structure (see here getting started with Maven) you will have to download all jars manually.

If using only Hadoop (as in your example) this might not seem that much of a deal, but when working with big projects it is easier to declare your dependencies in a pom.xml file. It is much more easier than downloading X different jars, and you can easily move to a newer version of a library, rather than having to delete and and download another.

I saw that someone asked in a comment why people like Maven so much. Well, to be honest, I personally find it easy to use and very useful. Furthermore, a Maven project can be easily imported in IntelliJ, Eclipse or Netbeans, whereas creating for example an IntelliJ project can cause difficulties in importing it in Eclipse or NetBeans.

To get started using Maven with Netbeans, you can go to: New Project, Categories:Maven Projects:{Best Option}. Then in the project files, open pom.xml. Here is where dependencies for your project are added. If you are not sure what to insert try searching for your jar name + "maven" on the internet. The plugin for Netbeans is able to connect to the maven repository and autocomplete most fields.

Sample from: http://mvnrepository.com/artifact/mysql/mysql-connector-java/5.1.6

<project...>
....
<dependencies>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-core</artifactId>
        <version>1.0.3</version>
    </dependency>
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>5.1.6</version>
    </dependency>
</dependencies>
...
</project>

Solution 2

Download the .jar file here: http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-core/0.20.2

In Eclipse, right-click on your project, click Properties, search in the text box for Library, click on Build Paths, click Add External JAR, and select the file you downloaded from the link above.

Solution 3

You will have to download the jar-files yourself. Unless you start using Maven or a similar dependency management tool.

Solution 4

You must download them. The name org.apache.hadoop is a package name, and we only use the name of the site as a convention. See this tutorial on packages for more information. Essentially a package is a folder on your computer, often in the Java\jre\lib\ext\ directory.

Solution 5

Refer tutorial http://www.tutorialspoint.com/hadoop/hadoop_mapreduce.htm

It mentions :-

Download Hadoop-core-1.2.1.jar, which is used to compile and execute the MapReduce program. Visit the following link http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-core/1.2.1 to download the jar.

View more solutions

Comments

[duffymo]

I do not understand why people like Maven so much.

[Stephen C]

I do not understand why people dislike Maven so much. :-)

[duffymo]

People? No, just me. It's the unnecessary complexity for insufficient benefit that kills me. Plus the mandates on directory structure that as far as I know cannot be altered.

[Yair Zaslavsky]

You can use ivy instead of maven. I can tell you that with Eclipse you can generate a maven project quite easy.

[duffymo]

I can generate with Maven and IntelliJ just fine. That's not my point. I see too many people coming here because they're stuck when the root cause is larding too many tools on top of a poor understanding. Better to learn without all the "conveniences". And I greatly prefer Ivy and Ant to Maven. I hate being forced to use their conventions.

[gobernador]

For a moment I asked myself why you were keeping your dependencies in a porn.xml file