Hadoop - Input path does not exist

Question

I did set up the hadoop Ubuntu OS, followed all the necessary steps, 1.created the hdfs file system 2.Moved the text files to input directory 3.having privilege to access all the directories. but when run the simple word count example, i got

Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/user/root/input
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:224)

but, input path is valid and even can able view the files in that path from eclipse itself, so plz assist were i am wrong.

attached the screenshot for the reference

Solution

Add the following 2 lines in your code :

config.addResource(new Path("/HADOOP_HOME/conf/core-site.xml"));
config.addResource(new Path("/HADOOP_HOME/conf/hdfs-site.xml"));

Your client is looking into the local FS.

Comments

[Quetzalcoatl]

In a terminal, what do you get back if you execute hadoop fs -ls /user/root/input? Just want to verify your DFS is accessible through there too, other than just eclipse.

[Jeevanantham]

@Quetzalcoatl, Result:root@master:/usr/local/hadoop-0.20.2# bin/hadoop fs -ls /user/root/input Found 2 items -rw-r--r-- 1 root supergroup 405 2013-04-16 15:07 /user/root/input/sigle -rw-r--r-- 1 root supergroup 302 2013-04-12 15:23 /user/root/input/single.txt Able to connect

[Jeevanantham]

hi both lines are looking same, so do i need to add both. Instead of /HADOOP_HOME/ do we have to mention the location of conf folder.

[Jeevanantham]

problem solved i have now used path with hdfs location, "hdfs://master:9000/user/root/input"

[Jeevanantham]

thanks, ur suggestion also worked, while replaced the HADOOP_HOME with my hadoop's conf folder location

[Kobe-Wan Kenobi]

@Tariq Can you please explain why does it look into local FS?