No such file or directory error when using Hadoop fs --copyFromLocal command

Question

I have a local VM that has Hortonworks Hadoop and hdfs installed on it. I ssh'ed into the VM from my machine and now I am trying to copy a file from my local filesystem into hdfs through following set of commands:

[root@sandbox ~]# sudo -u hdfs hadoop fs -mkdir /folder1/
[root@sandbox ~]# sudo -u hdfs hadoop fs -copyFromLocal /root/folder1/file1.txt /hdfs_folder1/

When I execute it I get following error as - copyFromLocal:/root/folder1/file1.txt': No such file or directory

I can see that file right in /root/folder1/ directory but with hdfs command its throwing above error. I also tried to cd to /root/folder1/ and then execute the command but same error comes. Why is the file not getting found when it is right there?

Solution

By running sudo -u hdfs hadoop fs..., it tries to read the file /root/folder1/file.txt as hdfs.

You can do this.

1. Run chmod 755 -R /root. It will change permissions on directory and file recursively. But it is not recommended to open up permission on root home directory.
2. Then you can run the copyFromLocal as sudo -u hdfs to copy file from local file system to hdfs.

Better practice is to create user space for root and copy files directly as root.

1. sudo -u hdfs hadoop fs -mkdir /user/root
2. sudo -u hdfs hadoop fs -chown root:root /user/root
3. hadoop fs -copyFromLocal

Comments

[Jason Donnald]

I did that but it still shows as No such file or directory. I executed sudo -u hdfs hadoop fs -copyFromLocal /root/folder1/file1.txt /hdfs_folder1/ after executing chmod 755 -R /root/folder1

[Jason Donnald]

the permission for this file is - rwxrwxrwx 1 root root

[Durga Viswanath Gadiraju]

It is not just about file. You can run command sudo -u hdfs ls -ltr /root/folder1 and see if it can list the file

[Jason Donnald]

I get pemission denied error. So what do I do? I changed the permission to 755 as you mentioned but still same issue

[Durga Viswanath Gadiraju]

In that case you have to run this command chmod 755 /root, which is not advisable. You should not open up permissions on root directory. I have modified the answer to create user space for root and copy files.