hadoop fs -put command
77,291
Solution 1
As user hdfs
, do you have access rights to /root/
(in your local hdd)?. Usually you don't.
You must copy file1.txt
to a place where local hdfs
user has read rights before trying to copy it to HDFS.
Try:
cp /root/MyHadoop/file1.txt /tmp
chown hdfs:hdfs /tmp/file1.txt
# older versions of Hadoop
sudo -u hdfs hadoop fs -put /tmp/file1.txt /
# newer versions of Hadoop
sudo -u hdfs hdfs dfs -put /tmp/file1.txt /
--- edit:
Take a look at the cleaner roman-nikitchenko's answer bellow.
Solution 2
I had the same situation and here is my solution:
HADOOP_USER_NAME=hdfs hdfs fs -put /root/MyHadoop/file1.txt /
Advantages:
- You don't need
sudo
. - You don't need actually appropriate local user 'hdfs' at all.
- You don't need to copy anything or change permissions because of previous points.
Solution 3
try to create a dir in the HDFS by usig: $ hadoop fs -mkdir your_dir
and then put it into it $ hadoop fs -put /root/MyHadoop/file1.txt your_dir
Related videos on Youtube
Author by
skfeng
Updated on December 13, 2020Comments
-
skfeng over 3 years
I have constructed a single-node Hadoop environment on CentOS using the Cloudera CDH repository. When I want to copy a local file to HDFS, I used the command:
sudo -u hdfs hadoop fs -put /root/MyHadoop/file1.txt /
But,the result depressed me:
put: '/root/MyHadoop/file1.txt': No such file or directory
I'm sure this file does exist.
Please help me,Thanks!
-
skfeng over 10 yearsThanks,but I tried to move this file to another place,type the follow command: {sudo -u hdfs hadoop fs -put /home/skfeng/Downloads/hadoop-book-master/test-data/ch1/file1.txt /} it still doesn't work and has the same results: {put: `/home/skfeng/Downloads/hadoop-book-master/test-data/ch1/file1.txt': No such file or directory }
-
Alfonso Nishikawa over 10 yearsAnd does user
hdfs
have access rights to the home of the userskfeng
? -
skfeng over 10 yearsYes! I understand!User hdfs have no access to home of other users,I make a new directory for hdfs,and it works!Thank you very much!:)
-
Alfonso Nishikawa over 9 yearsI like this solution, but I always forget what is the environment variable name :P +1
-
mourad m about 4 yearsThanks, i have to use DFS instead of FS, like this : HADOOP_USER_NAME=PUT_YOUR_USER_HERE hdfs dfs -put /source/xxx/yyy /destination/zzz