How to move files to hdfs in hadoop using unix command

32,290

First the source (local file system) and then the target (HDFS) have to be specified. Run the command as below and it should work.

bin/hadoop dfs -mkdir /hdfs
bin/hadoop dfs -copyFromLocal D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt /hdfs/StartUnit.txt
Share:
32,290

Related videos on Youtube

Shreyuth
Author by

Shreyuth

Updated on September 18, 2022

Comments

  • Shreyuth
    Shreyuth over 1 year

    I have 2 directories "dft" and "hdfs" in the unpacked Hadoop folder. I am trying to copy the file(StartUnit.txt) in dft into hdfs. The corresponding command and exception are as shown below

    313159@PC213710 ~/hadoop-0.19.1
    $ bin/hadoop dfs -copyFromLocal D:/eclipse/cygwin/home/313159/hadoop-0.19.1/hdfs              
    D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt
    
    HADOOP_NAMENODE_OPTS=-Dcom.sun.management.jmxremote
    HADOOP_SECONDARYNAMENODE_OPTS=-Dcom.sun.management.jmxremote
    HADOOP_DATANODE_OPTS=-Dcom.sun.management.jmxremote
    HADOOP_BALANCER_OPTS=-Dcom.sun.management.jmxremote
    HADOOP_JOBTRACKER_OPTS=-Dcom.sun.management.jmxremote
    HADOOP_PID_DIR=-Dcom.sun.management.jmxremote
    copyFromLocal: Pathname /D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt 
        from D:/eclipse/cygwin/home/313159/hadoop-0.19.1/dft/StartUnit.txt is not a  
    valid DFS filename.
    Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
    

    can someone suggest me...

  • NIMISHAN
    NIMISHAN over 8 years
    hadoop fs -moveFromLocal /home/Desktop/project/data/access-0.log hdfs://localhost:54310 project/data/access-0.log nothing happen?
  • simlev
    simlev about 7 years
    If this is the solution to your issue, you should marke the answer as accepted.
  • Ashok Kumar
    Ashok Kumar about 4 years
    Thanks works in 2020!