How to copy files from Amazon EC2 server

Question

I am desperately trying to copy some files from AWS EC2 machine but I am struggling with following.

• I am able to copy only some files from already existing folder (although I want to to copy all the files recursively to my local machine)
• scp and rsync complains that path doesn't exist when I create completely new file in some location (let's say /home/ec2-user/a.txt)

The command I am using is

scp -rpi ~/.ssh/my-key.pem  [email protected]:/home/ec2-user/a.txt

I created the file under ec2-user with touch a.txt so the permissions should be fine (if that even matters).

Contents of original folder I want to copy is

-rw-r--r--  1 webapp   webapp     90 17. úno  2015 cron.yaml
drw-r--r--  2 ec2-user ec2-user 4096  1. dub 13.09 db-backup
-rw-r--r--  1 webapp   webapp   2703 17. úno  2015 index.php
-rw-r--r--  1 webapp   webapp    189 17. úno  2015 logo_aws_reduced.gif
drwxrwxrwx 15 ec2-user ec2-user 4096  1. dub 17.35 martintour
drw-r--r-- 14 ec2-user ec2-user 4096 30. bře 20.52 master_martintour
-rw-r--r--  1 webapp   webapp    367 17. úno  2015 scheduled.php
-rw-r--r--  1 webapp   webapp   3490 17. úno  2015 styles.css

However only this part is copied

-rw-r--r-- 1 pmensik pmensik   90 úno 17  2015 cron.yaml
-rw-r--r-- 1 pmensik pmensik 2703 úno 17  2015 index.php
-rw-r--r-- 1 pmensik pmensik  189 úno 17  2015 logo_aws_reduced.gif
-rw-r--r-- 1 pmensik pmensik  367 úno 17  2015 scheduled.php
-rw-r--r-- 1 pmensik pmensik 3490 úno 17  2015 styles.css

So it seems like the -r option is not working. And also it doesn't explain why I cannot copy newly created files or folders like I mentioned previously.

The command I used for this is

scp -rpi ~/.ssh/my-key.pem  [email protected]:/var/app/current . 

I've already played with changing the permissions but no I am completely lost.

Solution 1

You are missing the destination directory.

Try:

scp -rp -i ~/.ssh/my-key.pem [email protected]:/home/ec2-user/a.txt /your/local/dir/

Solution 2

Besides destination - permission does matter, because you're logging in to the instance with scp/rsync as ec2-user, not root. Nevertheless, e.g. cron.yaml is readable to anyone, this is why it was transferred. Make sure a.txt owner is not 'root'.

Try:

rsync -Hva --progress -e "ssh -i ~/.ssh/my-key.pem" [email protected]:/home/ec2-user/ ./

where ./ is your local destination dir. Note trailing slashes at the end of both paths.

Comments

[100rabh]

in your first command, you're copying from /home/ec2-user2/a.txt ie, notice the extra 2 after ec2-user. which doesn't correspond to your user

[Petr Mensik]

Just a typo, sorry, it doesn't work even with the right path

[Jakuje]

Post exactly what errors you get. Increasing log level (-vvv) might help to show more information.

[Petr Mensik]

This is result of the scp command pastebin.com/RayDETG3

[Petr Mensik]

I tried exactly what you have written and no file other than those created by OS itself were transfered (like .bashrc). So it doesn't work though my files has the permissions and owner like the rest of the "system" files. Really don't know what's going on

[Petr Mensik]

Ok, found the solution - I was connection to the wrong machine all time, I mixed up IP addresses in the start and noticed it just now. I am gonna delete the question. Stupid me, thanks anyway :)