How to make tar save the archive on a remote machine using sftp or ftp?

Question

I would like to backup some of my very important data on a remote machine.
Currently I'm just saving it to my local machine by using this command: tar -cvjf ~/backups/Backup.tar.bz2 ~/importantfiles/*

I would prefer not using another command to transger it to the remote machine, meaning I would like to just have this command being upgraded so it can transfer the data to the remote machine.

This is designed to be in a script later that is suposed to run on its own, meaning any type of required user input would completly mess it up!

Something like

tar -cvjf sftp://user:pwassword@host/Backup.tar.bz2 ~/importantfiles/*
tar -cvjf  ftp://user:pwassword@host/Backup.tar.bz2 ~/importantfiles/*

would be perfect! (No pipes (etc.), just one command!)

Solution 1

For SSH:

tar czf - . | ssh remote "( cd /somewhere ; cat > file.tar.gz )"

For SFTP:

outfile=/tmp/test.tar.gz
tar cvf $outfile . && echo "put $outfile" | sftp remote:/tmp/
Connecting to remote...
Changing to: /tmp/
sftp> put /tmp/test.tar.gz
Uploading /tmp/test.tar.gz to /tmp/test.tar.gz
/tmp/test.tar.gz

Another SFTP:

outfile=/tmp/test.tar.gz
sftp -b /dev/stdin remote >/dev/null 2>&1 << EOF
cd /tmp
get $outfile
bye
EOF
echo $?
0

Solution 2

Tar doesn't speak ftp or sftp. That's not its job. You cannot do this with tar alone. Using appropriate tools for each job and combining them with the shell is the normal way of doing things on unix systems.

The most obvious solution is to create the archive locally, then copy it to the remote machine.

If you don't want to create the archive locally because you don't have enough room, you can create a named pipe, make tar write to this pipe, and find an (s)ftp client that can read from pipes. Unfortunately, sftp refuses to put a pipe. Some FTP clients work, for example lftp:

mkfifo f
tar -cvjf f ~/importantfiles/* &
sleep 2
lftp -f - <<EOF
open user@host
put f Backup.tar.bz2
EOF
wait
rm f

Put your FTP password in ~/.netrc.

Alternatively, there is a way to make your tar command save to the remote server directly, but you need some prior setup. Mount the remote server over SSHFS or curlftpfs.

mkdir -p ~/net/host
sshfs host: ~/net/host
tar -cvjf ~/net/host/Backup.tar.bz2 ~/importantfiles/*
fusermount -u ~/net/host

Comments

[terdon]

Does it need to be ftp? You don't have ssh access to this machine?

[priyanka -]

I do not have ssh access to the machine. It's just a backup server I rent. So yes it has to be ftp or sftp.

[kurtm]

@BrainStone SFTP is part of SSH

[priyanka -]

I know. But it is set up in a way that every ssh connection gets closed immediatley. SFTP works though. I know it is a wierd setup but that's the way it is.

[priyanka -]

@Jeight No. But as far as it is able to communicate with servers over ftp or sftp it is fine!

[Gilles 'SO- stop being evil']

@kurtm SFTP is part of SSH, but sometimes that's the only available part.

[kurtm]

@Jeight I wanted to make sure. It's common for folks to equate FTP and SFTP, even though SFTP is part of SSH. Since you were mentioning FTP and SFTP as the only options I wanted to rule that out.

[Gregg Leventhal]

SFTP is available when SSH is not if they are using sftp internal server and your shell is set to /sbin/nologin.

[priyanka -]

So how exactly would I use this? I mean how can I set which files should be in the tar ball. And where would I put the password? And how do I set it up so that I do not have to accept the hosts fingerprint?

[jirib]

Please learn more about ssh, man ssh, man ssh-agent, man ssh_config answer all your doubts. Which files you define locally with tar arguments, then it is piped via ssh to remote host and output is redirected to file.

[priyanka -]

Is it possible to do it without a pipe?

[priyanka -]

This also can't work because it needs to be sftp (no ssh access) or ftp

[jirib]

For ftp, see man lftp. If you need to have more complicated scenarios, use 'sftp -b' and have a file with commands or use 'here documents' (<< EOF).