Escaping spaces in a remote path when using rsync over a remote SSH connection

Question

When using SSH to connect rsync to a remote server, how do you escape spaces and such in the remote path? A simple backslash escapes the space for the local bash prompt, but on the remote machine the space is then being read as a break in the path, thus marking the end of that path.

So when I do rsync -avz /path/to/source/some\ dir/ [email protected]:/path/to/dest/some\ dir/ what happens is that the remote server is reading that as just /path/to/dest/some/ and since it can't find that destination remotely, because the actual destination is "some dir" rather than just "some".

If I try the same command and escape the backslash and the space to get past the local bash prompt and maintain the backslash for the remote server (three backslashes total: /path/to/dest/some\\\ dir/), it does indeed send the backslash to the remote server, but the remote server then interprets the path as /path/to/dest/some\/ rather than /path/to/dest/some\ dir/ still stripping the space and the characters after it.

If I try to wrap the path with quotes, it behaves pretty much the same way, effectively cutting the path off at the space. So it too only works to get past the local bash prompt.

Initially I was using a path that had a " - " (space-hyphen-space) segment in it, and the remote server was returning an error rsync: on remote machine: -: unknown option which is what started this whole space-escaping endeavor in the first place.

So what must I do to get this working properly with the remote server, without having to remove the spaces or other erroneous characters like hyphens from the remote path?

Solution 1

On the initiator machine, rsync builds up a command line that invokes the rsync target on the remote machine, then sends that command line using ssh.... as a single string. That single string is passed to the shell to parse, split into arguments and execute rsync. I have no idea why is that done, instead of packing the (already splitted, expanded and unquoted) arguments in some binary-safe container to the remote rsync.

That means that your arguments will be parsed by two different shells, quote and requote accordingly. Usually, I wrap each argument with double-quotes, and then the whole expression on single-quotes. sometimes it's not enough, or it can be complicated if you want the same expression to be used locally and remotely.

In that cases, I usually set some soft links with simple, no-spaces, all-ASCII names, and use that.

Solution 2

The problem:

rsync over ssh uses two shells to run: one local and one remote. The argument [email protected]:/path/to/dest/some\ dir/ is first interpreted by your local shell. Therefore, it becomes [email protected]:/path/to/dest/some dir/. This is the value passed to the remote shell, which will (of course) interpret it as two separate arguments: [email protected]:/path/to/dest/some and dir/.

---

The point answer:

There's no need for any kind of gimmicks to get the desired result. rsync has the solution built-in (unless you're using some ancient version):

 -s, --protect-args          no space-splitting; only wildcard special-chars

So, if you add the -s flag, you'll need to quote your arguments only for the local shell:

rsync -savz user@server:"/my path with spaces/another dir/" "/my destination/"

Solution 3

You were on the right track when you said:

If I try the same command and escape the backslash and the space to get past the local bash prompt and maintain the backslash for the remote server

Here's a way I find easiest to do it:

rsync -av dir\ with\ spaces/ server.tld:"dir\ with\ spaces"

and this way works as well.

rsync -av dir\ with\ spaces/ server.tld:dir\\\ with\\\ spaces

Can you post the exact output and any errors you're seeing?

Can you replace rsync on both sides with a wrapper script?

$ sudo su -
# cd /usr/bin
# mv rsync rsync.real
# cat <<'EOF' >rsync
#!/bin/bash
logfile=/home/yourname/rsync.log
date >> "$logfile"
i=1
for arg in "$@"; do
    echo "arg $i: $arg" >> "$logfile"
    i=$((i+1))
done

rsync.real "$@"
EOF
# chmod +x rsync

Then run your rsync again, and it should prove that this way of escaping works, e.g.

Client side:

Sun Feb 13 13:48:12 EST 2011
1: -av
2: dir with spaces/
3: server:dir\ with\ spaces

Server side:

Sun Feb 13 13:48:13 EST 2011
1: --server
2: -vlogDtpre.iL
3: .
4: dir with spaces

In the above example, the fact that the 4th argument on the server (dir with spaces) is all on one line says that the quoting is working correctly.

If this doesn't help, try re-running rsync -v, or rsync -vv, or rsync -vvv. It will give you extra debugging information.

Two other silly suggestions:

• is the other server a Linux server, and what is your default shell there?
  • maybe it is expanding file names differently than you expect
• did you forget to add the -a or -r option?
  • I can't tell without seeing your output

Comments

[jftuga]

Try both single & double quotes.

[purefusion]

Javier mentioned this too, and it did end up working, so I've pasted my working code segment in a reply to his answer.

[m000]

@purefusion Consider marking Grégory's answer as correct. The -s addresses the issue without the need to apply double-escaping manually.

[purefusion]

And the winner is... single+double quotes! Maybe this is different on every server, so if the other answers don't work for some people, perhaps this one will. Here's the successful code I used on the remote side of the rsync command: '[email protected]:"/path/to/dest/some\ dir/"'

[purefusion]

Now begs the question, why does THIS work (on my server), but not either of the other options (just wrapping the path in double quotes, or using triple-backslashes)? I'm running CentOS 5 if that matters.

[purefusion]

As you may have seen in the question's details, I have indeed already tried this. However, another answer suggested specific quote usage, and that did indeed end up working. :)

[purefusion]

As mentioned in the question's details, I did indeed try both of your first two methods. Perhaps they just don't work on my server. I also didn't feel like messing around with wrapper scripts. I try to stay away from hacking the /usr/bin/... In any case, another more specific way of quoting did indeed work for me, so perhaps it's just my server that's acting this way. Your suggestions may still certainly be viable for people on other servers, or those not running CentOS, if the OS is part of the problem after all.

[Mikel]

So how did you quote it to make it work?

[Mikel]

That shouldn't be necessary. How are you running it? Are you running it using eval or via a call to a function perhaps?

[Mikel]

You are not using single plus double quotes. You are using single quotes plus double quotes plus backslash escapes. THREE levels of quoting. This should be unnecessary. I ask again: How are you running it?

[Mikel]

You are leaving out some vital detail. Please post the actual command you are running, in full.

[Mikel]

@Javier "I have no idea why is that done". Presumably so that tilde expansion works.

[Mikel]

+1. Javier's description of how rsync works is good, and the advice to use double quotes with single quotes inside is very helpful in general. But it is not the solution to this problem.

[Sridhar Sarnobat]

You need BOTH quotes and backslashes.

[m000]

Thanks! This is a proper solution, rather than a workaround.

[cayhorstmann]

This is also the right answer for those who struggle with scp. Use rsync -sav instead.