Preserving file and folder permissions with rsync

Question

I maintain a backup of my email accounts using this command:

sudo rsync -av --delete --progress -e "ssh -p pNumber" --rsync-path="/usr/bin/rsync" /vmail/ user@my_backup_server:/home/user/backups/vmail/

Source: Most email folders are owned by user vmail.

Destination (backup server): System doesn't have a user named vmail.

My question, would the above command preserve file and directory permissions even if destination machine doesn't have a user named vmail? Would it be possible to restore the files and permissions completely from destination to source even if the user names between the two machines are not the same (some missing on backup server).

Solution 1

What rsync copies is the numerical user id of the file, regardless if it exists on the target system. If a user with that id doesn't exist, ls etc. will just show that number instead of a name. If that user id belongs to another username on the target system, this user will now own the file.

Backup and restore will work without a problem in this scenario.

Solution 2

How rsync preserves ownership of files depends on two things:

• Are you super-user (root) on the destination?
  Otherwise you can't create files and directories with a different user other than your own.

• Which option flags are you using?

The -a option includes the -o, --owner, -g, --group options designed to preserve ownership.

At the file-system level user and group ownership is stored in UID resp. GID numbers. When there is no mapping from UID/GID's to usernames and groupnames tools will simply display those numbers instead.
Users and groups with the same names can have different UID/GID numbers on different systems.

By default rsync will try to match the ownership by username resp. groupname. In other words when the user vmail is the owner of a file at the source, rsync will make the user vmail also the owner at the destination (even when they have different UID/GID numbers).
That is usually quite resilient and the most predictable for humans as we normally don't look at ownership in the form of UID/GID numbers.

When no matching user vmail is present on the remote destination, then a fall-back scenario will happen. Rsync will then preserve the actual underlying UID/GID numbers and the UID number of the vmail user on the source will used to set the owner.

That should preserver the correct ownership when you reverse the rsync direction and restore the backup.

man rsync :

   -o, --owner
          This  option  causes  rsync to set the owner of the destination file to be the same as the source file,
          but only if the receiving rsync is being run as the super-user (see also the --super  and  --fake-super
          options).   Without this option, the owner of new and/or transferred files are set to the invoking user
          on the receiving side.

          The preservation of ownership will associate matching names by default, but may fall back to using  the
          ID number in some circumstances (see also the --numeric-ids option for a full discussion).


   --numeric-ids
          With  this option rsync will transfer numeric group and user IDs rather than using user and group names
          and mapping them at both ends.

          By default rsync will use the username and groupname to determine what ownership  to  give  files.  The
          special  uid  0 and the special group 0 are never mapped via user/group names even if the --numeric-ids
          option is not specified.

          If a user or group has no name on the source system or it has no match on the destination system,  then
          the  numeric ID from the source system is used instead.  See also the comments on the "use chroot" set‐
          ting in the rsyncd.conf manpage for information on how the chroot setting affects  rsync’s  ability  to
          look up the names of the users and groups and what you can do about it.

Solution 3

With your case specifically, the real issue arises when it comes time to restore the files. The key would be to specify the desired owner/group when you pull the files back. --chown=vmail:vmail

Assuming that you've already created the user vmail on the new machine to which you will restore, you'd issue something like the following:

sudo rsync -av --chown=vmail:vmail --force --delete --progress user@my_backup_server:/home/user/backups/vmail/ /vmail/

Doing it this way means it doesn't matter who owns the files on the backup server so long as you can rsync to/from that user (which is implied as already being true in your example).

Comments

[HBruijn]

By default rsync will use the username and groupname to determine what ownership to give files on the remote system, not the UID/GID numbers. If a user or group has no name on the source system or it has no match on the destination system, only then the numeric ID from the source system will be used.

[W.M.]

Thanks for the detailed answer. So, you're saying that I should connect (login) to the destination machine (backup server) as root and not as a normal user?

[HBruijn]

As a normal user you can't create files that don't belong to you, so you need to either log in as root on the destination, or you have to be root locally and run rsync in the opposite direction and pull the files in.

[Læti]

Just adding a note that on old versions of rsync the --chown option is not available. This has been added on 3.1.0 in 2013. And yes, MacOS 10.14 still has rsync 2.6.9...

[HBruijn]

Actually that won’t be a problem, on the backup destination files won’t be owned by the vmail user, as that user doesn’t exist there, but will have an owner with only numerical UID. When using rsync in the opposite direction, for a restore, rsync will preserve that numerical UID. On the mailserver that UID is assigned to the vmail user and ownership will be restored correctly

[CR.]

It's critically important to use --numeric-ids if you're using rsync as an OS backup solution. Especially if you're backing up system files (eg. /etc). Often when you're backing up or restoring you're using various different systems. Maybe one machine has the original data. Another machine does the backups. And a third machine is used to do a restore (maybe booting off a live-CD/USB). It's easy to get a uid/gid mismatch if you don't use pure numeric ids the whole time.