Using rsync on WSL (Bash for Windows 10) for incremental backups

Question

I was an avid user of Linux for many years but switched to Windows. One thing that I have always missed is using rsync to make incremental snapshots for the purposes of backup. You use a command like this:

rsync -aP --link-dest=PATHTO/$PREVIOUSBACKUP $SOURCE $CURRENTBACKUP

and it will create a complete directory structure of the current state of the folder you are backing up, using hard links to reference the previous backup. Only new or changed files are actually written to disk; all the others are saved as hard links. This is just plain awesome, and none of the solutions I have found in Windows have been able to do this. (For reference, I've tried Windows File History, and paid money for commercial software including Crashplan and the otherwise brilliant Beyond Compare.)

Now that Microsoft have introduced WSL I have tried to do this in bash for Windows. It appears to have worked perfectly, but Windows is reporting the size on disk incorrectly, as shown here:

To explain, in one shapshot snapshot-170831__07h08m48s I added a large video file, 1.40 GB, which took several seconds to copy. In the next shapshot, snapshot-170831__07h09m09s the file took no time to copy, which suggests the hard link was created correctly and that no space has been taken up on my hard drive.

However all my Windows tools (Windows Explorer, Directory Opus, WinDirStat) report that the second snapshot folder is taking up 1.40 GB on disk.

The first question then is why is Windows incorrectly reporting the size on disk for these hard links?

But the more important question is should I be doing this: using Linux tools under Windows for my regular file backups. I've already seen on WSL that modifying files like .bashrc using Windows tools will make the file unreadable inside WSL. I wonder whether it is too risky to trust my backups to a tool like this, or whether I risk corrupting the file system and losing important files.

Solution

Calculating the size of hard links in Windows

It's difficult to calculate the size of hard-linked files in Windows. One tool that allows you to do this is TreeSize Professional (not free, analysis of hard links is switched off by default). I used this tool and it correctly estimated the size of the hard-linked files.

For a more thorough discussion, see How can I check the actual size used in an NTFS directory with many hardlinks?

Are the files actually working?

As for the other part of the question, is it risky to backup files using Linux tools under Windows on WSL? I decided to test this simply by copying one of the snapshot directories to a separate external hard drive. There were no problems copying the files, or reading them from the external drive. In other words, the hard links are behaving exactly as expected, and the files are working.

Long-term data stability

So down to the final point, could using Linux tools under WSL as part of my regular backups break something, such as corrupting the file system? Do I trust WSL not to break things majorly? Anything can break at any time, so I will make sure these snapshot directories get copied periodically to a separate drive.

Comments

[laurent]

I've also been using rsync with hard links on Windows for several years and it seems to work. Only issue I had is with file permissions which seem to be messed up, but that could be because I was using Cygwin.

[Kit Johnson]

I've been using this system for two years now and have had no issues with the hard links.