Using rsync to create a complete OS backup/clone sufficient?

Question

Will using rsync to duplicate an entire file system, e.g. doing

rsync -avxHAX --progress / /dev/[backup disk]

as suggested here, result in a complete duplicate Linux system? I.e. will all system and security settings, network configurations (proxies, VPNs), user/group policies etc. be faithfully reproduced or is there anything - apart from things outside the file system (bootloader, partition tables) - that rsync will miss?

What if the source disk is LUKS-encrypted (example), i.e. doing

cryptsetup luksOpen /dev/[source disk] && rsync -avxHAX --progress / /dev/[backup disk]?

Solution 1

No, it won't work at all.

You should send it to a mounted filesystem, so not /dev/[backup disk] but something like /mnt/[backup disk]/ after doing mount /dev/[backup disk] /mnt/[backup disk]

Furthermore, if you have multiple 'real' filesystems (ext4, btrfs, zfs, ...) you will also miss those. If you run mount | grep \/dev\/sd you can, in most cases see all the ones you have.
(There are exceptions: filesystems that are not mounted at the moment, filesystems that are not a /dev/sdX device, ...)

Edit: I didn't answer the 2nd question. The same problem here, and the same fix would apply.
BUT: If the destination filesystem doesn't use encryption everything will be saved decrypted

Solution 2

Actually you can do it with rsync. But why bother doing it manually?

To make a complete copy of the OS the best tool would be Timeshift (it is offered by Debian and its offsprings, probably others too). It will create a complete copy of everything needed for a full backup-restore. Under the hood, the Timeshift will use rsync, but it already knows all pieces of OS which are required for complete copy.

Solution 3

I think you are asking two different questions.

Will rsync copy/duplicate a file system? Yes it will. It can also be used to convert between file systems (you can copy an ext3 filesystem into a ext4 partition for instance). The file systems need to support each others features: copying ext4 into fat will lose a lot of functionality. You do need to mount the file systems; I am assuming you know how to do that. You also can usually get the live CDs to mount the file systems for you then press Control-Alt-T to open a command prompt.

I'm not familiar with LUKS; I expect that if you can mount the filesystem such that the files can be read that it will copy.

Your original question also asks about a clone. The copy you make with rsync will almost certainly not boot. You need to do a couple more steps.

If the partitions have changed, you will need to modify /etc/fstab. If your existing system is using UUIDs to identify the partitions, you will need to modify fstab. You can change it back to the old non-UUID style if you want; that makes it easier to clone but it is not recommended.

Then you will need to add boot code. You will need to use chroot to modify the target system. Most commonly grub2 is installed. The best examples I've found are here: Ubuntu community help on grub2

And because links move around, I'll gratuitously copy some examples:

this is a way to install grub

sudo mount /dev/sdXY /mnt # Example: sudo mount /dev/sda5 /mnt
sudo grub-install --boot-directory=/mnt/boot /dev/sdX # Example: sudo grub-install --boot-directory=/mnt/boot /dev/sda

And to update the directory of images in grub:

for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done
sudo chroot /mnt
sudo update-grub

Comments

[Andrew Henle]

Not only that, by trying to take an image of the live, mounted root filesystem there's no guarantee that the image is coherent and consistent.

[david]

Thx for the answer/replies! Gee, that's a huge disappointment. It was suggested here that rsync supports hot-transferring of file trees. In the case of the above-mentioned example, what would then be a "canonical" way to duplicate the file system?

[david]

@Garo: What's the difference between just sending everything e.g. to a USB flash drive and first mounting it? Won't plugging in the pen drive usually automatically mount it?

[Garo]

rsync does indeed do a hot-transfer but that doesn't mean that during the rsync files don't change on your source. That being said, on pretty much every desktop, rsync -avxHAX --progress / /mnt/extradisk/ will indeed do exactly what you want. (If you have a extradisk with a partition with one of the default Linux filesystems mounted on /mnt/extradisk). See man rsync for the full info to get a idea what actually happens to see if it will work in your case

[Garo]

I don't really understand the question about the usb flash drive. On most systems it will indeed mount it automatically. But that does not mean you can just use /dev/[usb flashdrive] as a destination ! The 'files' in /dev are always representations of the disk it self, not of the mount point ! Run a mount | grep usb_flash_drive to find out the mount point.

[david]

Thanks for this bit! Does Timeshift also create standalone backups, i.e. will the backup be bootable by itself?

[david]

@Garo: Thank you for this important correction! I need to do some reading on /dev vs /mnt as I don't quite understand the difference. When e.g. wiping a drive by doing dd if=/dev/zero of=/dev/sdX, zeros are written to /dev/sdX and not /mnt/sdX - at least from the syntax

[david]

This is what I get from mount | grep \/dev\/sd: /dev/sda1 on /boot type ext4 (rw,relatime)

[david]

@AndrewHenle: You're right, but AFAIK rsync is designed to handle this issue, it knows when files are being changed while copying and will do an update afterwards

[White Owl]

No, the backup is not bootable by itself. But you can boot from LiveCD and (assuming you have the same structure of partitions on your HDD) you can restore the backup on a new machine.

[Kusalananda]

You have suggested this same software in answers three times in an hour. Your user account has so far only been used to promote this software. Are you associated with the software in any shape or form? See unix.stackexchange.com/help/promotion