How to best clone a running system to a new harddisk using rsync?

Question

I have a system running as a server that has a failing harddrive. While all important data is on a RAID and backed up and all that, I don't have an image of the system itself. There is no specific need as I can do the install of course, but I still want to try to do a hotcopy first before just going down the restore path. I know there are some downsides to this as a process, but there I don't think there are much downsides to trying it as a first resort.

• OS: Ubuntu 12.04.4 LTS
• Headless
• I'm not hoping to install too much new software as the disk is allready failing :)
• The system is running. I'm scared stopping it increases the chance of the disk not comming back up. This means dd might be out?
• The new disk is not the same size (it's twice as big) as the old one further complicating the dd issue.

My idea was to

• hotplug the new drive in the system
• make a filesystem
• mount it in /mnt/somedir
• rsync the files
• some fstab magic
• some booting magic

The questions I still have are:

What would be a good rsync command? I was planning on:

rsync -aAXx  / /mnt/somedir/ 
   --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found}

(I'm skipping some more dir's, e.g. my mounted raid's etc)

Where the options are:

-a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
-A, --acls                  preserve ACLs (implies -p)
-X, --xattrs                preserve extended attributes
-x, --one-file-system       don't cross filesystem boundaries

I'm specifically skipping -H, -v and --progress to speed up the process.
Would that work for Ubuntu? I'm not sure if Ubuntu uses any hardlinks, but I don't think I need the option, do I ?

After this run I could reboot (maybe with a live usb drive) and re-run the rsync if the old disk still starts. This would fix any non-readable/changed files because the system was running I suppose.

---

How to fix booting?
Then my plan would be to change the UUID for / in my fstab (still have to google how to find the uuid), and do some magic so the system actually boots from the new disk

Have I forgotten something or have I planned something specifically stupid?

Solution 1

You can grab the UUID for all block devices with the blkid command. (You want the one that just says UUID, not PARTUUID)

The rsync options I use are -avhPHAXx.

I don't think -v or --progress will speed anything up unless you're on a very slow console/tty.

Using -x eliminates the need for all of your exclusions assuming they are all on different filesystems (on my system, all except lost+found are).

The only frequently used program that I know of that uses hard links (at least on my system) is git, so that's why I add the -H option. The only problem I think you would have by not using -H is that it will take up a little more space.

As for the bootloader, if you are using GRUB2 with MBR, then the command I use is grub-install /dev/sda (replace sda with the correct drive for you). That should make the new drive bootable. If you're using a different bootloader or UEFI, then I would check google for how to get the new drive booting properly. Just remember that /boot on the new drive will need to be on the same partition as it currently is (assuming you're not using a UUID for /boot also), otherwise you will need to modify fstab accordingly.

Solution 2

I just did this successfully (after a couple trys).

I used

sudo rsync -ahPHAXx --delete --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found} / /mnt

Then I reset my /mnt/etc/fstab file for the boot partition and my swap space.

Then I needed to reset GRUB

for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done
sudo chroot /mnt
sudo grub-install --recheck /dev/sdX
sudo update-grub

Solution 3

Do not attempt to fix booting with this drive; my recommendation:

1. Plug in a drive on another machine instead of risking a hotswap on the failing machine.
2. rsync your non-system files across to the new disk.
3. On a separate hard disk, create a boot, a minimal root, and a swap. Install the same operating system that you wish to clone from the ailing hard disk.
4. Boot from this new disk (ideally on a separate machine, if you cannot risk downtime on the target host, otherwise you can boot the target host with this disk).
5. Add the disk from step #2 to this new system, with the correct mount points. Now you have a clone of the system drive. You can (optionally) copy the partitions over to this new disk, but my recommendation is to keep the disk as a minimum requirement for booting your OS. As you have realized, having too many things on the primary partition makes recovery difficult. Make sure you copy the network configuration correctly as your server is headless.
6. Simply replace the drive on the ailing system with this new disk pair (if you choose not to boot the target initially in step #4).
7. Reboot.

Comments

[Nanne]

I can risk downtime, it's a non-essential server. I was going for the hotswap (hot-add basically) because that copy would be much faster. There shouldn't be too many risks I thought, as we are talking SATA (II at least). Apart from that, what is the reason to suggest an extra boot disk? Wouldn't that complicate the matters for some allready installed software, homedirs etc? It sounds like more work which I'm trying to avoid :D

[Burhan Khalid]

Its what I use in production; I mount /home, /etc, /var, /usr and /opt on external disks (actually on LVM), and / and /boot on a separate disk. This way I don't have to worry when the main disk fails I just swap it out without affecting my services :) Its a bit of work initially but saves you in the long run. Especially when you run out of space on a partition and need to add to a volume without downtime.

[Nanne]

While you make a good point, it's not a requirement I currently have, nor need :). There are several other discs in the system (about 7) and I have to draw the line somewhere ;D. So OS doesn't need splitting at the moment. I mean: I agree the system is good, just not something I'm looking for now ;D. So if I leave it all on 1 disk, the fact I'm copying external will make it seriously slow, wouldn't it?

[Nanne]

I don't believe I have them on separate systems I'm afraid, so the exclude seems prudent. I'm reading mixed stuff about the speed of - v, so I'm not sure what to do there ;). Git is a good point, I'm sure /home have some of that!

[Burhan Khalid]

Depends on your interface to the external, over ethernet it should be fast, over USB 2 faster, on USB 3 even faster. It won't make it very slow - of course don't copy directories that are being written to (like say, /tmp, which you don't need anyway) or anywhere pipes are being written to.

[Nanne]

I'm not sure how and why, but the excludes were wrong anyway. They actuallywere in the dry-run I did, not sure why. (well, I copied the format from a semi-random place, so that might be the reason?). Anyway, with the -x firmly in place per your advice I didn't worry too much, only thing extra happened was that some unimportant stuff (old backups-of-backups for instance) have been 'saved' as well.

[Teque5]

Note that this is only for MBR formatted disks.

[retromuz]

This saved my a$$. I cloned 12GB+ system on Linode to brand new node. After 5+ years later, this still works. Thanks guys!

[TheArchitecta]

So when partitioning the cloned filesystem ready to accept the clones, should my root partition have the boot flag set while I am keeping them insync, or do I set it just before putting the cloned drive into production. Otherwise I would assume the cloned drive is then bootable which is not really wanted. In my situation my clone drive is a disk on a separate server. And Rsync is used to keep it aligned with prod.

[nsayer]

Isn't adding --del necessary to insure any extra files on the target are removed?