How can a filesystem be copied exactly as is?

Question

I have a file system for a device I am programming that I would like to make an exact copy of. Ideally I would like this copy to be identical to the folder that it was copied from. I have tried using cp -r cp -a and rsync -azvP to try to achieve this. Each one though results in a folder with different size (using du -s) and ultimately,even though my device runs off of the original folder, it won't run off of the one that I copied.

• What is getting left out that the commands I have used aren't accounting for?
• Is it possible to keep everything identical when copying a file system/folder? If so how would I go about doing that?

P.S. I posted a similar questions on StackOverflow but quickly realized I had asked it on the wrong exchange

---

Edit: This may not be helpful but no matter which method I use the copied directory always causes the machine in question to Kernel Panic with the following output.

VFS: Unable to mount root fs via NFS, trying floppy. VFS: Cannot open root device "nfs" or unknown-block(2,0) Please append a correct "root=" boot option; here are the available partitions: 1f00
64 mtdblock0 (driver?) 1f02 64 mtdblock2 (driver?) 1f04 2432 mtdblock4 (driver?) 1f05 128 mtdblock5 (driver?) 1f06 4352 mtdblock6 (driver?) 1f07 204928 mtdblock7 (driver?) 1f08 50304 mtdblock8 (driver?) 0800
8388608 sda driver: sd Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(2,0)

Solution 1

I generally use one of the following alternatives:

• rsync -aHAX (add v for verbosity) makes sure that you preserve any link structure and x-attrs in the target folder while copying. Don't forget a means archive and preserves time, ownership and permissions already.
• Simple tar cvf (don't compress to save time, just tar them up) is what I use if the first one doesn't meet what I need for whatever reason and I have no time, but I always try the first one.

To check that everything went as it should, you can run diff -r <folder1> <folder2> afterwards if you want.

Solution 2

Reading your reply in the comments it sounds like you are possibly trying to copy the root folder ("My device won't boot using the copied directory"). In that case there are several things you should do.

$ cd /
$ mkdir backups

$ tar -cvpf /backups/fullbackup.tar --directory=/ --exclude=proc --exclude=sys \
     --exclude=dev/pts --exclude=backups .

Once you've copied the root folder to the new system you will need to update grub before it will boot.

$ grub-install --recheck /dev/sdX (Where X is the partition number)
$ update-grub

Solution 3

You could try FSArchiver. It is the successor to the similar partimage, which is now unmaintained. I have used both partimage and FSArchiver in the past, and they have both worked well. As far as I know, they both make copies of a filesystem that is as close to identical to the original as possible. I think FSArchiver is a bit more powerful than partimage. For example, it works with ext4, while partimage does not. See the FSArchiver/partimage comparison table.

Solution 4

Using du to compare folders is inherently troublesome. If you want to really compare 2 directories based purely on their sizes then use du like this:

$ du -sh --apparent-size <dir>

This switch will report the directories acutual size vs. the amount of disk space that it consumes when stored on the physical drive medium. Disks are organized into blocks and files are written to these blocks. If a file only needs BLOCK + 1 of space it will consume 2 BLOCKS worth of space, and this is what du normally reports. Remember the name of the tool is disk usage!

Given you're dealing with what sounds like an entire filesystem I would be inclined to use dd to make an exact copy of the partition that the directory is on

$ dd if=/dev/sda1 of=/srv/boot.img

You can then use this boot.img to restore the partition wherever you want.

$ dd if=/srv/boot.img of=/dev/sdb1

View more solutions

Comments

[Ignacio Vazquez-Abrams]

A difference in du output is not an indicator that the contents aren't the same.

[user3369606]

My device won't boot using the copied directory. I am taking that as an indicator and wondering what exactly I am doing wrong that the copied file system isn't treated the same

[forcefsck]

You need to specify exactly what you're trying to do. rsync -a or cp -a do exact copies of directories, but you're trying to copy a bootable disk, there are a couple of things more you need to do to complete that.

[michas]

What kind of device? What kind of file system? What kind of storage media?

[Admin]

To me it sounds like you want to use something like dd if=/some/location of=/some/other/location bs=4M to read and write data as is.

[msw]

+1 I think the clue is in the failed boot message in the OP and this addresses it.

[user3369606]

As it turns out something very similar to this worked. I was dealing with a filesystem(FS) on my machine that I needed to copy. All it took was for me to tar c then tar x it. The kicker though was to make sure that my machine was off since I was booting it using the FS that was local to my machine, but remote to the device that I was booting (I was starting up the device through NFS)

[user3369606]

This did the trick! I also just needed to make sure that the device I was booting up (through NFS) was turned off and not able to access the file system while the copy was going on.

[Victor Roetman]

du will also report the size on disk (so sparse files and hard links take less space). For example, du -sh /var gives 21G but du -sh --apparent-size /var gives 209G on my system.

[0x2207]

What about --numeric-ids?

[bayindirh]

@0x2207 It's only useful when a user have different numeric IDs on different systems (e.g. user hbayindir is on both systems, and has UID 1000 on source and 1536 on destination). I never had to use that option, since I need to preserve the real owner, not UID and GID. If a user is not present either source or destination, numeric ID is used as a fallback.

[0x2207]

What real owners are if you copy root filesystem from one host to another?

[bayindirh]

@0x2207, if you are copying a complete rootfs, you are also copying user and group identification too. So, the target system will have the same user and group map with the source, which poses no problems.

[Vitaly Zdanevich]

grub-install: error: embedding is not possible, but this is required for cross-disk install.