Verify file integrity by comparison with original file

Question

I'm copying 1TB of a FreeNAS Server to a USB External Drive, I'm doing it with "cp", and after the copy has finished, I want to compare the files from original files to the copy one.

Is this possible?

Solution 1

You can use diff to compare two file hierarchies:

diff -qr /path0 /path1

Flag -r asks to walk directories recursively, while -q asks to just print a statement when files differ, not the detailed differences. This command prints nothing and exits with status 0 when the the directories are identical.

If you would like a message for each comparison, not just those that fail, use diff -qsr.

Solution 2

On the source drive use find, xargs and md5sum:

find . -type f | xargs md5sum > sums.md5

copy that file to the destination machine (if different) and there do:

md5sum -c sums.md5

The advantage of this is that if you could run the check on the server, which is normally much faster than reading (or comparing), over the network. Not having to have the files "next" to each other means you can (re-) run the check at a later date as well, when the originals might no longer be available.

Comments

[André M. Faria]

It`s possible to make it recursively folder to folder? 'cmp /path /copy'

[unfa]

What about the I/O cache? Could checksumming the files give a false positive if the files are being read twice from different locations on the same system? Theoretically the I/O cache could be the source of the data used to produce the md5 sums in such case, not the actual data read from the disk. - so it would fail to detect any corruption that happened on the disk. Am I right?