Shell script to compare directories recursively

Question

I have a file server backup on an external hard drive a few months old for a file server that went down since then. Most of the data was recovered onto a temporary file server thats been in use since then but there are some inconsistencies.

I am going to mount the external and rsync it with the current data to it but first I need to establish files that have gotten updated on the newer copy.

I can do diff -r -q /old/ /new/ to obtain this, I am trying to get better at scripting so I am trying to write something that will rename the old file to filename.old whenever diff returns a difference.

Solution

So after checking, I wasn't able to find an option in diff to only output the filename differences so we'll just work with what diff outputs.

If diff finds files that differ, the output is something like this:

Files old/file and new/file differ

Since all your bash script would be doing is renaming the changed file from the old directory, we want to extract old/file from this output. Let's start by only displaying lines like Files...differ (as other lines may be produced):

diff -rq old/ new/ | grep "^Files.*differ$"

Now you'll only get lines like the one shown before. Next step is getting the filename. You can do this with awk by adding something like awk '{print $2}' as another pipe but if the filename itself contains spaces, awk will break up that as two separate strings. We'll use sed instead:

diff -rq old/ new/ | grep "^Files.*differ$" | sed 's/^Files \(.*\) and .* differ$/\1/'

Now this will produce a list of files that have changed in the old directory. Using a simple for loop, you can now rename each of the files:

for old_file in `diff -rq old/ new/ | grep "^Files.*differ$" | sed 's/^Files \(.*\) and .* differ$/\1/'`
do
   mv $old_file $old_file.old
done

And that's it!

edit: actually, that's not all. This loop still fails on files with spaces so let's muck with it a bit. for will try to produce a list separated by a space by default. Let's change this to use newlines instead:

OLD_IFS=$IFS
# The extra space after is crucial
IFS=\

for old_file in `diff -rq old/ new/ | grep "^Files.*differ$" | sed 's/^Files \(.*\) and .* differ$/\1/'`
do
   mv $old_file $old_file.old
done
IFS=$OLD_IFS

This temporarily replaces bash's default separator ($IFS) to a newline and puts it back after it's done with the loop so you don't split by space.

Comments

[Brandon]

Thank you! I knew there was a way to work with the output using some simple regex, I'm just still getting used to piping commands.