How to output a list of changed files from rsync?

60,874

Solution 1

You can use rsync's --itemize-changes (-i) option to generate a parsable output that looks like this:

~ $ rsync src/ dest/ -ai
.d..t.... ./
>f+++++++ newfile
>f..t.... oldfile

~ $ echo 'new stuff' > src/newfile

~ $ !rsync
rsync src/ dest/ -ai
>f.st.... newfile

The > character in the first position indicates a file was updated, the remaining characters indicate why, for example here s and t indicate that the file size and timestamp changed.

A quick and dirty way to get the file list might be:

rsync -ai src/ dest/ | egrep '^>'

Obviously more advanced parsing could produce cleaner output :-)

I came across this great link while trying to find out when --itemize-changes was introduced, very useful:

http://andreafrancia.it/2010/03/understanding-the-output-of-rsync-itemize-changes.html (archived link)

Solution 2

Use the -n flag, combined with the -c checksum flag and the -i flag:

# rsync -naic test/ test-clone/
>fcst...... a.txt
>fcst...... abcde.txt
>fcst...... b.txt

In this example, three files have changed, based on the contents (as determined by the checksum) of the file itself. However, no file syncing is done because of the -n flag

Bonus

If you want to run chown on the changed files, parse them out with sed or similar and process with xargs, for example:

rsync -naic test/ test-clone/ | sed 's/............//' | xargs -I+ sudo chown root "test-clone/+"

Solution 3

This question is a little bit old, but I think it worth to be added:

-i is a shortcut of --out-format=%i%n%L

And %n means the filename, (section log format of man rsyncd.conf)

P.S. rsync version 3.1.0

Solution 4

Summarizing a few other answers (especially @Cychih's), you can get the list of changed files like so:

rsync --out-format='%n' src/ dest/

Which will print only the changed files, eg;

rsync --out-format='%n' src/ dest/
a.txt
bcde.txt
b.txt

You can save that to a list this way:

changed_items=($(rsync --out-format='%n' src/ dest/))
for item in "${items[@]}"; do
   echo $item
   echo $item
done

You can pipe them to another command like so:

rsync --out-format='%n' src/ dest/ | xargs open

Note that it's very common to include -acz (archive, checksum, and compress) flags as well.

Share:
60,874

Related videos on Youtube

Jason M.
Author by

Jason M.

Linux advocate, system engineer, PHP, and Go developer.

Updated on September 18, 2022

Comments

  • Jason M.
    Jason M. over 1 year

    I am using rsync in a bash script to keep files in sync between a few servers and a NAS. One issue I have run into is trying to generate a list of the files that have changed from the during the rsync.

    The idea is that when I run rsync, I can output the files that have changed into a text file - more hoping for an array in memory - then before the script exists I can run a chown on only the changed files.

    Has anyone found a way to perform such a task?

    # specify the source directory
    source_directory=/Users/jason/Desktop/source
    
    # specify the destination directory
    # DO NOT ADD THE SAME DIRECTORY NAME AS RSYNC WILL CREATE IT FOR YOU
    destination_directory=/Users/jason/Desktop/destination
    
    # run the rsync command
    rsync -avz $source_directory $destination_directory
    
    # grab the changed items and save to an array or temp file?
    
    # loop through and chown each changed file
    for changed_item in "${changed_items[@]}"
    do
            # chown the file owner and notify the user
            chown -R user:usergroup; echo '!! changed the user and group for:' $changed_item
    done
    
    • Frank Nocke
      Frank Nocke about 7 years
      → see my anwer here. Also -i for itemize, but with a few more twists...
  • MUY Belgium
    MUY Belgium almost 8 years
    sed -> cut -d ' ' -f2,-
  • MediaVince
    MediaVince over 7 years
    For a cleaner output as mentioned, rsync -zaic src/ dest/ | grep '^?c' | cut -d' ' -f2 --dry-run would only list the modified files (different checksum) , definitely a keeper, thx :) FYI putting --dry-run after the command instead of using the n option is for me a best practice
  • MediaVince
    MediaVince over 7 years
    Essentially the same as rsync -zavc src/ dest/ --dry-run without verbose
  • Cychih
    Cychih over 6 years
    Actually this answer mentioned that option
  • ThorSummoner
    ThorSummoner over 5 years
    -n --dry-run, -c --checksum, -i --itemize-changes
  • bobpaul
    bobpaul over 4 years
    I'm not following. All 3 of those files in the example are marked '>fcst' which I understand means 'being received from the remote', checksum differs, size differs, time differs. What in the output indicates that "only one file has changed"??
  • Seamus
    Seamus about 4 years
    @bobpaul: I was puzzling over the same question when I read your comment (great minds...? :)
  • JDS
    JDS over 3 years
    thanks for the clarification, @Seamus
  • Niloct
    Niloct over 3 years
    Awesome, I didn't know the ! syntax for using previous parameters... Thanks a lot!
  • Kingsley
    Kingsley over 2 years
    rsync -r --out-format='%n' src/ dest/ seemed to print every file for me.