How to mirror multiple sources to a single destination?

5,022

Instead of copying your many source drives to the root directory of B:\, create sub directories (B:\FromD\, B:\FromE\ etc) and use robocopy to mirror each source to the sub directories on the B: drive.

Share:
5,022

Related videos on Youtube

glenneroo
Author by

glenneroo

Updated on September 18, 2022

Comments

  • glenneroo
    glenneroo over 1 year

    I have multiple 2-3TB drives inside and connected to my working PC. The data I want to backup on each system lies in the folder "digitalized" (photos, time-lapse, etc).

    I have a NAS (PC with 8 drives in a JBOD array) with 9TB free.

    I would like to mirror each of my source drives to my backup PC:

    d:\digitalized -> b:\
    e:\digitalized -> b:\
    f:\digitalized -> b:\
    g:\digitalized -> b:\
    h:\digitalized -> b:\
    

    I manually copied of all source drives to the NAS (which took about 4 days). Here is a WinDirStat screenshot. (Full-size here: http://i.imgur.com/41YVQjz.jpg - 2 million baby jpegs, eeeeeh!)

    Now, one month later, a lot has been deleted, edited and added on the source drives so I need to synchronize everything.

    I also have a kazillion tiny useless catalog preview files (from Lightroom) which I would like to ignore. Here is what I have come up with so far:

    robocopy X:\DiGiTaLiZeD B:\ *.* /zb /e /purge /eta /xd *.lrdata /xf *.lrprev

    Replace 'X' with each drive letter and repeat. Put into a batch file which runs whenever NAS is pingable.

    Problem:

    Each time robocopy runs, /purge sees that 75% of the files located on dest do NOT exist on source (D:) because that data is from E:, F:, etc., so it begins to delete everything else! BAD ROBOCOPY BAD! :(

    Any clever ideas? It doesn't have to involve robocopy but I thought it would be the simplest since it's included with Windows.

    • Dave Lucre
      Dave Lucre almost 11 years
      When I want to mirror files from one location to another, I use robocopy with the /MIR switch. You can probably exclude your tiny annoying files by simply using the same /xd and /xf switches you already use. Generally: robocopy source destination /MIR /R:1 /W:1
    • glenneroo
      glenneroo almost 11 years
      Same problem. BTW /mir is the same as /e + /purge which is what I was doing.
    • glenneroo
      glenneroo almost 11 years
      Actually no, but that's a really good point which would probably solve my entire problem, wouldn't it?
    • Dave Lucre
      Dave Lucre almost 11 years
      If you'd like, I can post that as the answer?
    • glenneroo
      glenneroo almost 11 years
      Sure. Admittedly it isn't my ideal solution, but the alternative is to create a catalog of all sources before synchronizing, which will probably require some 3rd party software (which is worse;).
  • Synetech
    Synetech over 10 years
    Which would mean B:\FromD\images, B:\FromE\images, B:\FromF\images, B:\FromG\images, B:\FromH\images, B:\FromD\videos, B:\FromE\videos, B:\FromF\videos, B:\FromG\videosB:\FromD\misc, B:\FromE\misc, B:\FromF\miscB:\FromD\panoramas, B:\FromE\panoramas and so on. This causes all of the store to become extremely fragmented with related files scattered everywhere. Besides having a backup, the whole point to putting them all in one location is so that they can be organized, sorted, de-duped, and cataloged which this method prevents.