Robocopy exclude directories with wildcard

Question

I am trying to backup critical folders and their contents on a daily basis so that, should my data drive fail, I have a backup of the important project files but my working data drive is much larger than my backup drive (19:6) so I would like to restrict the backup to just the important files:

RoboCopy %Source% %Dest% *.* /s /xo /purge

works; the /xo is to speed up the backup by skipping over files not modified (necessary as it would take more than a day to backup 4+ TB of data) and /purge ensures the backup drive doesn't have copies of files I no longer need.

The problem is that there are files in folders named QA that I never want to keep backups of, so specifying /xd QA should skip over these files... but the naming isn't consistent, sometimes it's QA, other times QA_v2 (or 3 or 4) other examples include dates like QA_20160708. I have searched posts like this one that seem to say it's possible to use a wildcard but all combinations of:

RoboCopy %Source% %Dest% *.* /s /xo /purge /xd "*QA*"
RoboCopy %Source% %Dest% *.* /s /xo /purge /xd *QA*
RoboCopy %Source% %Dest% *.* /s /xo /purge /xf "*QA*"
RoboCopy %Source% %Dest% *.* /s /xo /purge /xf *QA*
RoboCopy %Source% %Dest% *.* /s /xo /purge /xd QA
RoboCopy %Source% %Dest% *.* /s /xo /purge /xf QA

still copy a folder called QA_v2 in %Source%.

Is there a reliable way to skip folders and subfolders of folders that contain a string with wildcards? It might be important (or not) that I am using a batch file as a scheduled task.

I could do this with a python script using os.walk but shutil.copyfile is really slow compared to RoboCopy so this would be an absolute last resort.

Solution 1

I'm sorry, but according to the documentation at https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/robocopy:

/xf <FileName>[ ...] Excludes files that match the specified names or paths. Note that FileName can include wildcard characters (* and ?).
/xd <Directory>[ ...] Excludes directories that match the specified names and paths.

So this specifically means that wildcard can be used in the /xf flag but not in the /xd flag.

Solution 2

You can see the following links and try it may be one work for you as PETER suggested that /xd don't accept wildcards as per documentations it sounds like that but I think nothing is impossible every problem have a solution

U can try it with GUI mode if robocopy GUI or any new version of robocopy or RICH COPY tool alterative command or tool I will try to search it for you more

https://docs.microsoft.com/en-us/previous-versions/technet-magazine/cc160891(v=msdn.10)?redirectedfrom=MSDN

https://docs.microsoft.com/en-us/previous-versions/technet-magazine/dd547088%28v%3dmsdn.10%29

https://www.gurusquad.com/GSRICHCOPY360?gclid=EAIaIQobChMI-srAw8yW6AIVlhePCh2JiwFNEAAYASAAEgJUh_D_BwE

Meanwhile u see the following discussions and advive links for robocopy and try some of them if any work for you

https://stackoverflow.com/questions/14511537/how-to-exclude-subdirectories-in-the-destination-while-using-mir-xd-switch-in

https://serverfault.com/questions/304896/wildcard-directory-exclusions-with-robocopy-weird-case

https://social.technet.microsoft.com/Forums/windows/en-US/3c20df97-b0a3-440b-9170-97d80a54145f/robocopy-wildcard-xd?forum=w7itprogeneral

https://www.winvistatips.com/threads/robocopy-xd-switch-with-wildcards.757807/

https://stackoverflow.com/questions/53087761/robocopy-xd-ignores-a-list-of-directories

https://social.technet.microsoft.com/Forums/ie/en-US/1094222a-ddfa-4357-9472-218816a2307c/robocopy-version-xp010-excluding-multiple-directories-using-xd?forum=winserverfiles

Using robocopy and excluding multiple directories

https://community.spiceworks.com/topic/514474-robocopy-xd-from-text-file

hope any of link tools references work for you if any worked do tell me in comment which one worked for you if not feel free to ask further queries and if worked then don't forget to vote the answer and accept the answer

Comments

[Alex]

Easier solution: DeltaCopy + option --exclude=*QA*

[user15507]

/xf with * wildcard does not work It refuses to exclude.

[Michael Stimson]

Thanks for the links, I wrote my own utility in python using shutil with an (FullPath, Dirs, Files) in os.path.walk which made it easier to decide if the string was present in os.path.basename(FullPath).upper() then os.path.getmtime if the file exists in the output to see if the from file is newer than the existing file. As it turns out shutil.copyfile geeksforgeeks.org/python-shutil-copyfile-method is approximately as fast as robocopy. This ties in with my other question superuser.com/questions/773090/…