Parallel copy using xcopy

Question

I need to copy multiple directories from one location to other. So, there are going to be multiple xcopy statements, one after another.

The number of files in each of the folders is huge. Is there some way by which I can run these xcopy statements in parallel? One option I can think of is- call each xcopy in a separate batch file, and call those batch files using @start instead of @call.

Is there any other alternative?

Solution

You can start xcopy directly, like so start xcopy [parameters]. This allows you to run many xcopy instances in parallel.

By the way: Have you tried robocopy? It's included in all recent Windows versions and offers more options (and sometimes performance) than xcopy.

But in general copying multiple directories in parallel is slower (at least when you copy from a drive to another drive), because it'll force the source drive to seek between the parallel copy jobs instead of reading files sequentially.

Comments

[Pramod Karandikar]

I haven't tried using Robocopy. It's using /MT[:number] right? Actually, I need to do this on Windows XP SP3, so need to get Robocopy explicitly. However, I did not fully understand its /MT usage. Could you please elaborate?

[Simon]

robocopy by default already uses 8 threads to do some of its operations in parallel. This means that you could use robocopy to copy all files of a directory in parallel. If you specified /MT:100 for example, robocopy would copy up to 100 files in parallel. I wouldn't recommend doing that - 8 is plenty. So if you do indeed chose robocopy instead of xcopy, just call it sequentially. It'll copy the first directory in parallel, then the second directory and so on.

[kayleeFrye_onDeck]

Can confirm Simon's comment. I measured a ROBOCOPY from a hefty remote directory with build output using /MT:%NUMBER_OF_PROCESSORS% versus /MT:100 and the latter performed on average 66% faster on a quad-core i5-6500. I wish I knew about this level of flexibility a long time ago!