what is the fastest way to copy all data to a new larger hard drive?

Question

I was certain this would have been covered before, but I cannot find an answer amongst all the almost-duplicates that come up; sorry if I've missed something obvious.

I have a full 320gb disk inside my machine, a new 1tb disk to replace it, and a USB 2.0 chassis.

It is only data on a single partition, no OS/apps involved, and the old drive will be kept somewhere as backup (no secure wiping etc).

The simple option would be to put new disk in USB chassis, copy files, then swap them over.

But for USB pen drives, reading is around 4x faster than writing. If the same is true for a USB SATA chassis (is it?) then it would be significantly faster to swap the drives first and read from the old drive over USB, right?

Then the other consideration is that copying lots of files is usually slower than a single file of equivalent size. Is Windows 7 smart enough to do everything in a single lump like that, or is there specialised software that should be used instead?

(Even if SATA->SATA copying is faster than involving USB, knowing what to do when it isn't an option is useful information.)

Summary:

• Does a USB SATA chassis suffer from a read/write inequality? (like a USB pen drive does, but unlike a direct SATA connection)

• Can Windows 7 do sequential access? (I can't find confirmation if Robocopy does this.) Or is it necessary to use a bootable CD/USB with something like Clonezilla to achieve sequential copy speeds?

Solution 1

Does a USB SATA chassis suffer from a read/write inequality? (like a USB pen drive does, but unlike a direct SATA connection)

No, not really. The read/write speed inequality is a result of the storage media used (cheap flash memory), not the USB interface.

You may see a slight difference in speed on reads and writes, but unlike with a thumbdrive the dominating factor will likely be file system overhead and fragmentation, not the storage media itself.

Can Windows 7 do sequential access? (I can't find confirmation if Robocopy does this.) Or is it necessary to use a bootable CD/USB with something like Clonezilla to achieve sequential copy speeds?

You won't need to do sequential access. In my experience, copying data over a USB interface can reasonably easily achieve speeds of a few tens of megabytes per second, if the storage media itself can sustain such speeds (which a hard disk drive can, unless the data on it is fairly heavily fragmented). That means that copying ~300 GB of data will take no more than a few hours. (As a data point, I recall that when I primed by backup drive, copying ~1.5TB of data took about one full day, for an overall average of ~17 MB/s, but that also involved a lot of seeking which is the real killer for performance.) It may be longer than that if you have lots and lots of very small files (on the order of a few tens of kilobytes), but given the amount of data you are talking about I find it unlikely that this would be the case.

It won't be a problem using the computer while the data is being copied, as long as you don't touch the data on the drive that you are copying from. If you want to work with the data while it's being copied, just copy those files first to a separate folder on the new drive and work with those copies, then copy those modified copies over the originals (on the new drive) when it's finished.

Or simply let the whole thing run over night.

Solution 2

If you connect both disks inside your machine by SATA you will have a quicker transfer than USB 2.0 but, due to probably using the same SATA processing chipset, you won't get the full SATA-to-SATA performance. It will be considerably quicker than by USB, however.

Comments

[slhck]

I'm sure the eight hours won't matter that much. Please don't roll-back the changes again. I've posted your exact words as an answer below. If you want to post the same answer again, just leave me a comment and I'll delete my placeholder.

[sawdust]

"Can Windows 7 do sequential access" -- You're misusing the word "sequential". The typical file, e.g. a document or video or audio recording, is written and read sequentially. You are probably intending to mean "raw block access", which would allow a sector by sector access and bypass any filesystem overhead.

[SUPER user]

If I had a spare SATA port I would be doing that, but I don't. Taking out the only other non-OS drive would render the machine unusable (it has apps/data I want to use), so unless that was definitely going to cut the time down to under an hour, I'd rather be able to still use the computer during the copy period.

[SUPER user]

The new drive is a WD Blue 5400 - I found this page suggesting it can manage upto 113MB/s - which is about 45 minutes for 300GB - but that is for a single large file, not lots of medium/small ones, so if there is a way to copy everything in one go then this might be an option.

[Kinnectus]

You're not leaving yourself with many options if you need to keep the other SATA ports occupied during the copy process... copying 320GB over USB 2.0 is going to take well over an hour. I copied a 320GB SATA disk to a USB 3.0 disk the other day and it took nearly 6hrs. Yes, this took many small files as well as large, but your time (given you're really only going to be limited to SATA-to-USB) is realistically going to be many hours.

[SUPER user]

If it takes 6 hours but I can still use the machine to do stuff then it's only a bit annoying, but if it takes 2 hours and I am sitting twiddling my thumbs for that time then it's a big PITA and waste of time.

[SUPER user]

I looked up the speeds for the old drive and average read is 64.1MB/s - potentially as high as 83.4MB/s - that's 60 to 80 minutes - if the data can be done sequentially (and performs at or above average). So this might be an option but I need to go find suitable software.

[Kinnectus]

From experience, that is a serious underestimate of what will actually happen. You won't get anywhere near 64mb/s with Windows running. I used an Ubuntu LiveCD to copy the USB 3.0 and the most I got was 22mb/s when a large file was copied. Expect between 10-18

[SUPER user]

Those are SATA speeds for sequential access (single block of data), not USB speeds for multiple files, but either way if they are going to be slower then it just underlines the point of not having the machine completely out of action for a long period of time.

[slhck]

Answer taken from OP.

[Hashim Aziz]

"...due to probably using the same SATA processing chipset, you won't get the full SATA-to-SATA performance". Could you elaborate on this, @Kinnectus?