USB transfer speed "logarithmically" decreases. Why and can it be improved?

Question

I have an external hard drive. Just today I tried to copy a bigger file (it was film of ~230 MB) and at first it rushed up until ~70%. There is started decreasing.

• At first it started at around 56 MB/s
• Then it rapidly dropped to 23 MB/s (File transfer was 70% complete)
• Then it slowly started decreasing until it was around 2 MB/s (File was ~90% complete)
• When it finished the transfer it was slightly above 1.5 MB/s.

---

To describe it graphically: If you drew a curve of the decrease it would probably resemble the graph of a logarithm function

---

So, what I'm really asking is: "Why does this happen?" and "Is there a way around it?"

Thank you!

Solution

Possible reasons:

• Most operating systems will not immediately write data to disk, but buffer or "cache" it in RAM and wait to actually write it. This is because
  1. it's more efficient from a speed standpoint to ship a bunch of data to a drive at once and
  2. NTFS/ext3/ext4 journaling works for efficiently if it logs a bunch of transactions instead of individual sector writes. So a small amount of initial write data will be reported as finished and this will occur quickly, with the actual write to the disk being done later. As the writes build up the buffer files and then speed matches actual physical write speed or medium speed. While IIRC on Windows, this is disabled by default for removable drives, the setting may not be that way on your system.
• Most hard drives have an amount of RAM as a buffer. This could be a second layer of "cache" between you and the hard disk. I'm not sure if hard drives use it for writing, but maybe some do.
• The hard drive may be fragmented at the point the file transfer dropped speed.
• Assuming Windows, I believe "I/O priorities" is a feature introduced in Windows Vista and continued on through 7. It may be I/O of the controlling process was deprioritized in the kernel after it ran a bit in favor of a more recently used or interactive process. This is just a guess and I'm not sure at all how this feature actually works.
• Other USB communication on that same USB root hub could cause bus contention and slowdown.

Comments

[Daniel R Hicks]

It's possible that the % complete display was simply being "optimistic". The algorithms for this tend to not be particularly precise. It's also likely that on-board cache confused the algorithm.

[sancho.s ReinstateMonicaCellio]

possible duplicate of Why does the speed decrease while copying from an external harddrive?

[omninonsense]

Ah, thanks. So, there's not anything I can do about it. And yes the Windows write-cache was enabled. Well, thanks, though!

[LawrenceC]

Defragging your external drive might help.