Transfer "freezes" when writing big files on USB drive

Question

When I try to transfer big files (1GB or more) to a USB drive (2.0, 3.0, either HDD or "pen drive") everything goes fine, the file is transferred correctly (tried md5 sum both of the original and the transferred one and they do match)...

However both at command line (mv *bigfile* /media/*username*/*name_of_the_drive*/) and using the GUI (currently running on MATE desktop of Ubuntu 15.10, but it happens on other flavours as well) when at the end of the transfer the progress bar on the dialog freezes (it just freezes when using console). Therefore, I have to manually close the dialog/terminal when I "think" the transfer is complete.

Also, when I tried to write a bootable USB drive with Make Startup Disk utility, the application faces the same issue and freezes, still keeping the "writing session" open and not allowing me to unmount the USB even when i close it. While trying another versions of Ubuntu 15.10 (I have tried several) it actually showed a dialog error saying something like:

"The files have been correctly transferred to the USB drive, however the program could not communicate to the kernel that the writing is completed."

I apologize for not reporting the exact warning, but I do remember that's the meaning of what i read.

Currently running 15.10 MATE, kernel 4.4.1 on x64 laptop machine with GTX960M and proprietary Nvidia drivers installed.

P.S.: I updated to 4.4.1 kernel to see if that could resolve the issue, since It would still do the same on 4.2.0_27. But it obviously didn't.

Thank you in advance for the help,

F.

Solution

I have actually found an answer here as in fact the issue was due to having 16GB RAM installed. The writing buffer was actually too big. I have resolved by logging in as root:

su root

then setting the dirty bytes to a set number (I guess the default setting 0 actually doesn't give any cap to how much should the writing cache be large), by typing:

echo $((16*2048*2048)) > /proc/sys/vm/dirty_background_bytes
echo $((48*2048*2048)) > /proc/sys/vm/dirty_bytes

I chose that size as a good compromise and the issue is now resolved.

Comments

[Byte Commander]

Possible duplicate of System freezes/unresponsive/unusable when copying large file to USB or askubuntu.com/q/730458/367990

[Byte Commander]

This seems like a caching problem. The system caches USB writes in a RAM buffer. Most copying tools display 100% progress as soon as all data is written to the buffer (which is fast because the buffer is in RAM), but don't close until the buffer is flushed to the USB drive (which is slow due to slow USB transfer speeds). You might want to shrink or disable the USB write cache.

[fredsat]

@Byte Commander I think the duplicate suggestion is appropriate, however, since the system was not exactly freezing, as I could do other stuff while the write dialog was sitting there frozen, I thought I would post the question anyway since I was not sure whether the issue was the same. The other suggestion...well I wish I could have found it before (although it suggests a different workaround than what I used to resolve the issue) but "Is there a good file-copying tool?" seems a little misleading when searching for my particular problem :)