How do I transfer 148GiB files from an Android phone to a computer?

Question

Please help, I need to transfer 148GiB worth of data from my Android phone to a computer.

My phone is Oppo A96 and it has run out of space:

It is filled screen recording videos, and they collectively take 148GiB of space, and I am going to move the files to my 4TB HDD (3725.29 GiB capacity), so that I can free up the space, and I can edit and compress the videos before I post them to my Youtube channel.

Of course I have a USB data cable, so I inserted one end of the cable to the phone and the other to a USB 3.0 port on the computer, to connect the phone to the computer, and selected "Transfer Files" in the prompt, and started copying the files using Windows Explorer:

It is taking ages.

It takes more than 4.5 hours to complete the transfer, I have cancelled the operation, I can't wait 4.5 hours just for the move operation to complete, and even in the unlikely case I could, by the time the operation completes I wouldn't be able to access the computer, it is some "technical difficulty" I would not go into here.

So I opened Task Manager and found:

Windows Explorer only moves the files at a miserable 37.1 MB/s...

The theoretical transfer speed of USB 3.0 is 4.8 Gbit/s:

4.8 * 1000^3 bit/s
600 * 1000^2 byte/s
600 * 1000^2/1024^2 mebibyte/s
572.20458984375 MiB/s

And the performance of that particular HDD:

How do I speed the process, to make it as fast as possible?

If it is a normal drive I would use FastCopy, but here I am accessing an emulated file system through a USB cable, so I don't know if it is the most efficient solution.

In case the contents of the files are important, about half of them are JPEG screenshots, all of these screenshots are less than 1MiB in size. The other files are MPEG-4 screen recordings that comes in all sizes, but most of them are between 0.5 - 5 GiB. And there are 95 pictures and 100 videos, for a total of 159636475587 bytes (148.67 GiB).

---

All the options involving Blue-tooth, Wi-Fi and cloud storage can only be slower than the physical connection, they all have a narrower bandwidth than USB 3.0.

---

Just checked, the USB interface of the phone is Type-C source, I don't know if it supports USB 3.0 or not though, it is not stated. As for the cable, I don't know, maybe I will buy a USB 3.0 cable for the phone.

Solution 1

According to specifications at GSMArena, your phone only has a USB 2.0 port. The highest operating mode in common between a USB 3.0 port on one end and a USB 2.0 port on the other end is still just USB 2.0 "High Speed" at 480 Mbps – your computer's USB 3.0 port cannot imbue the phone with more speed.

The practically achievable USB Mass Storage data transfer speed in this mode is around 42 "MB"/s. Your phone is an MTP device, not a mass storage device, but the calculations would be similar. Waiting is still probably your best option (802.11ac Wi-Fi could in theory exceed that speed, but in practice not by much).

(Also, if your phone came with a Type-C cable, it's probably also just a USB 2.0 cable made primarily for charging and only occassional data transfer – they won't bundle a more expensive USB 3.x-compliant cable with a USB 2.0 device. If the "computer" end of the cable is Type-A, you can visually inspect it – a USB 3.x cable would have an additional row of 5 pins hidden deep inside.)

Your phone seems to have a microSD storage slot – use it to move all files to a new SD card, then use an SD card reader to move them to the computer. While this would take more time in total, you no longer need continuous access to the computer for the entire duration; only the SD card needs to be left there.

Your phone also supports USB On-The-Go, which allows you to directly connect a USB disk to the phone and move files to it. (It might not be able to provide enough power for a 2.5" HDD, but you should be able to use a portable SSD or a high-capacity USB stick.) Later you can move the files from the USB SSD to your computer in a few minutes.

Solution 2

If both the computer port and the phone has USB3 the problem is maybe not related to USB but the speed of the flash memory.

If the flash memory is not fast enough at reading the data it won't help if the transfer speed over the cable is fast.

Solution 3

I found that using a FTP server on my phone and transfer the files via FTP worked much faster than USB (because it's USB 2.0 with horrible MTP). And all new files also showed up instantly which they often enough don't via USB. Not sure if this is all related to my P30 Pro, but my Axon 7 had similar issues.

I'm using SwiFTP works great.

I can get transfer rates up to 125 MB/s with this over WiFi.

Solution 4

about half of them are JPEG screenshots, all of these screenshots are less than 1MiB in size

This is a significant contributor to the issue.

There is overhead with each individual file that you write to a disk involving looking for space on the disk, updating the file tables, etc. (I'm sure someone will chime in with more details.) The more small files you transfer the more time it will take because the more general file system overhead you will incur.

For example, create 150GB of small 1MB files in TestDir1, then compress them (and added some more) to create 150GB .zip file in TestDir2. You would find that copying TestDir1 to TestDestination would take more time than copying TestDir2 to TestDestination simply because you're incurring the file system overhead ~150,000 more times for TestDir1 than for TestDir2 even though the amount of bytes being copied is essentially the same.

Solution 5

As said, if your phone only supports USB2, that's all you're going to get.

Your options as I see it:

Wait for windows.

Try linux. (It has less bloat/overhead and generally handles files better - but if you're on USB2, that's all you'll get)

If you can connect a disk directly to your phone via usb C, then it could pay to buy/borrow a powered external HDD enclosure, and connect your backup disk directly to the phone that way.

The suggestion of using an SD card is also valid, and assuming your phone has proper support for it, can transfer at up to 90MB/s. Then with a USB3 to SD device, can transfer off equally fast.

I would find an app that can compress all your files into one (linux tar for example) then you're moving one contiguous file rather than many smaller ones. However this obviously doesn't work if you're out of space.

View more solutions

Comments

[Admin]

37.1 MB/s, that is Pretty Darn Fast. We are stuck in a world of "Hurry-up Clock!" I wish I could read that fast. You need to roll with the punches. Copy or Move a little at a time next time, don't wait until it is jammed packed. Sorry, I have no better answer(s), I think nobody does for this. Just set it to copy and then take a nap. ;-)

[Admin]

Does your phone support USB 3.0? (Even Type-C ports are not necessarily 3.0 ports.) Does your cable support USB 3.0? (It needs to physically have the additional connections.) Your 37 MB/s sounds a lot like average USB 2.0 speed to me (around 400 Mbps not including overhead), and a faster port on one end won't make a 2.0-only port on the other end faster at all.

[Admin]

Try a third-party copier. The fastest I know is FastCopy.

[Admin]

Or TeraCopy. But what probably is going to be faster, is by using the android app "File Manager Plus" from File Manager Plus. It can make network copies. So share a folder locally, and copy over wifi. I can get very fast speeds that way and it runs on the phone, not the computer so may be faster too.

[Admin]

@user1686 37.1MB/s seems pretty fast for USB 2 once you count all the overhead. It's also faster than what I've seen off my phone.

[Admin]

I am moving even larger volumes of data regularly and my answer is: you can reduce the process to a few minutes by proper scheduling. Start the transfer before going to bed and find the finished task once you are back.

[Admin]

Writing a lot of files to a HDD will definitely not yield 160 MB/s as you see in the "sequential write" tests. It will be somewhere in between that and the much lower "random write" tests (2-3 MB/s), the exact value depending on the size of the files, the filesystem in use, possibly how full the drive is, and a lot of other parameters. Even though the USB 2.0 interface of your phone is probably the most limiting factor, the HDD is going to be the next one, at best you'll be able to get x2, probably less. And that's of course if the HDD is connected via SATA or USB3 rather than USB2 as well.

[Admin]

You could have waited out the 4 hours and 30 minutes faster than writing this question and interacting with Internet strangers. I really hope you started the transfer and went to sleep so it could be done by the time you woke up.

[Admin]

I think it isn't necessary to mention it, I have already moved all the files from the phone to the computer, but I will leave this comment here to prevent people from leaving a comment like the last comment.

[Admin]

For the future, install syncthing (syncthing.net/downloads) on your android phone and your PC, and you'll have your android files synced (and thereby backed up) automatically, so you'll never have to transfer a big chunk at once anymore. Beware of data and battery usage during the initial sync, though.

[Admin]

@ΞένηΓήινος what was your solution? Can you post it as an answer?

[Admin]

"Start the transfer before going to bed and find the finished task once you are back" - totally this. Plus, don't leave it so long that your storage is nearly full next time.

[Admin]

Strange times. "OMG, I have only 14 GB storage left on my phone, what should I do? Please help!" ;-) When I was young I had a Win/Linux dual boot on 500 MB... and I had to walk 10 feet across shag carpet to change the TV channel!

[Admin]

Strange times, indeed, @Peter-ReinstateMonica. I remember staring in awe at the directory listing showing nearly 10 MB of free space on that PC XT! How many of these DSDD floppies can that store???

[Admin]

.... this guy. What can we do with 16kb? Start a revolution.

[Admin]

I’m voting to close this question because new answers to this question cannot be actually helpful, there is nothing I can do in this situation other than waiting, but new answers just keep coming...

[Admin]

I think it was near year 2000 when I backed up a 4GB disk sector by sector. It took 2 hours. So actually that's not a long time (as others said, too). Also your phone will also be charged while copying, most likely. So what's the problem with time? How much time did it take to fill up the phone like that?

[Admin]

GUI transfers will always be slowest - either use adb via adb pull -p /sdcard D:\MyPhone or Robocopy (latter may depend on how the phone is mounted)

[Admin]

"Your phone also supports USB On-The-Go" -- no, it does not, USB OTG only applies to micro USB, this phone is USB C. What you wanted to say is "Your phone has a USB C connector so you can directly connect a USB disk..." Also, a data role swap might be necessary which, regrettably, is called OTG under Settings > Connected devices. But that's a misnomer from Android.

[Admin]

This problem is hardware caused and completely irrelevant to software, so changing operating system won't mitigate the problem even slightly.

[Admin]

Yes, you might notice the bit where I said "if you're limited to USB2..." But also I've seen the same file transfers, performed faster on a live linux distro, than on the baremetal windows install.

[Admin]

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

[Admin]

I meant this "answer"...

[Admin]

In what way is it irrelevant to the question? Your question does not include the information about the disk on the reading side only on the writing wide.

[Admin]

It's a perfectly reasonable answer to the question "how can I speed up the transfer?". This answer is simply saying that if the bottleneck is read speeds on the flash memory, then you can't. "You can't" is an acceptable answer.

[Admin]

People use to say you knew you had a Vax (minicomputer) if you measured compile times in weeks.

[Admin]

@chx, so you are saying that the answer calls it OTG, the Android menu calls it OTG it is quite clear what is being suggested, but for some technical obscure reasons you object to this term? ;)

[Admin]

Since when is the USB C specification a technical obscure reason??

[Admin]

@chx Since the general public doesn't use the term. Not all USB-C ports are dual-role ports (DRP). That the phrasing "this device supports USB-OTG" remains commonplace over "this device supports dual role / has a DRP" is obvious. Since it remains unambiguous to say a USB-C device "supports OTG", what's the harm?

[Admin]

Just two problems off head, there might be many more: 1. you buy an OTG adapter and then it's micro USB and you can't even plug it in. 2. you might think a special OTG cable is needed when that's not necessary. Finally, is there really any phone with a non-DRP USB C port?

[Admin]

Connecting a gigabit LAN adapter is also an option, although obviously limited by the speed of USB 2.0. I did this combined with a free FTP app the other day.

[Admin]

There are only 95 of those images. It is not a significant contributor to the issue.

[Admin]

tar can be used on the fly without requiring any more disk space, and does wonders with many very small files, although if the files are screen recordings it may not be the case

[Admin]

I think you always buy an UTG/OTG adapter for your phone. I had one that worked with Samsung, but would not with Huawei, so I bought another one that worked for Huawei.

[Admin]

Usually reading from the built-in flash memory is much faster than USB2 is, unless the device uses exceptionally slow flash memory.

[Admin]

@U.Windl Yes, it is unlikely but possible. But we also don't know what else is happening on the device at the same time. Maybe many apps are competing about doing stuff with the memory. It is not so uncommon to have many apps running simultaneously these days.

[Admin]

Also syncthing can be an option, in case you just want to keep synchronized media files between 2 or more devices.