How can I quickly decompress a 1TB (5 million file) 7-Zip (7z) file stored on a NAS

Question

There is a 7-Zip (7z) file (size: 1TB with about 5 million files inside) in a NAS device running in iSCSI mode. The 7-Zip file was compressed by the “only store not compress” mode.

When the files are in normal hard disk drive, the compressing speed is so fast(10-20 M/s). But when the 7-Zip (7z) file is in NAS (iSCSI), why the decompressing speed is just less than 1M/s?

Can anyone help that how I can decompress the huge 7-Zip (7z) file on NAS fast (at least reach half speed of compressing)?

Solution

If I were to take a guess, you are using your computer to decompress/unarchive the file on the NAS. This means you are transferring parts of the file from the NAS to your computer, which is a major bottleneck.

You would save time by transferring the archive to your computer first and then decompressing it.

Comments

[SparedWhisle]

what is the target of the decompression? if you are decompressing both from and to the ISCSI device, it could be a problem. maybe try decompress from the ISCSI device, but to your local hard drive.

[yeyl]

thank you for you attention. I decompress the file from iscsi to iscsi. My main purpose is to move the 5million files from local hard disk to iscsi. But the speed is very very low if I move the files in folders. So I compress the files into a 7z file in local hard disk, so that the speed of moving to iscsi is 50 times faster than in folders. But now comes the problem: I have to wait for the long time of decompressing T_T. I'm also confused that the speed of decompressing in iscsi is slower a lot than in local hard disk.

[Daniel B]

What make and model is your NAS?

[yeyl]

Synology DS1815+

[David Schwartz]

You've just made your problem worse. Now you still have to create all those files on the NAS device, and you have to get the data from the NAS at the same time.

[yeyl]

Thank you for your advice. Yes I'm using my computer to decompress the file on the NAS. My main purpose is to move the 5 million files from local hard disk to nas. If I directly transfer the files in folders the speed is too low. So I compress the files first then the speed of transferring one file is fast. But now I have met the problem: the low speed of decompressing. Is there any ways to copy the huge numbers of files from local hard disk to nas fast?

[David Schwartz]

@yeyl You've just made things more complicated. Forget about the compressed file, that's a dead end. Go back to looking for good solutions to your original problem -- how to transfer a large number of files to a NAS device.