Does any linux disk format support transparent folder compression? (Like NTFS)

Question

I feel like I could use transparent, seamless folder compression.

One way to achieve this, I think, is to compress normally, and mount an archive with Nautillus. AFAIK. you can run programs from the mountpoint at a performance hit. My only worry is that the mountpoint won't match the original file path that folder existed at.

With the NTFS "Compress this folder to save space" feature, all programs and files can do all file operations though the mount. Including software that really cares were things are, like linked libraries and hundreds of application assets.

Ideas? I Currently use ext4 like a nobody but am open to play with more exotic file systems.

Solution 1

I can't add formatting to comments so I'll just reiterate what I commented in the other answer. You can pick either Btrfs or ZFS, but ZFS should only be used in multi-disk scenarios because of the way ZFS is designed:

Technically you can do deduplication and compression.

But there is no protection from corruption since there is no redundancy. So any error can be detected, but cannot be corrected. This sounds like an acceptable compromise, but its actually not. The reason its not is that ZFS' metadata cannot be allowed to be corrupted. If it is it is likely the zpool will be impossible to mount (and will probably crash the system once the corruption is found). So a couple of bad sectors in the right place will mean that all data on the zpool will be lost. Not some, all. Also there's no ZFS recovery tools, so you cannot recover any data on the drives. You cannot use the standard recovery tools that are designed for NTFS, FAT32, etc either. They don't work correctly.

So what does all of this mean? It means that you run the risk of everything being just fine, and then suddenly (and without warning) all of the data is irretrievably lost.

source

Solution 2

ZFS. These days (after Solaris) it is (pretty much) open source, but development was delayed due to license issues when Sun (Solaris) stopped and Oracle bought it. Someone on another forum said it runs only on 64 bit Intel. I suspect that information is outdated. Information on http://zfsonlinux.org/.

EXT4. I think it is planned, but not implemented atm. Would be a nice feature though.

BTRFS. I could say a Linux variant of ZFS-alike filesystem you might consider. https://btrfs.wiki.kernel.org/index.php/Main_Page (I would choose it)

Comments

[MariusMatutiae]

ZFS does, you should read here, serverfault.com/questions/617648/…

[ThorSummoner]

Duplicate ofhttp://serverfault.com/questions/617648/transparent-compre‌​ssion-filesystem-in-‌​conjunction-with-ext‌​4, thanks to MariusMatutiae

[ThorSummoner]

I've started using zfs via ZOL on debian, and I love it <3, warning all the experimental stuff people say not to use... still doesn't work very well, but plain old compression is doing wonders! and I learned the value of snapshots!

[bviktor]

Just a note, with single disks (or partitions, even) you shouldn't use ZFS.

[ThorSummoner]

@bviktor do you have any citations? I know you can't raid6 one drive meaningfully, but I am under the impressions that even for a single disk, zfs does good things (specifically built in compression/paritity/logical volume management/snapshots/scrubbing. serverfault.com/a/851557/210994 comments that raid-z woudln't work, that obvious..., a redditor suggests settings copies to 2 or a greater number can even offer block redundancy on a single drive, that you'd not get from soemthing like ext4 AFAIK

[bviktor]

There you go: forums.freenas.org/…

[bviktor]

Also, I'd like to point out that Red Hat/CentOS 8 won't have Btrfs anymore, so essentially, RHEL 8 won't have official support for any kind of transparent compression at all, which is quite ridiculous.

[hochl]

This is actually an extremely important point. I have a set of rotating external block-level encrypted storage disks for backups and tested ZFS on one of these. After about 8 months the whole disk suddenly became inaccessible. Good that it was just one of several backup disks, but imagine it were the only backup and you needed it.

[YudhiWidyatama]

by 'multi-disk scenarios' you meant raid, (excluding raid 0), right?