Why is there a discrepancy in disk usage reported by df and du?

Question

I have a Linux CentOS server, the OS+packages used around 5GB. Then, I transferred 97GB data from a Windows server to two folders on this Linux server, after calculated the disk usage, I see the total size of the two folders is larger than the disk used size.

Run du -sh on each folder, one use 50GB, the other one use 47GB

But run df -h, the used space is 96GB. (50GB + 47GB + 5GB) > 96GB

Is there any problem? Those two folders contain lots of files (1 million+). Thanks.

Solution 1

This page gives some insight on why they have different values, however it seems to suggest that your du size should be the smaller of the two.

df uses total allocated blocks, while du only looks at files themselves, excluding metadata such as inodes, which still require blocks on the disk. Additionally, if a file is deleted while an application has it opened, du will report it as free space but df does not until the application exits.

Solution 2

When du is larger than df, the usual reason is "sparse blocks": if a program doesn't actually write to a disk block but instead seeks past it, it gets a zero pointer in the inode's block allocation map and no actual disk space is reserved for it. If you later write to it, an actual disk block will be allocated and the map will be changed to point to the new block.

Comments

[xenoterracide]

this explains why df claims the disk is full... when it's not.

[Barmar]

I don't think sparse blocks are counted in du.

[mattdm]

FWIW, in Linux coreutils (as of 4.5.8, way long ago), there's an --apparent-size option which takes sparse blocks into account.