du reports directory size as much bigger than the sum of its contents

Question

Running du -shc * in the top directory gives 110G for a particular folder, whereas running the same command inside that folder gives a total size of 11G. How is that possible?

Platform Details: OS: CentOS 6.6 x86_64 Drive type: solid-state Volume type: RAID 6 array RAID Controller: LSI MegaRAID SAS Filesystem: ext3

Solution

Most probably you have hidden files in the folder. The point is that glob * selects only files and folders that do not start with .. So, if they do they are not passed to du command. On the other hand from top directory you get size of the directory as a whole, including dot files.

To match all files in given folder, including hidden ones try (with bash)

du -shc -- {.[!.],..?,}*

or set option dotglob so that * matches hidden files too:

shopt -s dotglob
du -shc -- *

Comments

[orodbhen]

Boy, am I embarrassed. I didn't think of that. I wish there were a simpler shell syntax for including hidden files in the expansion. Especially with commands like du, where you usually want to see everything in the folder. Thanks.

[jimmij]

@orodbhen This glob is a little bit nerdy to take care of any "unusual" possibilities like files which start from two dots ..abc or which start from hyphen -abc. Most of the times you could just do du -shc * .??* or in shorter form du -shc {.??,}*. You cannot get rid of ? to not take parent directory .. into account. You can also set dotglob option in bash, so that it includes hidden files by default (I will add that to the answer).

[Gilles 'SO- stop being evil']

@orodbhen In zsh: *(D) (or make the inclusion of dot files the default with setopt glob_dots).

[HDave]

Been using Linux command line for 12 years, never even heard of the shopt command!