total size of group of files selected with 'find'

Question

For instance, I have a large filesystem that is filling up faster than I expected. So I look for what's being added:

find /rapidly_shrinking_drive/ -type f -mtime -1 -ls | less

And I find, well, lots of stuff. Thousands of files of six-seven types. I can single out a type and count them:

find /rapidly_shrinking_drive/ -name "*offender1*" -mtime -1 -ls | wc -l

but what I'd really like is to be able to get the total size on disk of these files:

find /rapidly_shrinking_drive/ -name "*offender1*" -mtime -1 | howmuchspace

I'm open to a Perl one-liner for this, if someone's got one, but I'm not going to use any solution that involves a multi-line script, or File::Find.

Solution 1

The command du tells you about disk usage. Example usage for your specific case:

find rapidly_shrinking_drive/ -name "offender1" -mtime -1 -print0 | du --files0-from=- -hc | tail -n1

(Previously I wrote du -hs, but on my machine that appears to disregard find's input and instead summarises the size of the cwd.)

Solution 2

Darn, Stephan202 is right. I didn't think about du -s (summarize), so instead I used awk:

find rapidly_shrinking_drive/ -name "offender1" -mtime -1 | du | awk '{total+=$1} END{print total}'

I like the other answer better though, and it's almost certainly more efficient.

Solution 3

with GNU find,

 find /path -name "offender" -printf "%s\n" | awk '{t+=$1}END{print t}'

Solution 4

I'd like to promote jason's comment above to the status of answer, because I believe it's the most mnemonic (though not the most generic, if you really gotta have the file list specified by find):

$ du -hs *.nc
6.1M  foo.nc
280K  foo_region_N2O.nc
8.0K  foo_region_PS.nc
844K  foo_region_xyz.nc
844K  foo_region_z.nc
37M   ETOPO1_Ice_g_gmt4.grd_region_zS.nc
$ du -ch *.nc | tail -n 1
45M total
$ du -cb *.nc | tail -n 1
47033368  total

Solution 5

Recently i faced the same(almost) problem and i came up with this solution.

find $path -type f -printf '%s '

It'll show files sizes in bytes, from man find:

-printf format
    True; print format on the standard output, interpreting `\' escapes and `%' directives.  Field widths and precisions can be spec‐
    ified as with the `printf' C function.  Please note that many of the fields are printed as %s rather than %d, and this  may  mean
    that  flags  don't  work as you might expect.  This also means that the `-' flag does work (it forces fields to be left-aligned).
    Unlike -print, -printf does not add a newline at the end of the string.
    ...
    %s  File's size in bytes.
    ...

And to get a total i used this:

echo $[ $(find $path -type f -printf %s+)0] #b
echo $[($(find $path -type f -printf %s+)0)/1024] #Kb
echo $[($(find $path -type f -printf %s+)0)/1024/1024] #Mb
echo $[($(find $path -type f -printf %s+)0)/1024/1024/1024] #Gb

View more solutions

Comments

[Johny Mahony]

Very nice. Although, remembering that brutal keyword to 'find' ("--files0-from=") may not actually be easier than remembering the 'awk' sequence.

[jason]

Using du version 8.13 the following gives me the same result: du -ch /rapidly_shrinking_drive/*offender1* | tail -n1

[Slipp D. Thompson]

My machine doesn't like the --files0-from= option. ;-/

[Slipp D. Thompson]

An alternative that seems to work: find rapidly_shrinking_drive/ -name "offender1" -mtime -1 -print0 | xargs -0 du -hc | tail -n1

[Sylvain]

Alternative using -exec in the find : find rapidly_shrinking_drive/ -name "offender1" -mtime -1 -exec du {} \; | awk '{ total += $1 }END{ print total }'

[Hennes]

+1 for the explicit mention of GNU find. (To bad it is less portable that way).

[spookypeanut]

If you're going to do it this way, you need to use xargs

[Zen]

is it able to convert the final output number into more human readable format like 103M?

[starbeamrainbowlabs]

Note: The flag --files0-from argument in the above is not a spelling mistake.

[Yves M.]

@zen you can use numfmt (parts of coreutils) and do something like find rapidly_shrinking_drive/ -name "offender1" -mtime -1 | du | awk '{total+=$1} END{print total}' | numfmt --to=si

[Ciprian Tomoiagă]

Interesting and uses only find functionality! Nice! Can you please explain the $[..] syntax for echo ?

[Ivan]

@CiprianTomoiagă sure it's bash math syntax echo $[1+1] will print 2