How does one find which files are taking up 80% of the space on a Linux webserver?

linux disk-space-utilization command-line-interface capacity

22,812

Solution 1

I find ncdu (http://dev.yorhel.nl/ncdu) to be quite helpful for this.

Solution 2

Surely there are more elaborate ways, but the one I remember is

du --max-depth=1 -h /

Now take the directory that uses up most space (du --max-depth=1 -h /yourdir) and go deeper until you find your culprit.
If want your output sorted by size and don't care for the human-readable format, you could also do du --max-depth=1 /your_dir | sort -n

Solution 3

I use the Gnome program baobab. You can run this on your desktop and t can connect via SSH to the server. It shows an easy to read graphical map of disk space usage. It's installed under Gnome as "Disk Usage Analyzer"

Solution 4

Give gt5 a try.

Solution 5

df -k shows which fs are the problem. Then cd to the top level dir for it and run du -xk | sort -n | tail -25 this will show the top 25 dir, sorted, for sun 9 or earlier, replace the x with a d.

View more solutions

22,812

Gareth

There is no excellent beauty that hath not some strangeness in the proportion. - Sir Francis Bacon (1561 - 1626)

Updated on September 17, 2022

Comments

Gareth over 1 year

The drive is constantly filling up. You've hunted down all the spare and random files you can. grep'd for coredump files and even removed some of the un-needed backups...

What would be your next move.

The actual server in question has 10GB of website files and the OS shouldn't take any more than 10GB so how do you track down what's filling a 50GB (virtual) drive?
- Ciro Santilli Путлер Капут 六四事 almost 9 years
  
  unix.stackexchange.com/questions/125429/…
Gareth almost 15 years

Yeah. I do pretty much the same thing "du -S | sort -n -r |less". I really would love to see a programme that looked like htop and cron'd like mlocate but when ran gave you accurate and contemporary info about the files on your system.
Gareth almost 15 years

Yeah, similar to what I just mentioned in @Marie Fischer's answer. Why use the -k (block size) though rather than -h for human?
Josh almost 15 years

I second this, it is great for tracking where space is going because it shows diffs.
Saurabh Barjatiya almost 15 years

-k is used so that all sizes are reported in kb. This is useful for sort else sort would put 10kb before 20mb while sorting.
Saurabh Barjatiya almost 15 years

Instead of starting from / on webserver try starting from http_root. If there is no success there then one can go for '/'. Directory running du on '/' will take lot of time.
fxmtor almost 15 years

I use /usr/sbin/lsof | grep deleted to get this.
Gareth over 14 years

Perfect. Brilliant interface and the ability to manipulate files from within the programme. Cheers!
PKHunter over 9 years

Lovely. This should be the right answer. What a super find. Thanks.
kasperd about 8 years

This is not going to work unless you change the order of head and sort. Also you should make use of the formatting features.