How can I track growing directories?

Question

I have a system that continually runs out of disk space. Im going to write a script that cleans up directories that grow larger. Usually these are log directories. Given the nature of the machine, i dont need to keep logs.

I want to write a script that shows me all the directories that are increasing in size, so i can make sure this list goes to 0.

I am thinking something like this

1. du /
2. wait 24 hrs
3. du /
4. find all instances where the file size increased

It looks simple enough, but my issue is that du works on files and directories, you cant do it on only dirs. How can i overcome that?

Solution 1

You could use find to tell whats been modified in 24 hours,

find / -type d name \* -mtime 1 -print

Should list all directories that have changed in the last 24 hours, but remember, realistically its files that change rather than the directories.

Another way would be to do

du -s *  | sort > currentsize

This summerises by directory, and files in the starting location, but, you then could store it each day, move that file to say oldsize, run the command again

then

diff currentsize oldsize

you can see whats changed.

You could use awk to decide if one is bigger or smaller..

All system logs tend to to change too.

But the original find I stated shows you files altered in the last 24 hours. You could also use something based on that.

Solution 2

If your problem is with log files, may I suggest using logrotate instead?

It has the capability of periodically 'zero'-ing log files, archiving and/or deleting old logs, and so on. Even to the point of stopping the related daemon before doing its magic (might be necessary if said daemon keeps the log file open).

Comments

[Nicolas Boisteault]

Related : serverfault.com/questions/332948/… gt5 and ncdu helped me a lot.

[Admin]

this sort of works, i will be doing a lot of double work though. so i could do it on root, find the top level dirs that increased, then go down and down and down and recursively find the invididual dirs that changed. but i will be repeating quite a lot of work.

[nalply]

You mean restarting, not stopping. Even after logrotate renamed or even deleted a file, the daemon process keeps writing to the same file (if you deleted it, it disappeared from the directory but it is not unlinked yet from the filesystem or respectively, if you renamed it, the renamed file continues to grow). A restart forces the daemon to drop its hold on the file.

[pepoluan]

@nalply well, a 'restart' is technically a 'stop' followed by a 'start'. I meant 'stop' not 'pause'. But your explanation helps in highlighting why the daemon needs to be stopped.