Linux tool to track directory space over time

Question

Are there any common Linux tools that track disk space over time, not just a filesystem overall, but for example, so I could easily see which directory trees swelled up and shrunk historically? The problem happens on servers which seem to keep hitting 0 free space despite routine house-cleaning of cache and tmp files, logs etc. I already use tools like df, du and lsof often and schedule at jobs to find aged files over a certain size/type when low water-marks are hit, etc.

Preferably a Debian/Ubuntu package.

Solution

Couple things would make the creation of a tool like this problematic:

1. The kernel doesn't keep directory or file-specific file size histories (like it does with memory usage, I/O activity, or CPU time), so anything written would have to be in userspace.
2. Even if something were written using inotify or something, on busy servers the overhead of a user space daemon recording the output of "du -s" after every file change would be huge. On less than busy servers you probably wouldn't have this concern. So you run into the situation where the tool is only useful to the people who can't afford its overhead.
3. Network Monitoring daemons such as Zabbix or Nagios contain some primitive disk utilization monitoring (of whole filesystems) with histories that serves the needs of most admins in this regard. Needs beyond that are typically break/fix where you go into the filesystem in question and check to see what's taking up the space after the fact (using network monitoring to do proactive warning of possible issues).
4. If you really really need to know this and waiting for your proactive warnings to notify you when it's happening again (so that you can check it out) isn't an option, you can ultimately write a script to monitor the directories you're concerned about. (basically recording the output of "find /path/to/dir -type d -exec du -sh {} \;" in some way that's going to be easy for you to review).

So there's no real demand for creating a user space tool to do this since it would be hard to do correctly and existing solutions get you pretty close to the ideal.

Short Answer: I'm not aware of such a tool, but it makes sense there wouldn't be.

Comments

[peterph]

I take it you want something more user-friendly than du -s running in cron, right?

[Bratchley]

I think they're wanting a historic calculation of disk space utilization like what you'd get for CPU or I/O utilization doing a "sar" command.

[WabuMike]

Can this be done with low overhead using a tool that monitors and extracts relevant data from the (ext3) journal?

[RIQ]

It's within my reach to hack together user-space scripts to eg. record periodic du snapshots from cron of all/some directories, store them in SQL/CSV for later analysis to narrow down where storage is being consumed when, and make friendly reports to pinpoint this. With ability to zoom into the dir hierarchy, one hopes. But that's time-consuming and I'm hoping something similar exists, even if imperfect (hardlinks etc.).

[WabuMike]

I'm wondering whether this, with much less overhead than "du -s", could be done with some tool that watches the filesystem journal?