How can I monitor disk I/O in a particular directory?

Question

I've got a few processes with a known name that all write to files in a single directory. I'd like to log the number of disk block reads and writes over a period (not just file access) to test whether a parameter change reduces the amount of I/O significantly. I'm currently using iostat -d -p, but that is limited to the whole partition.

Solution 1

I realize this is going to sound both simplistic and absurd, but if you have control over the apps in question (maybe in a test environment) you could mount ONLY that directory on a partition of its own, then iostat, etc. would tell you only about it, and nothing else on that spot.

If there are physical drives involved you could fake it up with a loopback mount à la

dd if=/dev/zero of=/bigdisk/LOOPFILE bs=1024m count=1024m # 1gb loopback file
mke2fs -j /bigdisk/LOOPFILE
mkdir /tmpcopy
mount -o loop /tmpcopy /bigdisk/LOOPFILE
cp -r -p $SPECIALDIR2MONITOR /tmpcopy
umount /tmpcopy
mount -o loop $SPECIALDIR2MONITOR  /bigdisk/LOOPFILE,

That would not completely remove all competing disk I/O, but I'm pretty sure iostat's output would be more specific to your need.

Solution 2

I don't think there's a direct way. One way to get the data you want would be to access the directory tree through a virtual filesystem that logs accesses. Loggedfs is one such filesystem, though I don't know if it can show all the data you're interested in. (If not it would probably be a modest coding effort to that data.)

mkdir /tmp/replica
loggedfs /path/to/directory /tmp/replica
mycommand --root=/tmp/replica
fusermount -u /tmp/replica

Solution 3

You can use inotifywait -m DIRNAME from the inotify-tools.

Comments

[Tobias Kienzler]

related: How can I have activity in a folder logged? and monitoring activity on my computer.

[l0b0]

This is different from those questions, because I'm not interested in the files themselves but rather the amount of I/O.

[l0b0]

-d doesn't exist in inotifywait 3.13. Do you mean -m?

[Tobias Kienzler]

@l0b0: oh, I use 3.14 from the git repo. sorry. But the manpage states it's the same as -m with the exception of running in the background and requiring an outfile.

[l0b0]

Tested it. I'm sorry for the misunderstanding, but I'm not interested in which files (or even how many files) were accessed, but rather the amount of I/O.

[Tobias Kienzler]

Oh I see. Maybe inotifywatch does what you want, but I don't know much about it

[Shawn J. Goff]

Note, this is Linux-specific. BSDs have kqueue and pnotify system calls, but I don't know if they do exactly what the author is requesting.