Reading from a continuously changing logfile

Question

There is /location/of/thefile, which is a continuously changing logfile. The average density of refreshes is 4 per minute, the possible maximal refresh rate could be 30-40 per minute. Every refresh adds 2-5 lines (average), but it could be hundreds in extreme cases. Every line begins with a [YYYY-MM-DD HH:MM:SS] timestamp followed by plaintext (100-200, max. a few hundred characters).

My task is to construct a simple command which continuously watches this logfile, and sends to the stdout every lines that contain the foo OR bar alphabetical strings. Before and after those (sub)strings there could be any characters (\n only after the (sub)string, of course), even \0. The capitalization of the words could be all of the possible variations.

Well, my ideas for the solution always contain syscalls for the timing, but I shouldn't use them. Please construct me a simple command. Thanks a very lot!

Solution

I might be misunderstanding the question, but is there a reason you can't use this?

tail -f /location/of/thefile | grep -i -E "foo|bar"

Comments

[Gilles 'SO- stop being evil']

This is often called “tailing”, from tail -f. See tail for other related questions, where you'll find fancier programs that can filter and color log lines.

[Kromey]

Does piping the output of tail -f to grep really work like that? If so, I'm going to have to start using that myself! For a case like this I would have suggested a watch command, but if this does indeed work it's so much better!

[mattdm]

@Kromey: how would you expect it to work?

[Kromey]

tail -f just continually streams output to stdout, right? I'd always been under the belief that all Unix redirection operators wait until all the output/input is ready and then move it along, i.e. buffer it all until the sending program/file is done. Thus I wouldn't expect the | in Sean's command here to send anything along to grep until tail is done spitting out lines, which of course with the -f flag it won't ever do until it is interrupted. (I'm not at a *nix box to try this out, though, otherwise I would have just tested it instead of asking.)

[Sean C.]

It works, I use it lots; most of the time to track mail for whiney users. tail -f /var/log/mail.log | grep -i "[email protected]"

[forcefsck]

@Kromey, depends on the command after the pipe. If it is sort or wc it will wait the end-of-file to start sorting, if it is grep or sed or another line processing command, it will process input every end-of-line, which is the default character for flushing the i/o stream buffer.

[Kromey]

Thanks, guys! Another useful tool to add to my belt! :-)