tail -v a file while excluding a list of words

Question

I'd like to tail a log file that's continuously being written to but would like to exclude various entries that are 'common' so I only see errors, etc. as they are thrown. I can pipe to grep -v "pattern" but would ideally like to use a file containing entries to skip. Any ideas?

Solution 1

You can use grep -v "pattern" like you said. But instead of pattern use

`cat file_with_entires_to_skip`

Remember to include the backtics. So your command would look like:

tail -v FILE | grep -v "`cat FILE_WITH_ENTRIES_TO_SKIP`"

Solution 2

The -f option to grep allows one to specify a file containing patterns, one pattern per line. See the grep(1) man page.

In addition, the unbuffer command available on some systems will disable the output buffering that normally occurs in pipelines. The addition of the grep filter may otherwise delay the output of your pipeline. See the unbuffer(1) man page for details.

Solution 3

My grep includes an --exclude-from=FILE option that lets me add exclusion patterns from a file. I've got:

my-macbook-pro:Sun America ianchesal$ grep --version
grep (GNU grep) 2.5.1

So you could do:

tail -f <file> | grep --exclude-from=excludes.txt

Comments

[Gilles 'SO- stop being evil']

Make that grep -v "cat FILE_WITH_ENTRIES_TO_SKIP", or the shell will expand special characters in the pattern list.

[Wuffers]

You mean grep -v "`cat FILE_WITH_ENTIRES_TO_SKIP`"? (You have to escape backtics in Markdown.)

[Janus Troelsen]

"Skip files whose base name matches any of the file-name globs read from FILE (using wildcard matching as described under --exclude)." So it's for skipping files, not matches

[Hammer Bro.]

It sounds like they are looking to exclude specific results, not patterns, so a fixed-string search (-F) would be more appropriate. I'd go with tail -v [file_to_tail] | grep -Fvf [file_with_list_of_lines_to_exclude]