Continuously monitor logs with tail that are occasionally rotated
Solution 1
Ah, there's a flag for this.
instead of using tail -f /var/log/file
we should be using tail -F /var/log/file
tail -F
translates to tail --follow=name --retry
as in;
--follow=name
: follow the name of the file instead of the file descriptor--retry
: if the file is inaccessible, try again later instead of dying
Solution 2
# tail --follow=mylog.log
From man tail:
With --follow (-f), tail defaults to following the file descriptor,
which means that even if a tail’ed file is renamed, tail will continue
to track its end. This default behavior is not desirable when you
really want to track the actual name of the file, not the file descrip‐
tor (e.g., log rotation). Use --follow=name in that case. That causes
tail to track the named file by reopening it periodically to see if it
has been removed and recreated by some other program.
So in this case using the -F
option would be correct.
-F same as --follow=name --retry
Solution 3
The exact answer depends on your OS - but in many cases, tail -F
will do the right thing.
Solution 4
tail -F or tail --follow=name
Solution 5
IMHO, it's a little odd to change your log file by SIZE rather than by date. Most system logs (in unix or linux) rotate on a weekly or monthly basis, and not based on size...This is something I like for various reasons, and also something which, if implemented, would solve your problem.
Eight years later, I don't know what the hell I was talking about here: there are tons of places where you want to rotate by size, because daily/weekly/monthly rotations can yield MASSIVE files which can cause serious issues.
From a more experienced perspective, the real question is why you'd want to sit and continuously tail a file that's growing so fast that you're rotating it more than daily...It'd be like watching the Matrix stream by.
These days you'd be better looking into some big data log aggregation like Splunk or Sumologic, where it can filter log events into classes and trigger based on specific log values...No need for watching live logs at all.
Related videos on Youtube
Comments
-
Stanly almost 2 years
We're using tail to continuously monitor several logs, but when a log is rotated the tail for that file will cease.
As far as I understand, the problem is that when the log is rotated, there is a new file created, and the running tail process doesn't know anything about that new file handle.
-
user202729 over 2 yearsSometimes even
-F
doesn't work when file is repeatedly truncated. See overwrite - Is there atail -f
equivalent for when a file is overwritten (instead of appended to)? - Server Fault and logging -tail -f
sometimes stops updating - and the file hasn't moved - Server Fault
-
-
chris over 14 yearsIf tail -F doesn't work, compile a version of tail -F that does. The other alternative is a short road to crazy town.
-
David over 14 yearsI agree and this is one thing I realized I should look into. Hopefully log4j supports it. Thanks!
-
Jim Zajkowski over 14 yearsOne clear advantage of rotation by size is you can be sure the entire set of logs will be less than some known size.
-
Dominic Cerisano over 6 yearsWhy would they even have the "-f" option? This should be the default!