tail a log file from a specific line number

Question

I know how to tail a text file with a specific number of lines,

tail -n50 /this/is/my.log

However, how do I make that line count a variable?

Let's say I have a large log file which is appended to daily by some program, all lines in the log file start with a datetime in this format:

Day Mon YY HH:MM:SS

Every day I want to output the tail of the log file but only for the previous days records. Let's say this output runs just after midnight, I'm not worried about the tail spilling over into the next day.

I just want to be able to work out how many rows to tail, based on the first occurrence of yesterdays date...

Is that possible?

Solution 1

Answering the question of the title, for anyone who comes here that way, head and tail can both accept a code for how much of the file to exclude.

• For tail, use -n +num for the line number num to start at
• For head, use -n -num for the number of lines not to print

This is relevant to the actual question if you have remembered the number of lines from the previous time you did the command, and then used that number for tail -n +$prevlines to get the next portion of the partial log, regardless of how often the log is checked.

Answering the actual question, one way to print everything after a certain line that you can grep is to use the -A option with a ridiculous count. This may be more useful than the other answers here as you can get a number of days of results. So to get everything from yesterday and so-far today:

grep "^`date -d yesterday '+%d %b %y'`" -A1000000 log_file.txt

You can combine 2 greps to print between 2 date ranges.

Note that this relies on the date actually occurring in the log file. It has the weakness that if no events were logged on a particular day used as the range marker, then it will fail to find anything.

To resolve that you could inject dummy records for the start and end dates and sort the file before grepping. This is probably overkill, though, and the sort may be expensive, so I won't example it.

Solution 2

I don't think tail has any functionality like this.

You could work out the beginning and ending line numbers using awk, but if you just want to exact those lines from the log file, the simplest way is probably to use grep combined with date to do it. Matching yesterday's date at beginning of line should work:

grep "^`date -d yesterday '+%d %b %y'`" < log_file.txt

You may need to adjust the date format to match exactly what you've got in the log file.

Comments

[user3439894]

I'm glad you worked it out however, in your answer there is and undefined variable in the code, i.e. datestring, so in start= ... grep -no $datestring ... there is not a proper context for it in the code. In other words, you should show how datestring is assigned its value too!