Easy way to parse syslog date format

bash sed awk perl syslog

5,878

Solution 1

This is a bit too roundabout for my taste, but it may be of interest to you:

perl -p -e '
BEGIN{ 
        @months=qw(jan feb mar apr may jun jul aug sep oct nov dec);
        $month_number{$_}=++$i for @months;
        $months_regex= join "|",@months;
     }
s/^\s*($months_regex)[a-z]*\s+/$month_number{lc $1}-/i'
/var/log/syslog

This will print (to stdout) the log lines with the date in MM-DD format (without zero padding for month numbers) which you seem to be comfortable parsing. This handles month names given in full and month names given in 3-letter form, irrespective of case.

EDIT

If you want zero padding for month numbers, simply replace ++$i with sprintf "%02d",++$i

EDIT 2 Two mistakes fixed thanks to l0b0's comments below:

Added 'may' to the months list
Changed the recipe for zero padding to sprintf "%02d", ++$i

Solution 2

Would this be OK?

Every line holding date > 10th and < 18th:

awk '$2 > 10 && $2 < 18 {print}' file

Including month name:

awk '$1 == "May" && $2 > 10 && $2 < 18 {print}' file

Span months, April 27th to May 4th:

awk '($1 == "Apr" && $2 > 26) || ($1 == "May" && $2 < 5){print}' file

Update:

A sketchy variant using getline:

awk '"date '+%m%d' -d " $1$2 | getline date; close("date"); \
date > 426 && date < 505 {print}' file

Using script:

awk -v from=520 -v to=523 '
{
    d = ((match("JanFebMarAprMayJunJulAugSepOctNovDec", $1) + 2) / 3 )$2;
    if (d >= from && d <= to)
        print;

}
' file

Using switch:

awk -v from=520 -v to=523 '
function date2time()
{
    switch ($1) {
    case "Jan": return 1$2; break;
    case "Feb": return 2$2; break;
    case "Mar": return 3$2; break;
    case "Apr": return 4$2; break;
    case "May": return 5$2; break;
    case "Jun": return 6$2; break;
    case "Jul": return 7$2; break;
    case "Aug": return 8$2; break;
    case "Sep": return 9$2; break;
    case "Oct": return 10$2; break;
    case "Nov": return 11$2; break;
    case "Dec": return 12$2; break;
    }
}
{
    d = date2time();
    if (d >= from && d <= to)
        print;

}
' file

Using array:

... oh see you have gotten your answer ;)

Solution 3

Here's how I did it in Perl, although I prefer some of the other answers!

use DateTime::Format::Strptime;
my $parser = DateTime::Format::Strptime->new( pattern => '%B %d %Y');
m/^(\w+ \d+)/;
print $parser->parse_datetime("$1 " . DateTime->now->year)->ymd;

5,878

artfulrobot

Stitching together websites, databases and open source cunning to help you change the world.

Updated on September 18, 2022

Comments

artfulrobot over 1 year

I'd like to write a script that analyses the last week's syslog.

But my syslog logs in a stupid date format, e.g. "May 22". If it logged as 2013-05-22, this task would be trivial.

Is there a way using awk / bash / perl / sed that I can easily 'grep' the logs for all lines between two datetimes?
- Bratchley almost 11 years
  
  what version of syslog are we talking about? Most daemons will let you configure the date format using some kind of directive. You don't have to accept the defaults. It's set up that way by default because some log parsers/analyzers break if you change the date format.
- Bratchley almost 11 years
  
  for example in rsyslog and syslog-ng, you specify the date format in the template you've selected to be default.
- artfulrobot almost 11 years
  
  @JoelDavis "some log parsers.. break" exactly why I daredn't change it! Thanks for the tip, will check the man pages for whatever I'm using...
- artfulrobot almost 11 years
  
  I'm using syslogd - can't find how to set date format!
- Bratchley almost 11 years
  
  syslogd is a bit ambiguous (some syslog variations share that name for their main executable). Is this Mac OS X's syslogd?
- Bratchley almost 11 years
  
  Is this Mac OS X's syslogd? I'm seeing online that it's based on FreeBSD's syslog, in which case, the date format is hardcoded it looks like. You may try changing the daemon to syslog-ng if that's more convenient than finding an awk statement that extracts what you need.
artfulrobot almost 11 years

what about periods which span months?
artfulrobot almost 11 years

Thanks, but it's still difficult to generalise this for a given span. Would have to choose one of 2nd or 3rd forms based on whether the month was different.
l0b0 almost 11 years

One-liner replacing the date: sudo perl -n -e 'use DateTime::Format::Strptime; my $parser = DateTime::Format::Strptime->new( pattern => "%B %d %Y"); m/^(\w+ \d+ )(.*)/; print ($parser->parse_datetime("$1" . DateTime->now->year)->ymd, " ", $2, "\n");' /var/log/syslog
artfulrobot almost 11 years

@l0b0 Handy :-P