Query specific logs from event log using nxlog

Question

Below is my nxlog configuration

define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension json>
    Module      xm_json
</Extension>
<Input internal>
        Module      im_internal
</Input>
<Input eventlog>
    Module  im_msvistalog
    Query   <QueryList>\
            <Query Id="0">\
            <Select Path="Security">*</Select>\
            </Query>\
            </QueryList>
    </Input>
<Output out>
    Module  om_tcp
    Host    localhost
    Port    3515
    Exec    $EventReceivedTime = integer($EventReceivedTime) / 1000000; \
            to_json();
</Output>
<Route 1>
    Path    eventlog, internal => out
</Route>

<Select Path="Security">*</Select>\ -> * gets everything from the Security log, but my requirement is to get specific logs starting with EventId - 4663. How do i do this? Please help. Thanks.

Solution 1

Doing a regexp match on $raw_event is a little ugly and inefficient.

I suggest using the following form:

Exec if string($EventID) !~ /^42/ drop()

The alternative is to use the XML event selection:

Query <QueryList> \
           <Query Id="0">\
              <Select Path="Security">*[System[(EventID='4663')]]</Select>\
           </Query>\
      </QueryList>

Although it looks like the starts-with match won't work here:

XPath 1.0 Limitations:

Windows Event Log supports a subset of XPath 1.0. There are limitations to what functions work in the query. For instance, you can use the "position", "Band", and "timediff" functions within the query but other functions like "starts-with" and "contains" are not currently supported.

Solution 2

I am not sure if your event is INFO|WARNING|ERROR or what... but here...

Exec    if $raw_event !~ /INFO\s+4663/ drop();

Quick, Use Regex... if my $raw_event equals "2013-11-18 15:23:02 INFO 2013-12-18 15:23:01 ahost.adomain.local INFO 62464 UVD Information" I would use the following to DROP event:

Exec    if $raw_event =~ /INFO\s+62464/ drop();

Short Example, you need to use RegEx to find exactly what you need when you access the $raw_event variable. Please remove/adjust "log_info" after testing.

Exec if ($raw_event =~ /INFO\s+62464/) \
    { \
        log_info('Found amdkmdag EventID 62464, dropping it.'); \
        drop(); \
    }

Full Example, where I use nxlog-ce (Windows) to a Debian/Graylog SysLog Server in GELF format.

## This is a basic configuration file for Windows Server 2008 * 2012 
## to GrayLog2 with GELF support and filtering.
## See the nxlog reference manual about the configuration options. 
## It should be installed locally and is also available
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

define ROOT C:\Program Files (x86)\nxlog
# define ROOT C:\Program Files\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
    Module xm_gelf
</Extension>

<Input pr_mseventlog>
    Module      im_msvistalog
    ReadFromLast    True
    # http://msdn.microsoft.com/en-us/library/aa385231.aspx
    # http://msdn.microsoft.com/en-us/library/ff604025(v=office.14).aspx
    # Level 1 (ID=30  Critical)     severity level events
    # Level 2 (ID=40  Error)        severity level events
    # Level 3 (ID=50  Warning)      severity level events
    # Level 4 (ID=80  Information)  severity level events
    # Level 5 (ID=100 Verbose)      severity level events
    # All channels are included by default which are listed in the registry under these:
    # HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\WINEVT\Channels 
    # HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\eventlog\System
    #
    # <Select Path='Key Management Service'>*</Select></Query>\
    # <Select Path='Internet Explorer'>*</Select></Query>\
    # <Select Path='HardwareEvents'>*</Select></Query>\
    #
    Query   <QueryList>\
        <Query Id="0">\
            <Select Path="Security">*</Select>\
            <Select Path="System">*[System/Level=4]</Select>\
            <Select Path="Application">*[Application/Level=2]</Select>\
            <Select Path="Setup">*[System/Level=3]</Select>\
            <Select Path='Windows PowerShell'>*</Select>\
        </Query>\
    </QueryList>

    # REGEX EXAMPLES:
    # "\s" equals one white space character, and ".*" equals any one char 
    # Line Contains both "bubble" and "gum"
    #   Search pattern: ^(?=.*?\bbubble\b)(?=.*?\bgum\b).*
    # Line does Not Contain "boy"
    #   Search pattern: ^(?!.*boy).*
    # Line Contains "bubble" but Neither "gum" Nor "bath"
    #   Search pattern: ^(?=.*bubble)(?!.*gum)(?!.*bath).*

    # Uncomment next line to view all logs, we can view output to help 
    # create the regex, next line shows my $raw_event data to parse:
    # 2013-11-18 15:23:02 INFO 2013-12-18 15:23:01 ahost.adomain.local INFO 62464 UVD Information
    # Exec   log_info($raw_event) ;
    Exec if ($raw_event =~ /INFO\s+62464/) drop();

</Input>

<Output out>
    Module      om_udp
    Host        10.247.x.x
    Port        12201
    OutputType  GELF
</Output>

<Route 1>
    Path    pr_mseventlog  => out
</Route>

Comments

[squillman]

Welcome to Serverfault! Link-only answers are discouraged since links tend to break over time. It would be very helpful if you could add something to your answer that summarizes the relevant content from the link.

[b0ti]

There is an $EventID field which you can use instead of matching on $raw_event, see my answer below.

[Logman]

I am fairly new to nxlog, so I was not sure if that variable would be set post or pre INPUT as was as the other fields generated from im_mseventlog - Core variables as well as $Message, $Hostname , $EventType and the others... thanks