syslog ip ranges to specific files using `rsyslog`

Question

I have many Cisco / JunOS routers and switches that send logs to my Debian server, which uses rsyslogd.

How can I configure rsyslogd to send these router / switch logs to a specific file, based on their source IP address? I do not want to pollute general system logs with these entries.

For instance:

• all routers in Chicago (source ip block: 172.17.25.0/24) to only log to /var/log/net/chicago.log.
• all routers in Dallas (source ip block 172.17.27.0/24) to only log to /var/log/net/dallas.log.
• Delete all APF-3-RCV_UNSUPP_MSG messages without logging them
• Send logs for 172.17.4.4 to a file named /var/log/net/firewall.log
• Forward firewall logs to 10.14.12.12 using UDP port 514

Finally, these logs should be rotated daily for up to 30 days and compressed.

---

NOTE: I am answering my own question

Solution 1

rsyslogd Configuration

In /etc/rsyslogd.conf

# provides remote UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

# If logging to an NFS mount, use these settings...
#    "OMFileFlushOnTXEnd off" avoids fsync on every write...
#     mount -o hard,rsize=32768,wsize=32768,noacl,noatime,nodiratime -t nfs
$OMFileIOBufferSize 768k
$OMFileAsyncWriting on
$OMFileFlushOnTXEnd off
$OMFileFlushInterval 10
$MainMsgQueueSize 100000


# kill all INTF-FLAP messages...
if $msg contains 'INTF-FLAP' then /dev/null
&~
## Cisco ACS Accounting...
if ($fromhost-ip=='172.17.16.20') and ($programname == 'CSCOacs_TACACS_Accounting') then /var/log/tacacs_acct.log
&~
## CiscoACS 5.4 TACACS Authentication
if ($fromhost-ip=='172.17.16.20') and ($programname == 'CSCOacs_Passed_Authentications') then /var/log/tacacs_auth.log
&~

# Logging for Chicago issues...
if $fromhost-ip startswith '172.17.25' then /var/log/net/chicago.log
& ~
# Logging for Dallas issues...
if $fromhost-ip startswith '172.17.27' then /var/log/net/dallas.log
& ~
# Logging for firewall...
if $fromhost-ip=='172.17.4.4' then @10.14.12.12
if $fromhost-ip=='172.17.4.4' then /var/log/net/firewall.log
& ~

Each of the &~ entries prevents fall-through to the rest of the rsyslog.conf configuration; thus I won't see router syslog entries in /var/log/messages.

Touch all syslog files:

• touch /var/log/net/chicago.log
• touch /var/log/net/dallas.log
• touch /var/log/net/firewall.log

Restart rsyslogd with /etc/init.d/rsyslogd restart

Log rotation

In /etc/logrotate.d/rsyslog

/var/log/net/*.log
{
        copytruncate
        rotate 30
        daily
        missingok
        dateext
        notifempty
        delaycompress
        create root 664 root root
        compress
        maxage 31
        sharedscripts
        lastaction
                # RHEL: Use "/sbin/service rsyslog restart"
                # Debian / Ubuntu: Use "invoke-rc.d rsyslog reload > /dev/null"
                invoke-rc.d rsyslog reload > /dev/null
        endscript
}

Solution 2

Also, I found this on the rsyslog wiki that could serve as future refernce for someone.

http://www.rsyslog.com/storing-messages-from-a-remote-system-into-a-specific-file/

Comments

[Qiang Xu]

Is there a way for an external program, like a python script, to read the value of $fromhost-ip for each log entry received by rsyslogd? The purpose here is to determine if the log entry comes from a certain ip address.

[Mike Pennington]

The first thing that comes to mind is to ensure you save $fromhost-ip in your rsyslog lines and then parse $fromhost-ip out as your python script reads each log file line by line

[Qiang Xu]

Thanks, Mike. This is good advice. I'll have to look at /etc/rsyslogd.conf to make sure $fromhost-ip is contained in the log message template.