How can I forward an application log to a remote log server?

Question

I have an application which writes its own log file in /var/log/app/app.log. How can I forward these logs to a remote Rsyslog server?

Solution 1

You can do this with the imfile module.

On the sending server's rsyslog config;

$ModLoad imfile
$InputFileName /var/log/app/app.log
$InputFileTag tag_app_log:
$InputFileStateFile app_log1
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor

# Send over TCP
local7.*                                @@remoteserver
# Send over UDP
local7.*                                @remoteserver

On the receiving server's rsyslog config;

$template YourApp, "/path/to/yourlogs/app/app.log"
local7.*                                -?YourApp

Solution 2

You can use syslog-ng to forward the logs.

source s_all {
internal();
unix-stream("/dev/log");
file("/path/to/your/file" follow_freq(1) flags(no-parse));
};
destination d_remotelogger {
udp("192.168.254.254" port(5514));
};
log {
source(s_all); destination(d_remotelogger);
};

Comments

[Tom O'Connor]

You've not made any reference on how to actually forward the logs over TCP or UDP. Don't use TCP with imfile and rsyslog, however. I've seen some very dodgy behavior if the remote end goes away (for whatever reason).

[Aaron Copley]

You're right. I rushed to get this answered on my way out yesterday. I'll edit to include that bit. I included both methods, since any issue with rsyslog/tcp/and imfile should be submitted as a bug report. It's not unsupported so there should be an expectation for it to work. Additionally, you may want to configure local queuing and RELP for guaranteed delivery. (Outside the scope of the question.)

[user2284355]

Thank you very much for your help. This has gotten me halfway. For some reason logs from app.log are not being sent (tcpdump reports no traffic) but daemon specific traffic is. I opened another question on security stack-exchange with the specific application and where I am at the moment: security.stackexchange.com/questions/38709/…. Could you lend me a hand? Cheers

[Aaron Copley]

The daemon messages are probably covered by the *.* rule on the sender. You need to focus on the configuration above. What version of rsyslog and on what distribution? This is generic for Red Hat 6, rsyslog 5.8.x which is what I have notes for in front of me.

[user2284355]

I have tried both TCP and UDP none of them seem to relay any messages apart from the specific daemon logs. Currently I have the rsyslog server listening on UDP and have local7.* @remoteserver on my client. TCP dump is being checked on the sending server.

[user2284355]

Sending server is:Linux 3.2.0-48-generic #74-Ubuntu x86_64 x86_64 x86_64 GNU/Linux - Ubuntu Precise.rsyslogd 5.8.6 ---- Receiving server is: Linux 2.6.32-5-amd64 x86_64 GNU/Linux Debian based.rsyslogd 4.6.4

[user2284355]

Using: logger -t honeyd "my little pony" correctly logs the string "my little pony" into /var/log/honeyd.log on my receiving server.

[gparent]

Having done this very recently, some things to check: Make sure the modules are properly loaded for sending/receiving TCP/UDP, make sure your rsyslog server is listening on the port it's supposed to, make sure there's an established connection to that port from your client. Preferably, you'd use RELP, although contrarily to what Tom is saying, the documentation tends to suggest TCP instead of UDP if you have to pick between those two.

[user2284355]

Thank you all very much for your help. The proposed syntax was perfect. Reinstalling rsyslog (apt-get install --reinstall rsyslog) did the trick for me.