Compress Log4j files

Question

Is it possible to compress log files (I doing it via RollingFileAppender )?

Solution 1

log4j extras has support for that. Just add the following to your RollingFileAppender configuration and have the filename end in .gz to automagically compress your log files:

<appender...>
    <rollingPolicy class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
      <param name="FileNamePattern" value="/wombat/foo.%d{yyyy-MM}.gz"/>
    </rollingPolicy>
</appender>

For more details check the Javadoc

Solution 2

Try Log4j v2.x

I came upon this great answer, but then kept investigating a bit and log4j 2 is out!

Now you can have compression with the basic library and much, much more... it's just awesome!

RollingFileAppender - I was looking for something that had both timeBased and sizeBased rolling, and with compression... it has both! and I can drop my syslog4j library since this one has it also!

Please don't use log4j 1.2 + extras unless you really have to.

Solution 3

I know this does not exactly answer your question, but it does propose an alternate solution.

The way we handle that is by having a batch process run at the end of the day that compresses any prior log files to today's date, get's rid of any prior to a week old and then copies them off to another file server. This way the application does not need to consume any CPU cycles doing this and on the server we have logs that are no older than a week and on another file server we have older log files.

Solution 4

The only other thing I see is upgrade log4j to more than 1.3.15/1.4 and use TimeBasedRollingPolicy roller to have automatic compression enabled, but that would require an upgrade of log4j essentially (and compatibility of the app as well)

Hmmm..may be another better thing to have is let have log4j execute a command once the file is rolled. So instead of gzip, I can use some other compression logic like bzip, or lzop etc.. that would be another approach if I have this facility in log4j today.

View more solutions

Comments

[MiKu]

How can i add it to the property file? > log4j.appender.DefaultLog=org.apache.log4j.RollingFileAppend‌​er log4j.appender.DefaultLog.File=${mailbox.root}/logs/applicat‌​ion.log log4j.appender.DefaultLog.MaxFileSize=50MB log4j.appender.DefaultLog.MaxBackupIndex=100 log4j.appender.DefaultLog.layout=org.apache.log4j.PatternLay‌​out log4j.appender.DefaultLog.layout.ConversionPattern=%d %p [%c] - %m%n

[hd1]

It's how you specify a wildcard in Java to include directories, as opposed to * which is only for files.

[Ondrej Skalicka]

Please note that you HAVE to use org.apache.log4j.rolling.RollingFileAppender (from log4j-extras), NOT org.apache.log4j.RollingFileAppender (from log4j)!

[Brett VanderVeen]

I agree. It is awesome that we now have it standard in log4j 2, but as of the time of this comment, log4j 2 is in beta. Still I encourage anyone that can, use the beta and help improve it before it's release.

[Daren]

yeah, the beta was scheduled to end in June but it is still ongoing... a pity. It has worked fine for me but policy dictates no beta code in production.

[xlm]

@MiKu / for those looking to configure this via properties file, check out stackoverflow.com/a/6037141/885922