Configuring RollingFileAppender in log4j

Question

I'm working on a set of web services and we'd like to have a daily rotated log.

I'm trying to get org.apache.log4j.rolling.RollingFileAppender from the log4j extras companion working, since the documentation suggests this is best for production environments.

I have both the main log4J library (log4j-1.2.15.jar) and the log4j extras library (apache-log4j-extras-1.1.jar) on the classpath.

I have the following configuration for the appender in the log4j.properties file:

### SOAP Request Appender
log4j.appender.request=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.request.File=SOAPmessages.log
log4j.appender.request.RollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.request.RollingPolicy.ActiveFileName =SOAPmessages-%d.log
log4j.appender.request.RollingPolicy.FileNamePattern=SOAPmessages-%d.log.zip
log4j.appender.request.layout = org.apache.log4j.PatternLayout
log4j.appender.request.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n

However, when I start the web service with log4j in debug mode I get these error messages:

log4j: Parsing appender named "request".
log4j: Parsing layout options for "request".
log4j: Setting property [conversionPattern] to [%d{ABSOLUTE} %5p %c{1}:%L - %m%n].
log4j: End of parsing for "request".
log4j: Setting property [file] to [/logs/SOAPmessages.log].
log4j:WARN Failed to set property [rollingPolicy] to value "org.apache.log4j.rolling.TimeBasedRollingPolicy". 
log4j:WARN Please set a rolling policy for the RollingFileAppender named 'request'
log4j: Parsed "request" options.

I've found documentation about how to configure this appender a little sparse, so can anyone help me fix my configuration?

EDIT0: Added debug mode output, rather than just the standard warnings

Solution 1

Update: at least as early as 2013 (see Mubashar's comment) this started working.

---

According to Log4jXmlFormat you cannot configure it with log4j.properties, but only using the XML config format:

Note that TimeBasedRollingPolicy can only be configured with xml, not log4j.properties

Unfortunately, the example log4j.xml they provide doesn't work either:

log4j:ERROR Parsing error on line 14 and column 76
log4j:ERROR Element type "rollingPolicy" must be declared.
...
log4j:WARN Please set a rolling policy for the RollingFileAppender named 'FILE'

Solution 2

I had a similar problem and just found a way to solve it (by single-stepping through log4j-extras source, no less...)

The good news is that, unlike what's written everywhere, it turns out that you actually CAN configure TimeBasedRollingPolicy using log4j.properties (XML config not needed! At least in versions of log4j >1.2.16 see this bug report)

Here is an example:

log4j.appender.File = org.apache.log4j.rolling.RollingFileAppender
log4j.appender.File.rollingPolicy = org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.File.rollingPolicy.FileNamePattern = logs/worker-${instanceId}.%d{yyyyMMdd-HHmm}.log

BTW the ${instanceId} bit is something I am using on Amazon's EC2 to distinguish the logs from all my workers -- I just need to set that property before calling PropertyConfigurator.configure(), as follow:

p.setProperty("instanceId", EC2Util.getMyInstanceId());
PropertyConfigurator.configure(p);

Solution 3

Faced more issues while making this work. Here are the details:

1. To download and add apache-log4j-extras-1.1.jar in the classpath, didn't notice this at first.
2. The RollingFileAppender should be org.apache.log4j.rolling.RollingFileAppender instead of org.apache.log4j.RollingFileAppender. This can give the error: log4j:ERROR No output stream or file set for the appender named [file].
3. We had to upgrade the log4j library from log4j-1.2.14.jar to log4j-1.2.16.jar.

Below is the appender configuration which worked for me:

<appender name="file" class="org.apache.log4j.rolling.RollingFileAppender">
        <param name="threshold" value="debug" />
        <rollingPolicy name="file"
            class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
            <param name="FileNamePattern" value="logs/MyLog-%d{yyyy-MM-dd-HH-mm}.log.gz" />
            <!-- The below param will keep the live update file in a different location-->
            <!-- param name="ActiveFileName" value="current/MyLog.log" /-->
        </rollingPolicy>

        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%5p %d{ISO8601} [%t][%x] %c - %m%n" />
        </layout>
</appender>

Solution 4

Toolbear74 is right log4j.XML is required. In order to get the XML to validate the <param> tags need to be BEFORE the <rollingPolicy> I suggest setting a logging threshold <param name="threshold" value="info"/>

When Creating a Log4j.xml file don't forget to to copy the log4j.dtd into the same location.

Here is an example:

<?xml version="1.0" encoding="windows-1252"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd" >
<log4j:configuration>
<!-- Daily Rolling File Appender that compresses old files -->
  <appender name="file" class="org.apache.log4j.rolling.RollingFileAppender" >
     <param name="threshold" value="info"/>
     <rollingPolicy name="file"  
                      class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
        <param name="FileNamePattern" 
               value="${catalina.base}/logs/myapp.log.%d{yyyy-MM-dd}.gz"/>
        <param name="ActiveFileName" value="${catalina.base}/logs/myapp.log"/>
     </rollingPolicy>
     <layout class="org.apache.log4j.EnhancedPatternLayout" >
        <param name="ConversionPattern" 
               value="%d{ISO8601} %-5p - %-26.26c{1} - %m%n" />
    </layout>
  </appender>

  <root>
    <priority value="debug"></priority>
    <appender-ref ref="file" />
  </root>
</log4j:configuration>

Considering that your setting a FileNamePattern and an ActiveFileName I think that setting a File property is redundant and possibly even erroneous

Try renaming your log4j.properties and dropping in a log4j.xml similar to my example and see what happens.

Solution 5

Regarding error: log4j:ERROR Element type "rollingPolicy" must be declared

1. Use a log4j.jar version newer than log4j-1.2.14.jar, which has a log4j.dtd defining rollingPolicy.
2. of course you also need apache-log4j-extras-1.1.jar
3. Check if any other third party jars you are using perhaps have an older version of log4j.jar packed inside. If so, make sure your log4j.jar comes first in the order before the third party containing the older log4j.jar.

View more solutions

Comments

[Kevin]

FYI: This problem was fixed in log4j v1.2.16. it now supports setting rollingPolicy via the properties file. issues.apache.org/bugzilla/show_bug.cgi?id=36384

[chrisbunney]

The ActiveFileName property is documented here: logging.apache.org/log4j/companions/extras/apidocs/org/apach‌​e/… I'll try taking it out and see what happens

[chrisbunney]

Commenting out the ActiveFileName property doesn't alter the observed behaviour

[chrisbunney]

I see, didn't notice that in any of the documentation I read, nor any links to the wiki which would have been helpful. Thanks for pointing that out +1

[chrisbunney]

Yep, it needed to be configured using XML +1

[chrisbunney]

Actually, both are valid package names. One refers to the implementation of a RollingFileAppender in the main log4j library, whilst the other refers to the implementation in the log4j extras library, which provides a variety of alternative filters and appenders. The implementation in the main package specifically recommends using the extras implementation.

[Mubashar]

This might be true on the date you answered but it is working fine with log4j.properties file, atleast for me

[msangel]

I'd rather will put one additional library to the classpath than use XML for this task, becouse I have already a lot of configuration in properties style, so rewritting all just because one appender is not logical

[ikaerom]

Recently, I've created a patch to also allow for recursive directories to be configured. You can for example store daily zipped log files in monthly subdirectories like so: log4j.appender.File.rollingPolicy.FileNamePattern=Logs/%d{yy‌​yyMM}/ServerLog_%d{y‌​yyyMMdd}.log.zip. The needed directories will be created by log4j correctly when needed.

[Raghvendra Singh]

There is a "log4j-extras" package from apache, where you can find the "rollingPolicy" defined. It will solve many of your problems

[Sagar Kharab]

@toolbear : I beg to differ because I am using log4j.properties and it's working fine. Is it the case that they have added the support later.

[toolbear]

@SagarKharab answer is out of date. In 2011 it didn't work. As early as 2013 (see Mubashar's comment) it worked. I've updated my answer to indicate this.