log4j redirect stdout to DailyRollingFileAppender

Question

I have a java app that uses log4j.

Config:

log4j.rootLogger=info, file

log4j.appender.file=org.apache.log4j.DailyRollingFileAppender
log4j.appender.file.File=${user.home}/logs/app.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d [%t] %c %p %m%n

So all the log statements are correctly appended to the file, but i am losing stdout and stderr. How do i redirect exception stack traces and sysouts to the daily rolled file ?

Solution 1

// I set up a ConsoleAppender in Log4J to format Stdout/Stderr
log4j.rootLogger=DEBUG, CONSOLE
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=[%t] %-5p %c - %m%n


// And I call this StdOutErrLog.tieSystemOutAndErrToLog() on startup

public class StdOutErrLog {

    private static final Logger logger = Logger.getLogger(StdOutErrLog.class);

    public static void tieSystemOutAndErrToLog() {
        System.setOut(createLoggingProxy(System.out));
        System.setErr(createLoggingProxy(System.err));
    }

    public static PrintStream createLoggingProxy(final PrintStream realPrintStream) {
        return new PrintStream(realPrintStream) {
            public void print(final String string) {
                realPrintStream.print(string);
                logger.info(string);
            }
        };
    }
}

Solution 2

In Skaffman code : To remove empty lines in log4j logs, just add "println" method to PrintStream of createLoggingProxy

public static PrintStream createLoggingProxy(final PrintStream realPrintStream) {
    return new PrintStream(realPrintStream) {
        public void print(final String string) {
            logger.warn(string);
        }
        public void println(final String string) {
            logger.warn(string);
        }
    };
}

Solution 3

I picked up the idea from Michael S., but like mentioned in one comment, it has some problems: it doesn't capture everything, and it prints some empty lines.

Also I wanted to separate System.out and System.err, so that System.out gets logged with log level 'INFO' and System.err gets logged with 'ERROR' (or 'WARN' if you like).

So this is my solution: First a class that extends OutputStream (it's easier to override all methods for OutputStream than for PrintStream). It logs with a specified log level and also copies everything to another OutputStream. And also it detects "empty" strings (containing whitespace only) and does not log them.

import java.io.IOException;
import java.io.OutputStream;

import org.apache.log4j.Level;
import org.apache.log4j.Logger;

public class LoggerStream extends OutputStream
{
private final Logger logger;
private final Level logLevel;
private final OutputStream outputStream;

public LoggerStream(Logger logger, Level logLevel, OutputStream outputStream)
{
    super();

    this.logger = logger;
    this.logLevel = logLevel;
    this.outputStream = outputStream;
}

@Override
public void write(byte[] b) throws IOException
{
    outputStream.write(b);
    String string = new String(b);
    if (!string.trim().isEmpty())
        logger.log(logLevel, string);
}

@Override
public void write(byte[] b, int off, int len) throws IOException
{
    outputStream.write(b, off, len);
    String string = new String(b, off, len);
    if (!string.trim().isEmpty())
        logger.log(logLevel, string);
}

@Override
public void write(int b) throws IOException
{
    outputStream.write(b);
    String string = String.valueOf((char) b);
    if (!string.trim().isEmpty())
        logger.log(logLevel, string);
}
}

And then a very simple utility class to set out and err:

import java.io.PrintStream;

import org.apache.log4j.Level;
import org.apache.log4j.Logger;

public class OutErrLogger
{
public static void setOutAndErrToLog()
{
    setOutToLog();
    setErrToLog();
}

public static void setOutToLog()
{
    System.setOut(new PrintStream(new LoggerStream(Logger.getLogger("out"), Level.INFO, System.out)));
}

public static void setErrToLog()
{
    System.setErr(new PrintStream(new LoggerStream(Logger.getLogger("err"), Level.ERROR, System.err)));
}

}

Solution 4

If you are using an application server, servlet container or something similar, see kgiannakakis's answer.

For standalone apps see this. You can reassing stdin, stdout and stderr using java.lang.System class. Basically you create a new subclass of PrintStream and set that instance to System.out.

Something along these lines in start of your app (untested).

// PrintStream object that prints stuff to log4j logger
public class Log4jStream extends PrintStream {
      public void write(byte buf[], int off, int len) {
        try {
           // write stuff to Log4J logger
        } catch (Exception e) {
       }
    }
}

// reassign standard output to go to log4j
System.setOut(new Log4jStream());

Solution 5

For those looking for how to do this in log4j2. There is now a component to create these streams for you.

Requires including the log4j-iostreams jar
See: https://logging.apache.org/log4j/2.x/log4j-iostreams/index.html

Example:

PrintStream logger = IoBuilder.forLogger("System.out").setLevel(Level.DEBUG).buildPrintStream();
PrintStream errorLogger = IoBuilder.forLogger("System.err").setLevel(Level.ERROR).buildPrintStream();
System.setOut(logger);
System.setErr(errorLogger);

View more solutions

Comments

[Juha Syrjälä]

How about standalone java apps that do not use a container? Surely there is a way to catch stdout and log it to log4j?

[NikolaDjokic]

As suggested by Brian Agnew you can redirect the stream to log4j. I however recommend against it. The solution is not to print at standard output, but use log4j instead. There isn't a single decent java library that uses standard output. If you have third party code that does this, ask them to use log4j or commons logging instead. Redirecting standard output should be the last resort.

[matt b]

Do not use System.out.println() for logging. Use log4j (or another logging framework) in your code, and simply have the config send the output to stdout.

[sbridges]

this works with exceptions, but it doesn't capture all output written to stdout. For example if you called System.out.print(5); it would bypass the proxy print stream. as well, even with exceptions this will end up printing extra blank lines to stderr if you do someException.printStackTrace()

[Aniket]

Where to add the path to the the log file ??

[Aniket]

can you give me an idea about how the log4j.properties or log4j.xml file should look like to get this working ?

[Dario Seidl]

@acoolguy See Log4j XML Configuration Primer, use a FileAppender.

[Nicolas Mommaerts]

LoggingOutputStream doesn't seem to exist anymore in the latest log4j contrib release (1.0.4)

[itshorty]

just take it from svn.apache.org/viewvc/logging/log4j/trunk/contribs/JimMoore/‌​… ! it's pretty stand alone.

[tysonjh]

@sbridges you can just override more of the methods if you require. For example the printStackTrace calls the println(Object o) method which you could also override to eliminate those nasty blank lines.

[Lalith J.]

I learned much more than logging while examine your code. brilliant! Thank you.

[Dojo]

Does this work for the whole Tomcat instance or only the web app that has this class?

[abhiyenta]

The only downside to this implementation is that if you have a stack trace printed to StdErr, it will print one line at a time like this, 2014-12-22 19:55:04,581 [main] INFO {ERR} at org.springframework.beans.factory.xml.XmlBeanDefinitionReade‌​r.doLoadBeanDefiniti‌​‌​ons(XmlBeanDefinit‌​ionReader.java:396) 2014-12-22 19:55:04,581 [main] INFO {ERR} at org.springframework.beans.factory.xml.XmlBeanDefinitionReade‌​r.loadBeanDefinition‌​‌​s(XmlBeanDefinitio‌​nReader.java:334)

[abhiyenta]

If you use this method, I think you are best off implementing all the methods in PrintStream. If you want to be lazy, you can use Dario's answer that implements OutputStream. Just note the caveat with that method that I noted in the comments.

[Jörg]

oh, boys ;-) assumably, the whole point is about capturing third parties printing to System.out - e.g. if you turn on log4j.debug it'll print to System.out (see also LogLog)

[AlienOnEarth]

Will this go to infinite loop if logback is pointing to standard out?

[WindyFields]

What is createLoggingProxy?