Log4J: How do I redirect an OutputStream or Writer to logger's writer(s)?

Question

I have a method which runs asynchronously after start, using OutputStream or Writer as parameter.

It acts as a recording adapter for an OutputStream or Writer (it's a third party API I can't change).

How could I pass Log4J's internal OutputStream or Writer to that method?
...because Log4J swallows System.out and System.err, I was using before.

Solution 1

My suggestion is, why dont you write your OutputStream then?! I was about to write one for you, but I found this good example on the net, check it out!

LogOutputStream.java

/*
 * Jacareto Copyright (c) 2002-2005
 * Applied Computer Science Research Group, Darmstadt University of
 * Technology, Institute of Mathematics & Computer Science,
 * Ludwigsburg University of Education, and Computer Based
 * Learning Research Group, Aachen University. All rights reserved.
 *
 * Jacareto is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public
 * License as published by the Free Software Foundation; either
 * version 2 of the License, or (at your option) any later version.
 *
 * Jacareto is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 * General Public License for more details.
 *
 * You should have received a copy of the GNU General Public
 * License along with Jacareto; if not, write to the Free
 * Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 *
 */

package jacareto.toolkit.log4j;


import org.apache.log4j.Level;
import org.apache.log4j.Logger;

import java.io.OutputStream;

/**
 * This class logs all bytes written to it as output stream with a specified logging level.
 *
 * @author <a href="mailto:[email protected]">Christian Spannagel</a>
 * @version 1.0
 */
public class LogOutputStream extends OutputStream {
    /** The logger where to log the written bytes. */
    private Logger logger;

    /** The level. */
    private Level level;

    /** The internal memory for the written bytes. */
    private String mem;

    /**
     * Creates a new log output stream which logs bytes to the specified logger with the specified
     * level.
     *
     * @param logger the logger where to log the written bytes
     * @param level the level
     */
    public LogOutputStream (Logger logger, Level level) {
        setLogger (logger);
        setLevel (level);
        mem = "";
    }

    /**
     * Sets the logger where to log the bytes.
     *
     * @param logger the logger
     */
    public void setLogger (Logger logger) {
        this.logger = logger;
    }

    /**
     * Returns the logger.
     *
     * @return DOCUMENT ME!
     */
    public Logger getLogger () {
        return logger;
    }

    /**
     * Sets the logging level.
     *
     * @param level DOCUMENT ME!
     */
    public void setLevel (Level level) {
        this.level = level;
    }

    /**
     * Returns the logging level.
     *
     * @return DOCUMENT ME!
     */
    public Level getLevel () {
        return level;
    }

    /**
     * Writes a byte to the output stream. This method flushes automatically at the end of a line.
     *
     * @param b DOCUMENT ME!
     */
    public void write (int b) {
        byte[] bytes = new byte[1];
        bytes[0] = (byte) (b & 0xff);
        mem = mem + new String(bytes);

        if (mem.endsWith ("\n")) {
            mem = mem.substring (0, mem.length () - 1);
            flush ();
        }
    }

    /**
     * Flushes the output stream.
     */
    public void flush () {
        logger.log (level, mem);
        mem = "";
    }
}

Solution 2

You can use Log4j IOStreams

The IOStreams component is a Log4j API extension that provides numerous classes from java.io that can either write to a Logger while writing to another OutputStream or Writer, or the contents read by an InputStream or Reader can be wiretapped by a Logger.

You can create an OutputStream in this way:

OutputStream outputStream = IoBuilder
            .forLogger(logger)
            .buildOutputStream();

Below is an example with Appium, starting it programmatically and controlling its log with log4j.

    final Logger logger = LogManager.getLogger(getClass());

    cap = new DesiredCapabilities();
    cap.setCapability("noReset", "false");

    //Build the Appium service
    builder = new AppiumServiceBuilder();
    builder.withIPAddress("127.0.0.1");
    builder.usingPort(4723);
    builder.withCapabilities(cap);
    builder.withArgument(GeneralServerFlag.SESSION_OVERRIDE);
    builder.withArgument(GeneralServerFlag.LOG_LEVEL,"debug");

    //Start the server with the builder
    service = AppiumDriverLocalService.buildService(builder);

    OutputStream outputStream = IoBuilder
            .forLogger(logger)
            .buildOutputStream();
    service.addOutPutStream(outputStream);

    service.start();

Hope this helps!!!

Solution 3

Source: http://sysgears.com/articles/how-to-redirect-stdout-and-stderr-writing-to-a-log4j-appender/

Blockquote

Log4j doesn't allow to catch stdout and stderr messages out of the box. However, if you are using third party components and have to log messages that they flush to the streams, then you can do a little trick and implement custom output stream that supports logging.

This has already been done by Jim Moore (see the LoggingOutputStream in log4j source code). The only issue is that the JimMoore's LoggingOutputStream requires org.apache.log4j.Category and org.apache.log4j.Priority which are now partially deprecated.

Here is modified LoggingOutputStream that avoids deprecated methods:

public class LoggingOutputStream extends OutputStream {

    /**
     * Default number of bytes in the buffer.
     */
    private static final int DEFAULT_BUFFER_LENGTH = 2048;

    /**
     * Indicates stream state.
     */
    private boolean hasBeenClosed = false;

    /**
     * Internal buffer where data is stored.
     */
    private byte[] buf;

    /**
     * The number of valid bytes in the buffer.
     */
    private int count;

    /**
     * Remembers the size of the buffer.
     */
    private int curBufLength;

    /**
     * The logger to write to.
     */
    private Logger log;

    /**
     * The log level.
     */
    private Level level;

    /**
     * Creates the Logging instance to flush to the given logger.
     *
     * @param log         the Logger to write to
     * @param level       the log level
     * @throws IllegalArgumentException in case if one of arguments
     *                                  is  null.
     */
    public LoggingOutputStream(final Logger log,
                               final Level level)
            throws IllegalArgumentException {
        if (log == null || level == null) {
            throw new IllegalArgumentException(
                    "Logger or log level must be not null");
        }
        this.log = log;
        this.level = level;
        curBufLength = DEFAULT_BUFFER_LENGTH;
        buf = new byte[curBufLength];
        count = 0;
    }

    /**
     * Writes the specified byte to this output stream.
     *
     * @param b the byte to write
     * @throws IOException if an I/O error occurs.
     */
    public void write(final int b) throws IOException {
        if (hasBeenClosed) {
            throw new IOException("The stream has been closed.");
        }
        // don't log nulls
        if (b == 0) {
            return;
        }
        // would this be writing past the buffer?
        if (count == curBufLength) {
            // grow the buffer
            final int newBufLength = curBufLength +
                    DEFAULT_BUFFER_LENGTH;
            final byte[] newBuf = new byte[newBufLength];
            System.arraycopy(buf, 0, newBuf, 0, curBufLength);
            buf = newBuf;
            curBufLength = newBufLength;
        }

        buf[count] = (byte) b;
        count++;
    }

    /**
     * Flushes this output stream and forces any buffered output
     * bytes to be written out.
     */
    public void flush() {
        if (count == 0) {
            return;
        }
        final byte[] bytes = new byte[count];
        System.arraycopy(buf, 0, bytes, 0, count);
        String str = new String(bytes);
        log.log(level, str);
        count = 0;
    }

    /**
     * Closes this output stream and releases any system resources
     * associated with this stream.
     */
    public void close() {
        flush();
        hasBeenClosed = true;
    }
}

Now you can catch messages that are flushed to stderr or stdout in the following way:

System.setErr(new PrintStream(new LoggingOutputStream(
        Logger.getLogger("outLog"), Level.ERROR)));

The log4j.properties configuration:

log4j.logger.outLog=error, out_log

log4j.appender.out_log=org.apache.log4j.RollingFileAppender
log4j.appender.out_log.file=/logs/error.log
log4j.appender.out_log.MaxFileSize=10MB
log4j.appender.out_log.threshold=error

Dmitriy Pavlenko, SysGears

Blockquote

Solution 4

Building on Arthur Neves answer, I transferred this for Slf4J. I also improved this a bit using StringBuffer and directly casting byte to char:

import java.io.OutputStream;

import org.slf4j.Logger;

public class LogOutputStream extends OutputStream {
    private final Logger logger;

    /** The internal memory for the written bytes. */
    private StringBuffer mem;

    public LogOutputStream( final Logger logger ) {
        this.logger = logger;
        mem = new StringBuffer();
    }

    @Override
    public void write( final int b ) {
        if ( (char) b == '\n' ) {
            flush();
            return;
        }
        mem = mem.append( (char) b );
    }

    @Override
    public void flush() {
        logger.info( mem.toString() );
        mem = new StringBuffer();
    }
}

Solution 5

Looking through the answers here, it seems that none of them account clearly for decoding of bytes into Strings (extends CharSequence). bytes are chars are not equivalent (see OutputStream vs Writer in Java). A simple non-latin character e.g. 羼 may be represented as a series of bytes: E7 BE BC (UTF-8 sequence of 羼).

Reasons why others don't account for specific encoding:

• (char) b will transform non-latin characters without interpreting UTF-8, so 羼 becomes ç¾¼, or "Róbert" becomes "Róbert" (oh, home many times I've seen this). You might be more familiar with this beauty:  (UTF-8 BOM)
• new String(bytes) creates a String "using the platform's default charset", which is dependent on where you run the code, so you will probably get different behavior on server and local machines. This is one better than (char)b, because you can at least specify the encoding.
• log4j-iostreams's IoBuilder by default also uses the platform's default charset, but is configurable. Also this is not a generic solution, only works if you use SLF4J over Log4j 2. Although that was OP's question 😅.

(Forgive my Kotlin, you can do exactly the same in Java with different syntax.)

private fun loggerStream(outputLine: (line: String) -> Unit): PipedOutputStream {
    val output = PipedOutputStream()
    val input = PipedInputStream(output).bufferedReader()
    thread(isDaemon = true) {
        input.lineSequence().forEach(outputLine)
    }
    return output
}

With this solution:

• The transition is performant (buffered)
• The coding is specified consistently (bufferedReader has a default parameter: charset = Charsets.UTF_8, can be changed if necessary)
• The overhead of encoding is in a background thread, although a bit strange, but this is how pipes work.
• The code is simple/high level
  (no byte arrays, indices, copies, String constructors, etc.)

---

Note: I'm using this to redirect Selenium ChromeDriver's output (defaults to stderr) into SLF4J over Log4J 2:

    val service = ChromeDriverService.createServiceWithConfig(options).apply {
        sendOutputTo(loggerStream(LoggerFactory.getLogger(ChromeDriver::class.java)::info))
    }
    val driver = ChromeDriver(service, options)

View more solutions

Comments

[hokr]

Happy to have found this. In my usage scenario the given LogOutputStream has the following problem, I needed to fix: 1. It relies on log4j instead commons-logging which I changed and I hard-coded logger.info() as commons-logging has no Levels (as far as I see in seconds) 1. It assumes newline to be \n which I needed to change to System.getProperty("line.separator") 1. Since its flush() does not check if mem is empty, you get an extra empty line on output when the wrapping Writer calls flush() after last println()

[Kalle Richter]

Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference.

[saygley]

and in your class Logger logger = Logger.getLogger(new Object(){}.getClass().getEnclosingClass()); OutputStream stdout = new PrintStream(new LogOutputStream(logger, Level.INFO)); ----Then write stuff into your stdout ;)

[Jimmy T.]

That's a very inefficient implementation though. Not only does it only write byte by byte, writing each line has a runtime and memory consumption of O(n²) with the new string it creates with each byte. It could already been made better by using a StringBuffer instead of String and reuse its memory after flushing