How does SLF4J support structured logging

Question

Anyone knows how structured logging is usually implemented with SLF4J?

Is there any open source already out there handling this?

Solution 1

Slf4j added support for structured logging (and fluent API) with v2.0.0 (Alpha as of Oct 2019):

• http://www.slf4j.org/api/org/slf4j/event/KeyValuePair.html
• http://www.slf4j.org/api/org/slf4j/event/LoggingEvent.html

• http://www.slf4j.org/apidocs/org/slf4j/spi/LoggingEventBuilder.html

  int newT = 15;
  int oldT = 16;
  
  // using classical API
  logger.debug("oldT={} newT={} Temperature changed.", newT, oldT);
  
  // using fluent API
  logger.atDebug()
          .addKeyValue("oldT", oldT)
          .addKeyValue("newT", newT)
          .log("Temperature changed.");          
  

Solution 2

If you use SLF4J in conjunction with Logback and Logstash, structured logging is supported with StructuredArguments. You can find documentation about this on the logstash logback encoder page on Github.

A simple example of how it works. This log line..

log.debug("Retrieved file {}", StructuredArguments.value("filename", upload.getOriginalFilename()))

..yields the following log json output:

{
  "filename": "simple.zip",
  "@timestamp": "2019-02-12T14:31:31.631+00:00",
  "severity": "DEBUG",
  "service": "upload",
  "thread": "http-nio-9091-exec-1",
  "logger": "some.great.ClassName",
  "message": "Retrieved file simple.zip"
}

Solution 3

There is an example in github which is implemented using SLF4J. Hope it will help you.

For more learning you can go through this tutorial.

1. Structured logging in logback using MDC
2. Master Java Logging

Solution 4

FYI - I've just open sourced a structured logging wrapper for SLF4J. We're using it at my day job to front logging into Splunk and it's been a big improvement. Maybe you will find it useful.

https://github.com/Randgalt/maple

You define a "schema" and then wrap an SLF4J logger. E.g.

public interface LoggingSchema {
    LoggingSchema name(String name);

    LoggingSchema date(Instant date);

    ... etc ...
}    

...

MapleLogger<LoggingSchema> logger = MapleFactory.getLogger(slf4j, LoggingSchema.class);
logger.info("my message", s -> s.name("john doe").date(Instant.now());

Generates an slf4j log ala:

slf4j.info("my message name=\"john doe\" date=2019-10-08T18:52:15.820177Z");

Solution 5

For anyone who stumbles upon this rather old question: As an alternative approach that is way more developer friendly than manual setting of each param into MDC, you could use a structured logger like https://github.com/jacek99/structlog4j ideally along with yaml or json formatter. Then it is really easy to feed the logs to ELK stack, and query all logs based on parameters(you wont have to create log entry parser manually, as all relevant fields will be there in structured form already). Or create your own simple logger on top of slf4j, that would take any varargs passed the the .log method and automatically create key-value pairs in MDC, and then you can pair it with a structured formatter e.g. if your runtime uses Logstash: https://github.com/logstash/logstash-logback-encoder#mdc-fields

View more solutions

Comments

[Boris the Spider]

It doesn't. SLF4j is a logging facade - it doesn't log anything. It merely provides a common API to other logging frameworks. This is a little like asking "how does JDBC handle compound indexes?"

[Ceki]

I am the author of SLF4J. Can you explain what you mean by structured logging?

[Fritz Duchardt]

@Ceki I believe structured logging in this context means added extra json fields to the log output, e.g. {"severity":"INFO", "custom-property", "test-value"}

[thg]

still in alpha ... slf4j seems dead

[gavenkoa]

Slf4j state is somewhat hanging. Ceki (the author) should partake project rights. It looks he keeps exclusive rights for the future and personally have no time to dedicate to the project. Everyone just migrates to Log4j 2.

[RAM]

@thg SLF4J seems far from dead. Not only did they released v2.0.0alpha2 last month, they are also releasing updates to SLF4J 1.7 rather frequently. Moreover, SLF4J is just a simple logging facade.

[thg]

@RAM The question was how slf4j does support structured logging. It does not matter if it is a facade. Also before the latest update last month the project was inactive for 18 months with no response from the owner at all.

[gavenkoa]

This Logstash Logback library is cool when your goal is Json files (when you use net.logstash.logback.encoder.LogstashEncoder). Unfortunately it is not compatible with Log4j or other logging frameworks and even incompatible with other Logback formatters. It only works with own supplied formaters ((