Logging globally (across packages)

Question

I have spent a lot of time now searching and reading posts on the subject but have note yet managed to fully get my question answered - or perhaps I just need more clarification on existing answers.

I have read this post which has the same title as mine, but it's about logging across go routines rather than packages.

What I'm trying to solve is logging across the main app and any packages it uses. I need a logger that can log to multiple locations (which can be done with io.MultiWriter) and can do things like log.Error() and log.Debug()

I know there are packages that do this and I know how to implement those things myself.

What I can't get my head around is how to properly use it with my packages.

One way is of course to create the logger in main and then pass it around to all functions that need logging. But it seems awkward.

My ideal solution would be to have a logger like the built in global logger from the log package, but with the added functionality as above.

I mostly want this for optional debug logging within packages, so I can turn this on in a production version if needed.

What is the proper way to do this?

Solution 1

The proper way is what you think is the ideal way. Just create a package (preferably following Go's conventions https://golang.org/doc/code.html) and make your Log global:

package mylog

// Define your custom logger type.
type logger struct { /* Whatever you want */ }

// Optionally make it a interface.
type Logger interface { /* Your functions */ }

// And just go global.
var defaultLogger *Logger

func init(){
   defaultLogger = new(logger)
}

func Debug(params ...string){
   // Have some fun.
}

// ...

Also I would recommend to describe in documentation that your project uses that logging feature.

Solution 2

I'm posting the solution worked for me!. I just created my own package, and I used the init function.

package logging

import (
    "io"
    logging "log"
    "os"

    "github.com/Sirupsen/logrus"
)

var (
    log *logrus.Logger
)

func init() {
    f, err := os.OpenFile("logs/application.log", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)

    if err != nil {
        logging.Fatalf("error opening file: %v", err)
    }

    log = logrus.New()

    //log.Formatter = &logrus.JSONFormatter{}

    log.SetReportCaller(true)

    mw := io.MultiWriter(os.Stdout, f)
    log.SetOutput(mw)
}

// Info ...
func Info(format string, v ...interface{}) {
    log.Infof(format, v...)
}

// Warn ...
func Warn(format string, v ...interface{}) {
    log.Warnf(format, v...)
}

// Error ...
func Error(format string, v ...interface{}) {
    log.Errorf(format, v...)
}

var (

    // ConfigError ...
    ConfigError = "%v type=config.error"

    // HTTPError ...
    HTTPError = "%v type=http.error"

    // HTTPWarn ...
    HTTPWarn = "%v type=http.warn"

    // HTTPInfo ...
    HTTPInfo = "%v type=http.info"
)

And on any package, just import my package and I execute the (Info, Warn, Error) function

package main

import (

    log "logging"
)

func main() {
    log.Error(log.ConfigError, "Testing the error")
}

The log entry will be saved rendered on the screen and it will be saved on a file.

Solution 3

@CedmundoMartinez answer rattled my head enough to come up with the (extremely simple and rather obvious, now that I can use my rear view mirrors) answer.

I'm posting my answer here for anyone who is interested in a similar solution.

What I did was to make a copy of the standard log package (src/log/log.go) and extend it. It couldn't be easier to get a global logger that already does everything the standard logger does plus anything else you want it to do! In this case support leveled logging.

The only modifications I had to make:

type Logger struct {
    mu     sync.Mutex // ensures atomic writes; protects the following fields
    prefix string     // prefix to write at beginning of each line
    flag   int        // properties
    out    io.Writer  // destination for output
    buf    []byte     // for accumulating text to write
    level  int        // One of DEBUG, ERROR, INFO
}

Only the last line was added. The log package sets a global variable std which can then be used to access the struct fields from any function in the package.

Next I added constants for the different log levels:

const (
    DEBUG = 1 << iota
    INFO
    ERROR
)

Next I added my functions:

(Note: ct is package https://github.com/seago/go-colortext which allows to color console text on windows. So the errors here all print in red)

func Error(v ...interface{}) {
    if std.level <= ERROR {
        ct.ChangeColor(ct.Red, true, ct.None, false)
        s := fmt.Sprintf("ERROR: %v", v...)
        std.Output(2, s)
        ct.ResetColor()
    }
}

func Info(format string, v ...interface{}) {
    if std.level <= INFO {
        s := fmt.Sprintf("INFO: "+format, v...)
        std.Output(2, s)
    }
}

func Debug(v ...interface{}) {
    if std.level <= DEBUG {
        s := fmt.Sprintf("DEBUG: %v", v...)
        std.Output(2, s)
    }
}

func SetLogLevel(lvl int) {
    std.level = lvl
}

And that's it! With that I can now use it by simply importing the modified package instead of the standard log package and log away:

import (
    "errors"
    "tryme/log" 
)

func main() {
    log.SetLogLevel(log.INFO)
    log.Info("This is a test Info")
    err := errors.New("This is a test error!!!")
    log.Error(err)
    log.Debug("Testing debugging") // won't be printed with log.INFO
}

This is of course just a demo and can easily be extended further with more log levels, output formatting etc.

You can use all the functions the standard log package provides like SetOutput to write to a file or a MultiWriter to write to file and console etc.

Solution 4

To add, if you want to do logging over multiple Go applications, you can use RPC and create a logging service. This would be an application sits on its own, providing services to other applications. Golang has its own package for it

View more solutions

Comments

[ANisus]

Maybe you can try out github.com/golang/glog . While it doesn't have Debug, it has Info, Warning, Error and Fatal. It also allows verbosity level logging. Hmm, but it doesn't allow setting writer either. Oh well! Worth a comment

[rpkamp]

Why did you copy the whole logger and not just embed Logger in a new struct of your own, that would then also implement Error, Info etc? It seems a bit overkill to copy it all...

[IamNaN]

@ScallioXT - Because I wanted a global logger. If I make my own type and embed the standard logger I would then have to wrap all the functions (those that aren't methods) of the log package to be able to access them globally. And that would be a lot more work then doing what I did above, but I don't see how else I'd do that. Perhaps I'm missing something. If so please point me in the right direction.

[rpkamp]

I thought it would be easier to just embed log.Logger, but I just tried it and you get into trouble that the fields of log.Logger are not exported so you can't refer to them. To solve that you'd have to jump though a lot of hoops and it's indeed not worth it. Just copying it all over like you're doing now seems the best solution as far as I can see.

[JeyJ]

why did you use the github.com/Sirupsen/logrus instead of the regular logging package of go ?