How to write log to file

Question

I'm trying to write to a log file with Go.

I have tried several approaches, all of which have failed. This is what I have tried:

func TestLogging(t *testing.T) {
    if !FileExists("logfile") {
        CreateFile("logfile")
    }
    f, err := os.Open("logfile")
    if err != nil {
        t.Fatalf("error: %v", err)
    }

    // attempt #1
    log.SetOutput(io.MultiWriter(os.Stderr, f))
    log.Println("hello, logfile")

    // attempt #2
    log.SetOutput(io.Writer(f))
    log.Println("hello, logfile")

    // attempt #3
    log.SetOutput(f)
    log.Println("hello, logfile")
}

func FileExists(name string) bool {
    if _, err := os.Stat(name); err != nil {
       if os.IsNotExist(err) {
            return false
        }
    }
    return true
}

func CreateFile(name string) error {
    fo, err := os.Create(name)
    if err != nil {
        return err
    }
    defer func() {
        fo.Close()
    }()
    return nil
}

The log file gets created, but nothing ever gets printed or appended to it. Why?

Solution 1

os.Open() must have worked differently in the past, but this works for me:

f, err := os.OpenFile("testlogfile", os.O_RDWR | os.O_CREATE | os.O_APPEND, 0666)
if err != nil {
    log.Fatalf("error opening file: %v", err)
}
defer f.Close()

log.SetOutput(f)
log.Println("This is a test log entry")

Based on the Go docs, os.Open() can't work for log.SetOutput, because it opens the file "for reading:"

func Open

func Open(name string) (file *File, err error) Open opens the named file for reading. If successful, methods on the returned file can be used for reading; the associated file descriptor has mode O_RDONLY. If there is an error, it will be of type *PathError.

EDIT

Moved defer f.Close() to after if err != nil check

Solution 2

I prefer the simplicity and flexibility of the 12 factor app recommendation for logging. To append to a log file you can use shell redirection. The default logger in Go writes to stderr (2).

./app 2>> logfile

See also: http://12factor.net/logs

Solution 3

I usually print the logs on screen and write into a file as well. Hope this helps someone.

f, err := os.OpenFile("/tmp/orders.log", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
if err != nil {
    log.Fatalf("error opening file: %v", err)
}
defer f.Close()
wrt := io.MultiWriter(os.Stdout, f)
log.SetOutput(wrt)
log.Println(" Orders API Called")

Solution 4

This works for me

1. created a package called logger.go

   package logger
   
   import (
     "flag"
     "os"
     "log"
     "go/build"
   )
   
   var (
     Log      *log.Logger
   )
   
   
   func init() {
       // set location of log file
       var logpath = build.Default.GOPATH + "/src/chat/logger/info.log"
   
      flag.Parse()
      var file, err1 = os.Create(logpath)
   
      if err1 != nil {
         panic(err1)
      }
         Log = log.New(file, "", log.LstdFlags|log.Lshortfile)
         Log.Println("LogFile : " + logpath)
   }
   

   2. import the package wherever you want to log e.g main.go

      package main
      
      import (
         "logger"
      )
      
      const (
         VERSION = "0.13"
       )
      
      func main() {
      
          // time to use our logger, print version, processID and number of running process
          logger.Log.Printf("Server v%s pid=%d started with processes: %d", VERSION, os.Getpid(),runtime.GOMAXPROCS(runtime.NumCPU()))
      
      }
      

Solution 5

If you run binary on linux machine you could use shell script.

overwrite into a file

./binaryapp > binaryapp.log

append into a file

./binaryapp >> binaryapp.log

overwrite stderr into a file

./binaryapp &> binaryapp.error.log

append stderr into a file

./binaryapp &>> binalyapp.error.log

it can be more dynamic using shell script file.

View more solutions

Comments

[nvcnvn]

If you deploy your program in Linux you can just write your log to std output then pipe the output to a file like: ./program 2>&1 | tee logs.txt. There must be some other way in other system.

[Volker]

Do not defer Close before checking err for nil!

[Dustin]

It's not activity actually harmful to close in all cases iirc. That isn't true for all types, though.

[nemo]

@Dustin f might be nil, which would result in a panic. So checking for err before deferring the call is advisable.

[nemo]

@AllisonA care to explain why Open won't work with log.SetOutput?

[Allison A]

@nemo because os.Open() is O_RDONLY (readonly cursor), and for the log to write it needs to be writeable. RE the defer, I was assuming that if err was not nil, that f might still have a value, is that not true?

[nemo]

@AllisonA ah, sorry, missed the read-only part, now it makes sense :). Re. defer, it is common to return a nil-value when an error occurs as it is often impossible to return a valid object (due to that error). So you can't rely on calling anything on the returned value when err is non-nil.

[Jonathan]

The safer permissions are 0644 or even 0664 to allow user read/write, user and group read/write, and in both cases disallow everyone write.

[Shrey]

wont be a good practice when you want to daemonize things, esp with start-tsop-daemon

[WarGasm]

@Shrey Systemd could easily take care of logging, as well as about start-stop functions.

[openwonk]

Hey @CostaHuang, please leave detailed feedback. Thanks

[openwonk]

@CostaHuang, I just ran my code snippet and it works.

[Costa Huang]

Hi @openwonk, I have tested again and it did not work on my computer. My version is go version go1.10.2 windows/amd64, what's yours?

[openwonk]

@CostaHuang, I just ran example with same set up as you. The example assumes that you already have a folder structure set up. There are easy ways to check for this, however my goal with example is to show how relatively simple writing to a log file is. Change your code to outfile, _ = os.Create("my.log") and it will work as expected.

[Costa Huang]

Your code works. I was using outfile, _ = os.Create("./path/to/my.log"). Somehow I had the expectation that the code will create the path/to folders and the my.log file, but apparently it didn't work. I would suggest that you modify your answer to be outfile, _ = os.Create("./my.log"). That way we clearly know that it's creating a log in the current folder.

[impossible]

Nice to know, how do we override stderr to log.

[addicted]

Despite this is a good practice or not, this is the type of logging that I have been looking for in Golang. Thanks for sharing this!

[surfmuggle]

Is there something similar under windows?

[Ryosuke Hujisawa]

Was like $ cd /etc/systemd/system $ sudo vi app.service ExecStart=/bin/bash -c 'sudo go run main.go >> /home/ubuntu/go/src/html_menu_1/logfile' Me NOT work Ubuntu 18.04.3

[Parm]

f.Close returns an error value that isn't being checked. Would it be better to wrap f.Close in some function that checks the error?

[Dean coakley]

Afaik this leaves an open file descriptor open for file. (Missing defer file.Close() after err check) Is that a concern?

[Gamora]

I'm using a very similar method to this. I don't want to keep logs indefinitely though. Do you have a way to age off older logs? I'm aware that you can't just delete lines from the file