How to create a new log file every time the application runs?

Question

Currently, this is what I have (testlog.py):

import logging
import logging.handlers

filename = "example.log"

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")

handler = logging.handlers.RotatingFileHandler(filename, mode = 'w', backupCount = 5)
handler.setLevel(logging.DEBUG)
handler.setFormatter(formatter)
logger.addHandler(handler)

ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
ch.setFormatter(formatter)
logger.addHandler(ch)

for i in range(10):
   logger.debug("testx") #where I alternate x from 1 thru 9 to see output

It currently successfully prints out to the console and to example.log, which is what I want.

Every time I run it, it makes a new file and replaces the old example.log like so:

• run with logger.debug("test1") - example.log will contain test1 10 times like it should.

• run with logger.debug("test2") - it rewrites example.log to contain test2 10 times.

• etc...

However, I would like for the code to make a new log file every time I run the program so that I have:

example.log
example.log1
example.log2 
...
example.log5

In conclusion, I'd like for this file to print the log message to the console, to the log file, and I would like a new log file (up to *.5) whenever I run the program.

Solution

logging.handlers.RotatingFileHandler rotates your logs based either on size or on date, but you can force it to rotate using RotatingFileHandler.doRollover() so something like:

import logging.handlers
import os

filename = "example.log"

# your logging setup

should_roll_over = os.path.isfile(filename)
handler = logging.handlers.RotatingFileHandler(filename, mode='w', backupCount=5)
if should_roll_over:  # log already exists, roll over!
    handler.doRollover()

# the rest of your setup...

Should work like a charm.

Comments

[practicemakesperfect]

SOLVED! all I did was add logging.handlers.RotatingFileHandler.doRollover(handler) at the end

[zwer]

Don't do it at the end - if your script fails mid-point end doesn't reach the end you won't be getting the new log file on the next start. Do it when initializing your logging.

[ybonda]

it creates empty log files on each run

[Dman Cannon]

set delay = True, it avoids creation of empty log files

[Vrishank]

How do I add a timestamp instead of .log.1, .log.2 ?

[zwer]

@Vrishank RotatingFileHandler uses the base filename to determine the rotated log name, you cannot set it to use timestamps or anything else except the <base_name>.<n> for its rotated files. That being said, this being Python, you can always override the internals to have it do your bidding, you can check out one way to do it in this answer.

[Andrei]

@ybonda, it works for me if I remove mode = 'w' from RotatingFileHandler