How to limit log file size in python
Solution 1
Lose basicConfig()
and use RotatingFileHandler:
import logging
from logging.handlers import RotatingFileHandler
log_formatter = logging.Formatter('%(asctime)s %(levelname)s %(funcName)s(%(lineno)d) %(message)s')
logFile = 'C:\\Temp\\log'
my_handler = RotatingFileHandler(logFile, mode='a', maxBytes=5*1024*1024,
backupCount=2, encoding=None, delay=0)
my_handler.setFormatter(log_formatter)
my_handler.setLevel(logging.INFO)
app_log = logging.getLogger('root')
app_log.setLevel(logging.INFO)
app_log.addHandler(my_handler)
while True:
app_log.info("data")
Solution 2
When you use logging.basicConfig with a file, the log is attached with a file handler to handle writing to the file. afterwards you created another file handler to the same file with logging.handlers.RotatingFileHandler
Now, once a rotate is needed, RotatingFileHandler is trying to remove the old file but it can't becuase there is an open file handler
this can be seen if you look directly at the log file handlers -
import logging
from logging.handlers import RotatingFileHandler
log_name = 'c:\\log.log'
logging.basicConfig(filename=log_name)
log = logging.getLogger()
handler = RotatingFileHandler(log_name, maxBytes=1024, backupCount=1)
log.addHandler(handler)
[<logging.FileHandler object at 0x02AB9B50>, <logging.handlers.RotatingFileHandler object at 0x02AC1D90>]
Solution 3
To use BasicConfig and RotatingFileHandler, add RotatingFileHandler as Handler in BasicConfig.
main.py:
import logging
rfh = logging.handlers.RotatingFileHandler(
filename='foo.log',
mode='a',
maxBytes=5*1024*1024,
backupCount=2,
encoding=None,
delay=0
)
logging.basicConfig(
level=logging.DEBUG,
format="%(asctime)s %(name)-25s %(levelname)-8s %(message)s",
datefmt="%y-%m-%d %H:%M:%S",
handlers=[
rfh
]
)
logger = logging.getLogger('main')
logger.debug("test")
other.py
import logging
class Other():
def __init(self):
self.logger = logging.getLogger('other')
self.logger.info("test2")
"test" will be written into foo.log with the tag 'main'
"test2" will be written into foo.log with the tag 'other'
Related videos on Youtube
imp
Updated on July 09, 2022Comments
-
imp almost 2 years
I am using windows 7 and python 2.7. I want to limit my log file size to 5MB. My app, when it starts, writes to log file, and then the app terminates. When my app starts again, it will write in same log file. So app is not continuously running. App initiates, processes and terminates.
My code for logging is:
import logging import logging.handlers logging.basicConfig(filename=logfile.log, level="info", format='%(asctime)s %(levelname)s %(funcName)s(%(lineno)d) %(message)s') logging.info("*************************************************")
I tried with RotatingFileHandler but it didn't work
logging.handlers.RotatingFileHandler(logFile, mode='a', maxBytes=5*1024*1024, backupCount=2, encoding=None, delay=0)
So, how can I enforce a file size limit in python?
-
J0HN almost 10 yearsRotatingFileHandler is a way to go. How exactly it didn't work?
-
imp almost 10 yearsmay be because app is not continuosly running or is there any wrong in coding
-
J0HN almost 10 yearsThat doesn't answer my question :) I'm not asking you to speculate on why it's so, but WHAT exactly is wrong with RotatingFileHandler. Errors, exceptions, crashes, anything? Does it write to log at all?
-
Shadow9043 almost 10 years@imp it doesn't matter how many times it gets interrupted. See getLogger. It will always return a reference to the same logger
-
-
geosmart over 4 yearsjust work in current py file ,how to make it work in other py files
-
Nazar about 3 yearsNo writes to foo.log happen for me (
-
Alex over 2 years@geosmart
logging.getLogger('root').info('some log')
in any python file should work -
Vishal Rangras almost 2 yearsFor Python 3, don't forget to update the delay flag from 0 to False. It may appear like an integer but it is actually a boolean parameter.