Django Setup Default Logging

Question

I can't seem to figure out how to setup a "default" logger for my Django installation. I would like to use Django 1.3's new LOGGING setting in settings.py.

I've looked at the Django Logging Doc's example, but it looks to me like they only setup handlers which will do logging for particular loggers. In the case of their example they setup handler for the loggers named 'django','django.request', and 'myproject.custom'.

All I want to do is setup a default logging.handlers.RotatingFileHandler which will handle all loggers by default. i.e., if I make a new module somewhere in my project and it is denoted by something like: my_app_name.my_new_module, I should be able to do this and have all logging goto the rotating file logs.

# In file './my_app_name/my_new_module.py'
import logging
logger = logging.getLogger('my_app_name.my_new_module')
logger.debug('Hello logs!') # <-- This should get logged to my RotatingFileHandler that I setup in `settings.py`!

Solution 1

Figured it out...

You set the 'catch all' logger by referencing it with the empty string: ''.

As an example, in the following setup I have the all log events getting saved to logs/mylog.log, with the exception of django.request log events which will be saved to logs/django_request.log. Because 'propagate' is set to False for my django.request logger, the log event will never reach the the 'catch all' logger.

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'formatters': {
        'standard': {
            'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'
        },
    },
    'handlers': {
        'default': {
            'level':'DEBUG',
            'class':'logging.handlers.RotatingFileHandler',
            'filename': 'logs/mylog.log',
            'maxBytes': 1024*1024*5, # 5 MB
            'backupCount': 5,
            'formatter':'standard',
        },  
        'request_handler': {
            'level':'DEBUG',
            'class':'logging.handlers.RotatingFileHandler',
            'filename': 'logs/django_request.log',
            'maxBytes': 1024*1024*5, # 5 MB
            'backupCount': 5,
            'formatter':'standard',
        },
    },
    'loggers': {
        '': {
            'handlers': ['default'],
            'level': 'DEBUG',
            'propagate': True
        },
        'django.request': {
            'handlers': ['request_handler'],
            'level': 'DEBUG',
            'propagate': False
        },
    }
}

Solution 2

As you said in your answer, Chris, one option to define a default logger is to use the empty string as its key.

However, I think the intended way is to define a special logger under the root key of the logging configuration dictionary. I found this in the Python documentation:

root - this will be the configuration for the root logger. Processing of the configuration will be as for any logger, except that the propagate setting will not be applicable.

Here's the configuration from your answer changed to use the root key:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'formatters': {
        'standard': {
            'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'
        },
    },
    'handlers': {
        'default': {
            'level':'DEBUG',
            'class':'logging.handlers.RotatingFileHandler',
            'filename': 'logs/mylog.log',
            'maxBytes': 1024*1024*5, # 5 MB
            'backupCount': 5,
            'formatter':'standard',
        },  
        'request_handler': {
            'level':'DEBUG',
            'class':'logging.handlers.RotatingFileHandler',
            'filename': 'logs/django_request.log',
            'maxBytes': 1024*1024*5, # 5 MB
            'backupCount': 5,
            'formatter':'standard',
        },
    },
    'root': {
        'handlers': ['default'],
        'level': 'DEBUG'
    },
    'loggers': {
        'django.request': {
            'handlers': ['request_handler'],
            'level': 'DEBUG',
            'propagate': False
        },
    }
}

To be fair, I can't see any difference in behaviour between the two configurations. It appears that defining a logger with an empty string key will modify the root logger, because logging.getLogger('') will return the root logger.

The only reason I prefer 'root' over '' is that it is explicit about modifying the root logger. In case you were curious, 'root' overrides '' if you define both, just because the root entry is processed last.

Solution 3

import logging
logger = logging.getLogger(__name__)

after add:

logging.basicConfig(
    level = logging.DEBUG,
    format = '%(name)s %(levelname)s %(message)s',
)

we may change format to:

format = '"%(levelname)s:%(name)s:%(message)s"  ',

or

format = '%(name)s %(asctime)s %(levelname)s %(message)s',

Solution 4

I made a quick sample to check what configuration is used when both root key and the empty '' logger are referenced in config dict.

import logging.config

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'fmt1': {
            'format': '[FMT1] %(asctime)-15s %(message)s',
        },
        'fmt2': {
            'format': '[FMT2] %(asctime)-15s %(message)s',
        }
    },
    'handlers': {
        'console1': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
            'formatter': 'fmt1',
        },
        'console2': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
            'formatter': 'fmt2',
        },
    },
    # First config for root logger: console1 -> fmt1
    'root': {
        'handlers': ['console1'],
        'level': 'DEBUG',
        'propagate': True,
    },
    'loggers': {
        # Second config for root logger: console2 -> fmt2
        '': {
            'handlers': ['console2'],
            'level': 'DEBUG',
            'propagate': True,
        },
    },
}

logging.config.dictConfig(LOGGING)

l1 = logging.getLogger()
l2 = logging.getLogger('')
root = logging.root

l1.info("l1")
l2.info("l2")
root.info("root logger")

Prints the following result:

[FMT1] 2018-12-18 17:24:47,691 l1
[FMT1] 2018-12-18 17:24:47,691 l2
[FMT1] 2018-12-18 17:24:47,691 root logger

indicating that configuration under root key has the highest priority. If the block is removed, the result is:

[FMT2] 2018-12-18 17:25:43,757 l1
[FMT2] 2018-12-18 17:25:43,757 l2
[FMT2] 2018-12-18 17:25:43,757 root logger

In both case, I was able to debug and determine that all three loggers (l1, l2 and root) referenced the same logger instance, the root logger.

Hope that will help others who, like me, were confused by the 2 different ways to configure the root logger.

View more solutions

Comments

[Admin]

Chris, the Django docs on this are not confusing. Thanks for this.

[rych]

Tiny correction: the comment implies sql logging would be affected by the django.request logger. To redirect sql logging, you'd define a logger for 'django.db'. The django.request logger handles 5xx & 4xx http responses.

[hobbes3]

In this helps other noobs like me: The logger will create the log files, but you have to create the logs/ folder first :-). Otherwise you'll get an error when you run ./manange.py runserver. @Chris W. Thanks for your example logging settings. It helped me out a lot!

[Arindam Roychowdhury]

Hi , can anyone tell , after the above config , how do i use the above config in my view function ? I mean , logger = logging.getLogger(someloggername) How do we use a handler in the view function?

[Chris W.]

@arindamroychowdhury With the above configuration if you do logger = logging.getLogger('foo'); logger.warn('bar'); then the default handler will catch that logging and something like <time> WARN: foo: bar will end up in logs/mylog.log

[Eino Mäkitalo]

Thanks it seems that this '' means root logger. This useful piece of information was not found in Django documentation.

[Daniel Sokolowski]

The 'django.request' IS required, even if you have the catch all logger - ERRORS in views are were not handled by the catch all logger in my case.

[berto]

This is great, and wanted to add my experience with disable_existing_loggers. When True, existing loggers are ... disabled; i.e. they stop working. If you, say, create a logger at the top of a module when the config is loaded it NO LONGER LOGS! I learned this the hard way: best practice is to get the logger when you need it and it will always "work" and have the latest configuration.

[Antony Hatchkins]

Yep, that's right, sorry for miscorrection! While using '' instead of 'root' is somewhat logical, I still find it a bit inconsistent of them to move root entry into the dict's root in the process of otherwise smooth transition from 2.6 fileConfig logic to 2.7 dictConfig one.