Logging in Django and gunicorn

Question

I'm running django application with gunicorn, and I can't see any log messages I'm wriging.

Here is the code that writes the logs:

logger = logging.getLogger(__name__)

def home_page(request):
    logger.warning('in home page')

(note: this code definitely runs, since this is a view that leads to the homepage)

This is my logs configuration from settings.py:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'stream': sys.stdout,
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
        },
    },
    'root': {'level': 'INFO'},
}

I run gunicorn as daemon with the following arguments:

--access-logfile ../access.log --error-logfile --log-level debug ../error.log

Both access.log and error.log are created and filled with gunicorn messages, but I can't see the messages I write.

Thanks

Solution 1

I have solved my problem. Providing the details so it might help somebody with similar issue.

Decided not to mix up with gunicorn and django logs and to create separate log file for django.

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
        },
        'logfile': {
            'level':'DEBUG',
            'class':'logging.FileHandler',
            'filename': BASE_DIR + "/../logfile",
        },
    },
    'root': {
        'level': 'INFO',
        'handlers': ['console', 'logfile']
    },
}

With this configuration every message written with severity >= INFO will be written to file "logfile" located just outside of source directory.

Solution 2

I'd like to add a 2021 answer for this, as server logs are so helpful and I couldn't find the answer I was looking for without having to merge a couple of different examples. I'm using django==3.1 and gunicorn==20.0.4 in a Dockerized application that's hosted on Heroku. I just wanted my Django server logs to show up both in my local stdout when I ran docker-compose up, and in papertrail logs on Heroku. This is working for me.

# In Dockerfile, I created a location for the logs, 
# and point there when starting gunicorn
RUN mkdir /logs
CMD gunicorn app.wsgi:application --bind 0.0.0.0:$PORT --workers 3 --capture-output --access-logfile /logs/gunicorn-access.log

Then in the Django settings, I used this for the logging_dict:

import logging.config
# Clear prev config
LOGGING_CONFIG = None
logging.config.dictConfig({
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'console': {
            'format': '%(message)s',
        },
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'formatter': 'console',
        },
    },
    'loggers': {
        'gunicorn': { # this was what I was missing, I kept using django and not seeing any server logs
            'level': 'INFO',
            'handlers': ['console'],
            'propagate': True,
        },
    },

})

Comments

[mirth23]

Thanks! I was having trouble finding a basic example like this that worked with gunicorn.

[Pavel Ryvintsev]

I think it is actually written to stderr by default. To write it to also to stdout you should add another handler with specified stream like this: 'console_stdout': { 'level': 'DEBUG', 'stream': sys.stdout, 'class': 'logging.StreamHandler' }

[ericn]

@PavelRyvintsev may I ask what the 'root' parameter for? Do we really need it?

[Pavel Ryvintsev]

Yes. The root parameter means that logs from all modules are handled by 'console' and 'logfile' handlers. You can configure various modules to be handled by different handlers.

[Anfernee]

Won't this introduce issues if you run multiple instances of the app, all writing to that logfile?

[Pavel Ryvintsev]

I don't think so. For example, lots of applications write to /var/log/syslog simultaneously and it does not cause any trouble.

[Anfernee]

Thats because they don't all write to the file. They either send their logs to syslog through the syslog protocol or they write to stdout/stderr and the process manager captures the logs and sends it to syslog. You should probably take a look at this github.com/benoitc/gunicorn/issues/157

[amsh]

Can you please point out to the location error logs and app logs are printed?

[Anthony Roberts]

This prints logs to stdout, the StreamHandler default, as stated in the opening paragraph. So the location is stdout, accessible via papertrail.

[Larry]

I wish I could upvote this a thousand times. This is the only mention of loggers.gunicorn I've found in 2 days of banging my head against a wall attempting to get New Relic "logs in context" to work with gunicorn

[Anthony Roberts]

Thanks Larry. This comment made me smile, I'm glad it helped.