Using python Logging with AWS Lambda

Question

As the AWS documentation suggests:

import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
def my_logging_handler(event, context):
    logger.info('got event{}'.format(event))
    logger.error('something went wrong')

Now I made:

import logging
logging.basicConfig(level = logging.INFO)
logging.info("Hello World!")

The first snippet of code prints in the Cloud Watch console, but the second one no.

I didn't see any difference as the two snippets are using the root logger.

Solution 1

The reason that logging does not seem to work is because the AWS Lambda Python runtime pre-configures a logging handler that, depending on the version of the runtime selected, might modify the format of the message logged, and might also add some metadata to the record if available. What is not preconfigured though is the log-level. This means that no matter the type of log-message you try to send, it will not actually print.

As AWS documents themselves, to correctly use the logging library in the AWS Lambda context, you only need to set the log-level for the root-logger:

import logging
logging.getLogger().setLevel(logging.INFO)

If you want your Python-script to be both executable on AWS Lambda, but also with your local Python interpreter, you can check whether a handler is configured or not, and fall back to basicConfig (which creates the default stderr-handler) otherwise:

if len(logging.getLogger().handlers) > 0:
    # The Lambda environment pre-configures a handler logging to stderr. If a handler is already configured,
    # `.basicConfig` does not execute. Thus we set the level directly.
    logging.getLogger().setLevel(logging.INFO)
else:
    logging.basicConfig(level=logging.INFO)

Solution 2

Copied straight from the top answer in the question @StevenBohrer's answer links to (this did the trick for me, replacing the last line with my own config):

root = logging.getLogger()
if root.handlers:
    for handler in root.handlers:
        root.removeHandler(handler)
logging.basicConfig(format='%(asctime)s %(message)s',level=logging.DEBUG)

Solution 3

I had a similar problem, and I suspect that the lambda container is calling logging.basicConfig to add handlers BEFORE the lambda code is imported. This seems like bad form...

Workaround was to see if root logger handlers were configured and if so, remove them, add my formatter and desired log level (using basicConfig), and restore the handlers.

See this article Python logging before you run logging.basicConfig?

Solution 4

I've struggled with this exact problem. The solution that works both locally and on AWS CloudWatch is to setup your logging like this:

import logging

# Initialize you log configuration using the base class
logging.basicConfig(level = logging.INFO)

# Retrieve the logger instance
logger = logging.getLogger()

# Log your output to the retrieved logger instance
logger.info("Python for the win!")

Solution 5

Probably not referencing the same logger, actually. In the first snippet, log the return of: logging.Logger.manager.loggerDict

It will return a dict of the loggers already initialized.

Also, from the logging documentation, an important note on logging.basicConfig:

Does basic configuration for the logging system by creating a StreamHandler with a default Formatter and adding it to the root logger. The functions debug(), info(), warning(), error() and critical() will call basicConfig() automatically if no handlers are defined for the root logger.

This function does nothing if the root logger already has handlers configured for it.

Source: https://docs.python.org/2/library/logging.html#logging.basicConfig

View more solutions