How can I log outside of main Flask module?

Question

I have a Python Flask application, the entry file configures a logger on the app, like so:

app = Flask(__name__)
handler = logging.StreamHandler(sys.stdout)
app.logger.addHandler(handler)
app.logger.setLevel(logging.DEBUG)

I then do a bunch of logging using

app.logger.debug("Log Message")

which works fine. However, I have a few API functions like:

@app.route('/api/my-stuff', methods=['GET'])
def get_my_stuff():
    db_manager = get_manager()
    query = create_query(request.args)

    service = Service(db_manager, query)
    app.logger.debug("Req: {}".format(request.url))

What I would like to know is how can I do logging within that Service module/python class. Do I have to pass the app to it? That seems like a bad practice, but I don't know how to get a handle to the app.logger from outside of the main Flask file...

Solution

Even though this is a possible duplicate I want to write out a tiny bit of python logging knowledge.

DON'T pass loggers around. You can always access any given logger by logging.getLogger(<log name as string>). By default it looks like* flask uses the name you provide to the Flask class.

So if your main module is called 'my_tool', you would want to do logger = logging.getLogger('my_tool')in the Service module.

To add onto that, I like to be explicit about naming my loggers and packages, so I would do Flask('my_tool')** and in other modules, have sub level loggers like. logger = logging.getLogger('my_tool.services') that all use the same root logger (and handlers).

* No experience, based off other answer.

** Again, don't use flask, dk if that is good practice

Edit: Super simple stupid example

Main Flask app

import sys
import logging

import flask

from module2 import hi

app = flask.Flask('tester')

handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logging.Formatter(
    '%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
app.logger.addHandler(handler)
app.logger.setLevel(logging.DEBUG)

@app.route("/index")
def index():
    app.logger.debug("TESTING!")
    hi()
    return "hi"

if __name__ == '__main__':
    app.run()

module2

import logging

log = logging.getLogger('tester.sub')


def hi():
    log.warning('warning test')

Outputs

127.0.0.1 - - [04/Oct/2016 20:08:29] "GET /index HTTP/1.1" 200 -
2016-10-04 20:08:29,098 - tester - DEBUG - TESTING!
2016-10-04 20:08:29,098 - tester.sub - WARNING - warning test

Edit 2: Messing with subloggers

Totally unneeded, just for general knowledge.

By defining a child logger, done by adding a .something after the root logger name in logging.getLogger('root.something') it gives you basiclly a different namespace to work with.

I personally like using it to group functionality in logging. So have some .tool or .db to know what type of code is logging. But it also allows so that those child loggers can have their own handlers. So if you only want some of your code to print to stderr, or to a log you can do so. Here is an example with a modified module2.

module2

import logging
import sys

log = logging.getLogger('tester.sub')
handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(logging.Formatter('%(name)s - %(levelname)s - %(message)s'))
log.addHandler(handler)
log.setLevel(logging.INFO)


def hi():
    log.warning("test")

Output

127.0.0.1 - - [04/Oct/2016 20:23:18] "GET /index HTTP/1.1" 200 -
2016-10-04 20:23:18,354 - tester - DEBUG - TESTING!
tester.sub - WARNING - test
2016-10-04 20:23:18,354 - tester.sub - WARNING - test

Comments

[user4184113]

One question - what is the sub part in tester.sub? Is sub the name of the submodule?

[CasualDemon]

Nope, it's anything you want it to be, set at logging.getLogger('tester.sub') in module2. It's literally a string I set at random, however because it's root was tester before the .sub, it uses those handlers set in the main app. To get fancy, you can also assign additional handlers that only affect that sub logger in module2 then.

[CasualDemon]

@user4184113 added info about child loggers to my answer for you. It's really not needed for what you are looking for, simply fun knowledge to share.

[Trondh]

I'm trying to figure out multi-module logging in Flask. One thing I can't figure out: If module2 is a "generic" library which is used in multiple apps it doesn't make sense to name it (tester.sub). How can I make the "parent" module (the main flask app) setup logging so that imported modules simply inherit the config without injecting app-specific settings?

[CasualDemon]

I would have it use it's own custom name for the logger, then set up that logger separably (can have same format and even write to same file or screen of course). When dealing with multiple loggers and larger projects, I find it useful to define logging in a file and load it through logging.dictConfig(my_settings), example two loggers using same handler stackoverflow.com/questions/7507825/…

[ryanjdillon]

This helped me with Flask, but also some good general logging tidbits.

[cp-stack]

After spending two days working on logging in Flask, this was the answer that solved it for me. Thanks so much!

[Adam Hughes]

Does this still work when the app creation is nested in a function? IE create_app()? Cant' get this to work (the second module doesn't pickup the logger correctly)

[Karel Macek]

Many thanks. Your solution really works. In my case, I had to very systematically prune all other attempts first. And also had to app.logger.handlers.clear() before app.logger.addHandler(handler) to avoid double logging.