Easy application logging/debugging with nginx, uwsgi, flask?

Question

I'm not looking to turn on the dangerous debugging console, but my application is getting a 500 error and doesn't seem to be writing any output for me to investigate more deeply.

I saw this exchange on the mailing list, which led me to this page on logging errors.

However, I still find this very confusing and have a couple of questions:

(1) In which file should the stuff below go?

ADMINS = ['[email protected]']
if not app.debug:
    import logging
    from logging.handlers import SMTPHandler
    mail_handler = SMTPHandler('127.0.0.1',
                               '[email protected]',
                               ADMINS, 'YourApplication Failed')
    mail_handler.setLevel(logging.ERROR)
    app.logger.addHandler(mail_handler)

...assuming the "getting bigger" file pattern for larger applications? __init__.py? config.py? run.py?

(2) I am overwhelmed by options there, and can't tell which I should use. Which loggers should I turn on, with what settings, to replicate the local python server debug I get to stdout when I run run.py? I find that default, local output stream very useful, more so than the interactive debugger in the page. Does anyone have a pattern they could share on setting up something replicating this with an nginx deployment, outputting to a log?

(3) Is there anything I need to change, not at the flask level, but in nginx, say in my /etc/nginx/sites-available/appname file, to enable logging?

UPDATE

Specifically, I'm looking for information like I get when python runs locally as to why, say, a package isn't working, or where my syntax error might be, or what variable doesn't exist:

$ python run.py 
Traceback (most recent call last):
  File "run.py", line 1, in <module>
    from myappname import app
  File "/home/me/myappname/myappname/__init__.py", line 27, in <module>
    file_handler.setLevel(logging.debug)
  File "/usr/lib/python2.7/logging/__init__.py", line 710, in setLevel
    self.level = _checkLevel(level)
  File "/usr/lib/python2.7/logging/__init__.py", line 190, in _checkLevel
    raise TypeError("Level not an integer or a valid string: %r" % level)

When I run flask on a server, I never see this. I just get a uWSGI error in the browser, and have no idea which code was problematic. I would just like something like the above to be written to a file.

I notice also that setting the following logging didn't really write much to file, even when I turn the log way up to the DEBUG level:

from logging import FileHandler
file_handler = FileHandler('mylog.log')
file_handler.setLevel(logging.DEBUG)
app.logger.addHandler(file_handler)

mylog.log is blank, even when my application errors out.

I'll also add that I've tried to set debug = True in the following ways, in __init__.py:

app = Flask(__name__)
app.debug = True
app.config['DEBUG'] = True
from werkzeug.debug import DebuggedApplication
app.wsgi_app = DebuggedApplication(app.wsgi_app, True)
app.config.from_object('config')
app.config.update(DEBUG=True)
app.config['DEBUG'] = True
if __name__ == '__main__':
    app.run(debug=True)

While in my config.py file, I have...

debug = True
Debug = True
DEBUG = True

Yet, no debugging happens, and without logging or debugging, this is rather hard to track down. Errors simply terminate the application with the un-useful browser message:

uWSGI Error
Python application not found

Solution 1

Set config['PROPAGATE_EXCEPTIONS'] to True when running app in production and you want tracebacks to be logged into log files. (I haven't tried with SMTP handler, though..)

Solution 2

1. The part where you create handlers, add to loggers etc. should be in the if __name__ == '__main__' clause, i.e. your main entry point. I assume that would be run.py.
2. I'm not sure I can answer this - it depends on what you want. I'd advise looking at the logging tutorial to see the various options available.
3. I don't believe you need to change anything at the nginx level.

Update: You might want to have an exception clause that covers uncaught exceptions, e.g.

if __name__ == '__main__':
    try:
        app.run(debug=True)
    except Exception:
        app.logger.exception('Failed')

which should write the traceback of any exception which occurred in app.run() to the log.

Solution 3

I know that this is a VERY old post, but I ran into the issue now, and it took me a bit to find the solution. Flask sends errors to the server. I was running Gunicorn with an upstart script on Ubuntu 14.04 LTS, and the place where I found the error logs was as follows:

/var/log/upstart/myapp.log

http://docs.gunicorn.org/en/stable/deploy.html#upstart

Just in case some other poor soul ends up in this situation.