How to debug in Django, the good way?

Question

So, I started learning to code in Python and later Django. The first times it was hard looking at tracebacks and actually figure out what I did wrong and where the syntax error was. Some time has passed now and some way along the way, I guess I got a routine in debugging my Django code. As this was done early in my coding experience, I sat down and wondered if how I was doing this was ineffective and could be done faster. I usually manage to find and correct the bugs in my code, but I wonder if I should be doing it faster?

I usually just use the debug info Django gives when enabled. When things do end up as I thought it would, I break the code flow a lot with a syntax error, and look at the variables at that point in the flow to figure out, where the code does something other than what I wanted.

But can this be improved? Are there some good tools or better ways to debug your Django code?

Solution 1

There are a bunch of ways to do it, but the most straightforward is to simply use the Python debugger. Just add following line in to a Django view function:

import pdb; pdb.set_trace()

or

breakpoint()  #from Python3.7

If you try to load that page in your browser, the browser will hang and you get a prompt to carry on debugging on actual executing code.

However there are other options (I am not recommending them):

* return HttpResponse({variable to inspect})

* print {variable to inspect}

* raise Exception({variable to inspect})

But the Python Debugger (pdb) is highly recommended for all types of Python code. If you are already into pdb, you'd also want to have a look at IPDB that uses ipython for debugging.

Some more useful extension to pdb are

pdb++, suggested by Antash.

pudb, suggested by PatDuJour.

Using the Python debugger in Django, suggested by Seafangs.

Solution 2

I really like Werkzeug's interactive debugger. It's similar to Django's debug page, except that you get an interactive shell on every level of the traceback. If you use the django-extensions, you get a runserver_plus managment command which starts the development server and gives you Werkzeug's debugger on exceptions.

Of course, you should only run this locally, as it gives anyone with a browser the rights to execute arbitrary python code in the context of the server.

Solution 3

A little quickie for template tags:

@register.filter 
def pdb(element):
    import pdb; pdb.set_trace()
    return element

Now, inside a template you can do {{ template_var|pdb }} and enter a pdb session (given you're running the local devel server) where you can inspect element to your heart's content.

It's a very nice way to see what's happened to your object when it arrives at the template.

Solution 4

There are a few tools that cooperate well and can make your debugging task easier.

Most important is the Django debug toolbar.

Then you need good logging using the Python logging facility. You can send logging output to a log file, but an easier option is sending log output to firepython. To use this you need to use the Firefox browser with the firebug extension. Firepython includes a firebug plugin that will display any server-side logging in a Firebug tab.

Firebug itself is also critical for debugging the Javascript side of any app you develop. (Assuming you have some JS code of course).

I also liked django-viewtools for debugging views interactively using pdb, but I don't use it that much.

There are more useful tools like dozer for tracking down memory leaks (there are also other good suggestions given in answers here on SO for memory tracking).

Solution 5

I use PyCharm (same pydev engine as eclipse). Really helps me to visually be able to step through my code and see what is happening.

View more solutions

Comments

[Tom Christie]

See also django-pdb as per my answer below. Gives you manage.py runserver --pdb and manage.py test --pdb commands.

[Phob]

@Daniel, see rconsole for having a console into an already running instance of python.

[Muhammad Rashid Afridi]

This would be great if you could specify the file / line number to break at (not just the view).

[hobbes3]

Check out ipython as well. Ipdb, which comes with ipython, features tab completion, colored syntax, and more :-).

[synthesizerpatel]

You do have to run the dev server through eclipse (for the low-effort debugging experience). PyDev claims to have remote debugging but having never used it I can't really speak to the quality of development experience. Details: pydev.org/manual_adv_remote_debugger.html

[Tayla Wilson]

this is great. Another thing you can do if you're having template problems is to switch to jinja2 (loaded through coffin) - it's an extension of django templates, which is an improvement in my opinion. It also integrates templates & template inheritance into traceback frames way better than django does.

[Tom Gruner]

Thanks, that worked great. The --noreload was what we needed

[floer_m]

I found your answer useful but Django was hanging forever on my breakpoints, when I was trying to debug a test. So I looked and found an informative article that helped me out: v3.mike.tig.as/blog/2010/09/14/pdb

[Glycerine]

To which I could leave in the code like comments of which are inert within production. Perhaps this is a bad paradim, but it would be great to effectively strip and apply breaks willy-nilly.

[Michael Bylstra]

best thing about it is it just works and is totally intuitive. Just click to the left of a line and hit the debug button. It works well for Django source code too if you want to get a better understanding of how the internal code works. It took me a while before I noticed it, but you can put breakpoints in any of the code in External Libraries folder of the file navigator.

[Daniel Naab]

raise Exception('garbage') actually works fantastically well when coupled with the Werkzeug debugger, considering that in most cases you simply want to know what locals exist at some scope.

[Daniel Sokolowski]

Is there a feature to debug on remote server - similar to Eclipse PyDev that I use at the moment?

[Medeiros]

Worth to mention that PyCharm uses PyDev debugger under the hood for credits.

[Seperman]

epdb is not updated since 2011. Do you ever run into problems using it on newer versions of Django and/or Python?

[Jacinda]

I've never run into issues using it against Python 2 (specifically 2.4-2.7). I used it just a few days ago, in fact. I've never tried with Python 3.

[blfuentes]

I am having problems with this. I followed your steps but still doesn't work. It only stops in the breakpoints of *.py files, not in the *.html ones.

[João dos Reis]

It could be the first IDE I ever buy. Debugging a project in a VM sounds like magic worth paying for.

[João dos Reis]

PyCharm just uses PyDev I think, not its own one.

[Denilson Sá Maia]

If you plan using pdb, check out pudb. It uses ncurses to provide a very nice UI while debugging the code. Just use import pudb; pu.db.

[coredumperror]

PyDev's remote debugger works quite wonderfully with Django's dev server. Just make sure you have the "When file is changed, automatically reload module?" option ''disabled'' in PyDev's Run/Debug settings. Otherwise the dev server and pydev will both try to reload the code while you're debugging, which gets them both extremely confused.

[Joseph Sheedy]

I installed this recently, but only today figured out to configure "POST_MORTEM=True" in my dev settings as documented by Tom's django-pdb. Now I can just cruise along and when things go bad I'm dropped right to the location of the problem. Thanks Tom!

[David Watson]

I'm running django 1.8 on python 2.7 and I can't get epdb.connect to talk to epdb.serve. I just get a timeout.

[Andre]

@hangtwenty your link no longer works. Here's a live cheatsheet of commands for the pdb degugger

[mknecht]

Another tool similar to rconsole: remote-pdb. Either could be activated by a signal, such as USR1.

[Ariel]

Is it possible to use tab completion in the interactive console shown in the browser? "Tab" just takes us to the next open console, I was wondering if there was a key combination, but I couldn't find one.

[Håken Lid]

@Ariel the werkzeug debugger does not have tab completion.

[Jon Kiparsky]

This is lovely. Unfortunately, it's hard to see a clean way to integrate this into a codebase which refuses any commit including an import of pdb.

[Jon Kiparsky]

pdb is fantastic, and I use it all the time, but there is always the danger of inadvertently committing a set_trace, and of that commit getting onto master, which would be a problem. If you're using pdb, it's a very good idea to add a pre-commit hook blocking commits which contain that import.

[Sławomir Lenart]

type less using just import pudb;pu.db

[elpaquete]

If you are debbugging APIs, you could try django-rundbg that adds a little twist to the Werkzeug debugger.

[Dunatotatos]

What is the difference with pdb?

[gustav]

Yup, i used today. Debugging each line of code. But it still quite hard. Since we have to create marker and save it whenever we need to evaluate python results. Perhaps there is another way to debug django soon.

[Sandeep]

I recommend pdb++ which provides a very useful sticky mode.

[PatDuJour]

check out pudb, a robust in shell debugging tool created by Professor Matloff from UC Davis. It traces variables, stacks, and other key information on top of what pdb can do.

[Antash]

It's also worth to check pdbpp which is a drop-in replacement for pdb, meaning that it overrides the later one, thus it's really nice to use with pytest --pdb and it's IMHO better than ipdb.

[Timo]

It is only up to python 3.3

[Martin Alexandersson]

stackoverflow.com/a/28256637/4366229

[mirek]

@Jon Kiparsky >> commiting set_trace : I use alias nodebug='git diff|grep alert ; git diff|grep console.log ; git diff|grep debugger ; git diff|grep alert ; git diff|grep set_trace ; git diff|grep localhost' and run this before the commit

[Mayank Jaiswal]

I have made a habit now to do this at the top of the file from IPython import embed and then whenever I want to quickly add a breakpoint in the code, I write embed(). Saves time. To avoid getting stuck in loops forever, I do embed();exit();

[Lie Ryan]

@MayankJaiswal: I had a key mapping on Vim to insert this snippet (and similar snippets for pudb and debugger; in JavaScript) into the file I'm editing. After I'm done, I just dd (delete entire line) to remove the breakpoint. This avoids the risk of committing the debugger import line into version control or having to preset the import first at the top of the file.

[Francisco]

Since Django 3 it's not necessary to install a library to run manage.py test --pdb

[Paolo]

doesn't support channels github.com/pallets/werkzeug/issues/1322

[ChiefTwoPencils]

Except that it's broken! Most recent stable version won't halt on a breakpoint after initialization; total POS.

[szabozoltan]

PyCharm or (django-)pdb is really fine. The main point is to find the usable mapping and breakpoint. I wanted to debug a docker-based Django at a local curl call, like "curl localhost:8080/api...." – and it was not trivial to catch this call. But finally succeeded. Tricky point is, that it's useful to put a breakpoint NOT into the application code, but into the django core code, like wsgi.py or rest_framework/viewsets.py. For this a good mapping is needed, but in PyCharm Professional it is semi-automatic. Finally you can see serializers, queries, anything. porgeto.hu/Py.png

[szabozoltan]

Useful breakpoints to catch a curl call: wsgiref/simple_server.py:get_app, 64: F9 to continue rest_framework/views.py:view, 94: F9 to continue rest_framework/views.py:dispatch, 502: F7 to step in rest_framework/mixins.py:list, queryset = ... F8 to explore