pydev breakpoints not working

Question

I am working on a project using python 2.7.2, sqlalchemy 0.7, unittest, eclipse 3.7.2 and pydev 2.4. I am setting breakpoints in python files (unit test files), but they are completely ignored (before, at some point, they worked). By now i have upgraded all related software (see above), started new projects, played around with settings, hypnotized my screen, but nothing works.

The only idea i got from some post is that it has something to de with changing some .py file names to lower case.

Does anyone have any ideas?

added: I even installed the aptana version of eclipse and copied the .py files to it => same result; breakpoints are still ignored.

still no progress: I have changed some code that might be seen as unusual and replaced it with a more straightforward solution.

some more info: it probably has something to do with module unittest:

• breakpoints in my files defining test suites work,
• breakpoints in the standard unittest files themselves work
• breakpoints in my tests methods in classes derived from unittest.TestCase do not work
• breakpoints in my code being tested in the test cases do not work
• at some point before i could define working breakpoints in test methods or the code being tested
• some things i changed after that are: started using test suites, changed some filenames to lowercase, ...
• this problem also occurs if my code works without exceptions or test failures.

what I already tried is:

• remove .pyc files
• define new project and copy only .py files to it
• rebooted several times in between
• upgraded to eclipse 3.7.2
• installed latest pydev on eclipse 3.7.2
• switch to aptana (and back)
• removed code that 'manually' added classes to my module
• fiddled with some configurations

what I can still do is:

• start new project with my code, start removing/changing code until breakpoints work and sort of black box figure out if this has something to do with some part of my code

---

• Does anyone have any idea what might cause these problems or how they might be solved?
• Is there any other place i could look for a solution?
• Do pydev developers look into the questions on stackoverflow?
• Is there an older version of pydev that i might try?

I have been working with pydev/eclipse for a long time and it works well for me, but without debugging i'd forced to switch IDE.

In answer to Fabio's questions below:

• The python version is 2.7.2,
• The sys.gettrace gives None (but I have no idea what in my code could influence that)
• This is the output of the debugger after changing the suggested parameters:

pydev debugger:

starting
('Executing file ', 'D:\\.eclipse\\org.eclipse.platform_3.7.0_248562372\\plugins\\org.python.pydev.debug_2.4.0.2012020116\\pysrc\\runfiles.py')
('arguments:', "['D:\\\\.eclipse\\\\org.eclipse.platform_3.7.0_248562372\\\\plugins\\\\org.python.pydev.debug_2.4.0.2012020116\\\\pysrc\\\\runfiles.py', 'D:\\\\Documents\\\\Code\\\\Eclipse\\\\workspace\\\\sqladata\\\\src\\\\unit_test.py', '--port', '49856', '--verbosity', '0']")
('Connecting to ', '127.0.0.1', ':', '49857')
('Connected.',)
('received command ', '501\t1\t1.1')
sending cmd: CMD_VERSION 501    1   1.1

sending cmd: CMD_THREAD_CREATE 103  2   <xml><thread name="pydevd.reader" id="-1"/></xml>

sending cmd: CMD_THREAD_CREATE 103  4   <xml><thread name="pydevd.writer" id="-1"/></xml>

('received command ', '111\t3\tD:\\Documents\\Code\\Eclipse\\workspace\\sqladata\\src\\testData.py\t85\t**FUNC**testAdjacency\tNone')
Added breakpoint:d:\documents\code\eclipse\workspace\sqladata\src\testdata.py - line:85 - func_name:testAdjacency
('received command ', '122\t5\t;;')
Exceptions to hook : []
('received command ', '124\t7\t')
('received command ', '101\t9\t')
Finding files... done.
Importing test modules ... testAtomic (testTypes.TypeTest) ... ok
testCyclic (testTypes.TypeTest) ... 

The rest is output of the unit test.

Continuing from Fabio's answer part 2:

I have added the code at the start of the program and the debugger stops working at the last line of following the method in sqlalchemy\orm\attributes.py (it is a descriptor, but how or whther it interferes with the debugging is beyond my current knowledge):

class InstrumentedAttribute(QueryableAttribute): """Class bound instrumented attribute which adds descriptor methods."""

def __set__(self, instance, value):
    self.impl.set(instance_state(instance), 
                    instance_dict(instance), value, None)

def __delete__(self, instance):
    self.impl.delete(instance_state(instance), instance_dict(instance))

def __get__(self, instance, owner):
    if instance is None:
        return self

    dict_ = instance_dict(instance)
    if self._supports_population and self.key in dict_:
        return dict_[self.key]
    else:
        return self.impl.get(instance_state(instance),dict_) #<= last line of debugging

From there the debugger steps into the __getattr__ method of one of my own classes, derived from a declarative_base() class of sqlalchemy.

Probably solved (though not understood):

The problem seemed to be that the __getattr__ mentioned above, created something similar to infinite recursion, however the program/unittest/sqlalchemy recovered without reporting any error. I do not understand the sqlalchemy code sufficiently to understand why the __getattr__ method was called.
I changed the __getattr__ method to call super for the attribute name for which the recursion occurred (most likely not my final solution) and the breakpoint problem seems gone. If i can formulate the problem in a consise manner, i will probably try to get some more info on the google sqlalchemy newsgroup, or at least check my solution for robustness.

Thank you Fabio for your support, the trace_func() function pinpointed the problem for me.

Solution 1

Seems really strange... I need some more info to better diagnose the issue:

Open \plugins\org.python.pydev.debug\pysrc\pydevd_constants.py and change

DEBUG_TRACE_LEVEL = 3 
DEBUG_TRACE_BREAKPOINTS = 3

run your use-case with the problem and add the output to your question...

Also, it could be that for some reason the debugging facility is reset in some library you use or in your code, so, do the following: in the same place that you'd put the breakpoint do:

import sys
print 'current trace function', sys.gettrace()

(note: when running in the debugger, it'd be expected that the trace function is something as: <bound method PyDB.trace_dispatch of <__main__.PyDB instance at 0x01D44878>> )

Also, please post which Python version you're using.

---

Answer part 2:

The fact that sys.gettrace() returns None is probably the real issue... I know some external libraries which mess with it (i.e.:DecoratorTools -- read: http://pydev.blogspot.com/2007/06/why-cant-pydev-debugger-work-with.html) and have even seen Python bugs and compiled extensions break it...

Still, the most common reason it breaks is probably because Python will silently disable the tracing (and thus the debugger) when a recursion throws a stack overflow error (i.e.: RuntimeError: maximum recursion depth exceeded).

You can probably put a breakpoint in the very beginning of your program and step in the debugger until it stops working.

Or maybe simpler is the following: Add the code below to the very beginning of your program and see how far it goes with the printing... The last thing printed is the code just before it broke (so, you could put a breakpoint at the last line printed knowing it should be the last line where it'd work) -- note that if it's a large program, printing may take a long time -- it may even be faster printing to a file instead of a console (such as cmd, bash or eclipse) and later opening that file (just redirect the print from the example to a file).

import sys

def trace_func(frame, event, arg):
    print 'Context: ', frame.f_code.co_name, '\tFile:', frame.f_code.co_filename, '\tLine:', frame.f_lineno, '\tEvent:', event
    return trace_func

sys.settrace(trace_func)

If you still can't figure it out, please post more information on the obtained results...

Note: a workaround until you don't find the actual place is using:

import pydevd;pydevd.settrace()

on the place where you'd put the breakpoint -- that way you'd have a breakpoint in code which should definitely work, as it'll force setting the tracing facility at that point (it's very similar to the remote debugging: http://pydev.org/manual_adv_remote_debugger.html except that as the debugger was already previously connected, you don't really have to start the remote debugger, just do the settrace to emulate a breakpoint)

Solution 2

Coming late into the conversation, but just in case it helps. I just run into a similar problem and I found that the debugger is very particular w.r.t. what lines it considers "executable" and available to break on.

If you are using line continuations, or multi-line expressions (e.g. inside a list), put the breakpoint in the last line of the statement.

I hope it helps.

Solution 3

Try removing the corresponding .pyc file (compiled) and then running. Also I have sometimes realized I was running more than one instance of a program.. which confused pydev. I've definitely seen this before too. Quite a few times.

Solution 4

I had similar-sounding symptoms. It turned out that my module import sequence was rexec'ing my entry-point python module because a binary (non-Python) library had to be dynamically loaded, i.e., the LD_LIBRARY_PATH was dynamically reset. I don't know why this causes the debugger to ignore subsequent breakpoints. Perhaps the rexec call is not specifying debug=true; it should specify debug=true/false based on the calling context state?

Try setting a breakpoint at your first import statement being cognizant of whether you are then s(tep)'ing into or n(ext)'ing over the imports. When I would "next" over the 3rdparty import that required the dynamic lib loading, the debug interpreter would just continue past all breakpoints.

Solution 5

Ran into a similar situation running a django app in Eclipse/pydev. what was happening was that the code that was running was the one installed in my virtualenv, not my source code. I removed my project from my virtual env site-packages, restarted the django up in the eclipse/pydev debugger and everything was fine.

View more solutions

Comments

[kyooryu]

Do you use threads or any other parallel comuting technique?

[Lars]

Nope, no threading, multiprocessing, etc. I am not sure about external modules (e.g. sqlalchemy) sqlalchemy however.

[Fabio Zadrozny]

Just to note, this relation from infinite recursion and the python tracing is 'known'. see: nedbatchelder.com/blog/201010/surprising_getattr_recursion.h‌​tml, although I'm not sure it was actually reported as a bug to python -- may be nice doing it :)

[Lars]

Hmm, interesting. This seems similar to what is happening here, though i think the recursion here has something to do with the use of a descriptor and maybe that getattr overrides getattr in a base class, but i have a hard time following the step through of the sqlalchemy code.

[Lars]

I just tried; it doesn't work and i already tried too copy all .py and no other files to a new project.

[Lars]

btw: I also have the problem when multiple instance are running, but this is some thing different: even after reboot, all breakpoints are ignored

[Lars]

was away for two days, adding the answers now

[Fabio Zadrozny]

Just improved the answer (part 2)

[Andrew Hundt]

I ran into a problem like this, and I had put my breakpoint on the same line as a comment. Make sure your breakpoints are set on lines that contain actual code or on the last line of a statement as Miguel said.

[Big Al]

Yes. Moving the breakpoint further down on the multi-line expression worked for me.

[cahen]

import pydevd;pydevd.settrace() was a good workaround, thanks