How do I start up remote debugging with PyCharm?

Question

I'm trying to get debugging up between PyCharm (on windows host) and a debian virtual host running my django application. The instructions say to install the egg, add the import, and then invoke a command. I assume these things need to be done on the debian host?

Ok, then, in what file should I put these two lines?

from pydev import pydevd
pydevd.settrace('not.local', port=21000, stdoutToServer=True, stderrToServer=True)

I tried putting it into the settings.py but got this kind of thing...

File "/django/conf/__init__.py", line 87, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
File "/django/utils/importlib.py", line 35, in import_module
    __import__(name)
File "/settings.py", line 10, in <module>
    pydevd.settrace('dan.local', port=21000, stdoutToServer=True, stderrToServer=True)
File "/pycharm-debug.egg/pydev/pydevd.py", line 1079, in settrace
    debugger.connect(host, port)
File "/pycharm-debug.egg/pydev/pydevd.py", line 241, in connect
    s = StartClient(host, port)
File "/pycharm-debug.egg/pydev/pydevd_comm.py", line 362, in StartClient
    sys.exit(1)
SystemExit: 1

Whilst pycharm just sat there "waiting for connection"

Solution 1

PyCharm (or your ide of choice) acts as the "server" and your application is the "client"; so you start the server first - tell the IDE to 'debug' - then run the client - which is some code with the settrace statement in it. When your python code hits the settrace it connects to the server - pycharm - and starts feeding it the debug data.

To make this happen:

1. copy the pydev library to the remote machine

So I had to copy the file from C:\Program Files\JetBrains\PyCharm 1.5.3\pycharm-debug.egg to my linux machine. I put it at /home/john/api-dependancies/pycharm-debug.egg

2. Put the egg in the PYTHONPATH

Hopefully you appreciate that you're not going to be able to use the egg unless python can find it. I guess most people use easy_install but in my instance I added it explicitly by putting this:

   import sys
   sys.path.append('/home/john/app-dependancies/pycharm-debug.egg')

This is only necessary because I've still had no success installing an egg. This is my workaround.

3. setup the debug server config

In PyCharm you can configure the debug server via:

• Run-> Edit Configurations: opens the 'Run/Debug Configurations' dialog
• Defaults -> "Python Remote Debug": is the template to use
• fill out the local host name and port and you'll probably want to 'use path mapping' but more on all this below...

• "OK"

  Local host name: means the name of the server - that's the windows host machine in my case - or actually the IP Address of the windows host machine since the hostname is not known to my remote machine. So the virtual (remote) machine has to be able to reach the host. ping and netstat are good for this.

  Port: can be any vacant non-priviledged port you like. eg: 21000 is unlikely to be in use.

  Don't worry about the path mappings for now.

4. Start the debug server

• Run-> Debug : start the debug server - choose the configuration you just created.

The debug console tab will appear and you should get

 Starting debug server at port 21000

in the console which means that the ide debug server is waiting for your code to open a connection to it.

5. Insert the code

This works inside a unit test:

from django.test import TestCase
class APITestCase(TestCase):
    def test_remote_debug(self):
        import sys
        sys.path.append('/home/john/dependancies/pycharm-debug.egg')
        from pydev import pydevd
        pydevd.settrace('192.168.33.1', port=21000, suspend=False)

        print "foo"

And in a django web application it's a bit finicky about where you put it - seems to work only after everything else is done:

if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
    from django.core.management import execute_from_command_line
    execute_from_command_line(sys.argv)

    sys.path.append('/vagrant/pycharm-debug.egg')
    import pydevd
    pydevd.settrace('192.168.33.1', port=21000, suspend=False)

Again that the IP address is the box where you're running Pycharm on; you should be able to ping that ip address from the box running your code/website. The port is your choice, just make sure you've told pycharm to listen on the same port. And I found the suspend=False less problematic than the defaults of, not only immediately halting so you're not sure if it's working, but also trying to stream to stdin/out which might give you grief also.

6. Open the firewall

Windows 7 firewall will, by default, block your incoming connection. Using netstat on the remote host you'll be able to see that SYN_SENT never becomes ESTABLISHED, at least not until you add an exception to the windows firewall for the application 'pycharm'.

OS/X and Ubuntu do not have firewalls to punch threw (by default, someone may have applied one later).

7. Set a breakpoint and run the code

After all that, when everything goes to plan, you can set a breakpoint - somewhere after the settrace has run - and pycharm console will show

Connected to pydev debugger (build 107.386)

and under the 'Debugger' tab the variables stack will start working and you can step through the code.

8. Mappings

Mapping tell pycharm where it can find the source code. So when the debugger says "i'm running line 393 of file /foo/bar/nang.py, Pycharm can translate that remote absolute path into an absolute local path... and show you the source code.

/Users/john/code/app/    /opt/bestprice/app/
/Users/john/code/master/lib    /opt/bestprice/lib/python2.7/site-packages

Done.

Solution 2

It is just a note , actually , but contains some info that may save hours.

1. Right now pip install pydevd worked for me on both ubuntu and centos 6

2. If you want to really debug remote server which is behind firewals and stuff, you can use the following trick:

   ssh -R  8081:localhost:8081 [email protected]
   

   This allows remote code to connect to your machine listening on localhost:8081

3. If remote debugger does not want to start, saying it can't find socket port, check your firewall rules. Note that rule with 127.0.0.1 is not the same as localhost.

Solution 3

It seems that for some reason debugger couldn't connect to your windows host with PyCharm. Haven't you got any other messages in stderr? If you have not, try to run it one more time, but with sterrToServer=false. That may show real reason why it doesn't connect.

Comments

[John Mee]

Sorry, didn't make any difference. No other messages that I can see. Can you tell me if pycharm is the client and my app is the server, or vice versa?

[John Mee]

That helped. One step closer: I was naming the wrong host in the settrace. I can see it establish the connection now (netstat). Unfortunately still no party yet... Works without the settrace lines, save them into a views.py page, restart the webserver, start 'debugging' in pycharm, now attempt to reload the page but the response never completes, it just sits there spinning whilst pycharm says nothing either - no 'play' or 'step' etc light up. Looks like the execution is suspended but pycharm doesn't know it?

[John Mee]

oh. wait. I have it working from inside a unit test now. Yay! Just the webserver to go.

[Dan Breen]

This was a fantastic write-up, thank you! I don't know if something was fixed in a newer version of PyCharm since this post, but I was able to put the settrace lines right in a Django view, and once that line was hit, the browser paused and I was successfully debugging.

[tcmb]

Thank you very much for this step-by-step explanation. Might I add: - Debugging web apps is probably done best by adding the bp in the view to be debugged, or in the get_response() of the handler class you're using, if you want to step in real early. - The path mappings can be added on the fly once a breakpoint is encountered. PyCharm will ask to provide a mapping if it does not no one yet and even make an educated auto-suggestion.

[RichVel]

I found the best place to put the pydev call was in manage.py, so that you can debug any part of your Django project using breakpoints.

[RichVel]

In PyCharm 2.7, the code to include in the remote Python script has changed slightly to use: import pydevd

[stevietheTV]

Hey! Great write up. Do you know what the error might be if the console outputs "Connected to debugger (buld null"?

[John Mee]

@stevietheTV does it explicitly state there's an error? build null suggests to me that it connected fine, there was simply nothing entered into the field that provides a version/build number.

[stevietheTV]

No error. What happens is that it states "Waiting for connection...", then in a mere second it says "connected build(null)", doesn't stop at any breakpoints, and then it says "Waiting for connection" again

[Altryne]

@stevietheTV this might be connected to the reloading mechanism. Try running your with manage.py --noreload

[user1380140]

I tried to follow these instructions to be able use PyCharm(localhost) to remotely debug django app running on a virtualbox vm. After starting debug in PyCharm ("waiting for connection...") then when I start the django app on the remote vm I never hit any breakpoints. I have the pydev.settrace() in manage.py. PyCharm debug window still shows "Waiting to connect....". Any idea what I might be doing wrong?

[Fran K.]

This was a great help, but I had a firewall blocking port 21000 to contend with. The solution I found was to open a reverse tunneling ssh session from the box running PyCharm to the remote server. I did this by inserting "-R 21000:localhost:21000" into a ssh connection to the remote machine (you can add -v to debug ssh). Then I open another ssh to the remote machine to start the script I want to debug, which then connects back to my PyCharm and allows debugging the code. The script output appears in this second ssh session.

[user1380140]

Does anyone know how to solve the "Waiting to connect..." problem with Django.? I have a similar configuration/setup of PyCharm (ver:3.4) running on ubuntu host and the Django application running remotely in a docker container.

[kevinarpe]

Cripes -- this is one of the 50 best Q&A ever seen on StackOverflow. Hugely helpful. Please also mention that PyCharm only offers remote debugging in the professional edition.

[Istiaque Ahmed]

@JohnMee, may I ask you to have a look at a remote-debugging related question here : tinyurl.com/ovglq92 ?

[John Mee]

@IstiaqueAhmed You may. I did. Sorry I can't help you. Remotely debugging the Opera browser is not something I have any experience with.

[Hussain]

Hey, I was using version 4.0.5 then I upgraded it to 4.5.4 but still I don't see the "Python Remote Debug" option.

[John Mee]

@Hussain Run->Debug

[Wassim Seifeddine]

Hey, but what if my local machine that's running Pycharm does not have a public IP address? any solutions ?

[John Mee]

@WassimSeifeddine You have to be able to route a TCP/IP packet from the remote machine to the local for this to work. If the 'remote' machine is 'external', for example a firewall which has the 'public' address sits between remote and local, then you have to configure that firewall to forward these packets to the 'private' address of your local machine.

[him229]

woah, this was super helpful :)