PyDev for Eclipse - Resolve Python dependencies (unresolved imports)

Question

I am using PyDev for Eclipse as my IDE and pip as my package management tool, running virtualenv.

Every time I want to use/include some new libraries or new dependencies in my project, I add them into the pip-requires file. The dependencies are installed in my virtual environment with no problem after running pip install -r pip-requires.

My Env:

• Ubuntu 12.04
• PyDev for Eclipse 2.7.1.2012100913
• Python 2.7.3
• Eclipse indigo

Update: My Pydev Setup:

Questions:

• eclipse/pydev knows nothing about the new libraries being added in my virtual env. I had to manually add the .egg source folder into eclipse project's PYTHONPATH one by one in order to resolve all the unresolved imports which was very annoying! Can the python dependencies be resolved in a easier manner?

Update:

• I found that if I use virtual env Env/bin/python as the interpreter as the screenshot indicates, some of the basic modules are not resolved in eclipse, eg. import copy, import json etc. should I use default /usr/bin/python as interpreter and just add virtual env site-package to the PYTHONPATH? so that I can resolve both standard python libraries(eg. json) and my own project dependencies?

Solution 1

Make sure your system PYTHONPATH include the site-packages folder when you choose python interpreter from your virtualenv. Just like the snapshot.

Then you don't need to add them one by one into PYTHONPATH. You will need to restart eclipse (Refresh doesn't work). New models will be added.

Update: I checked the PYTHONPATH in the virtualenv. I found there are some other paths in the sys.path. They are the system default python installed location. These path includes the basic models, like copy and json.

PS: My environment is Max OSX 10.8. Please check the output of your own platform by print sys.path

/Users/username/.virtualenvs/test/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
/Users/username/.virtualenvs/test/lib/python2.7/site-packages/pip-1.1-py2.7.egg
/Users/username/.virtualenvs/test/lib/python27.zip
/Users/username/.virtualenvs/test/lib/python2.7
/Users/username/.virtualenvs/test/lib/python2.7/plat-darwin
/Users/username/.virtualenvs/test/lib/python2.7/plat-mac
/Users/username/.virtualenvs/test/lib/python2.7/plat-mac/lib-scriptpackages
/Users/username/.virtualenvs/test/lib/python2.7/lib-tk
/Users/username/.virtualenvs/test/lib/python2.7/lib-old
/Users/username/.virtualenvs/test/lib/python2.7/lib-dynload
/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac
/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/Users/username/.virtualenvs/test/lib/python2.7/site-packages

I think this is the requirement of virtualenv. You need to add these paths into pydev system PYTHONPATH. Then you can use these basic modules.

Solution 2

From your questions I understood that you're manually adding the Python Egg packages to the path. Instead of doing this, you can just go to the Project Settings window, then open "PyDev - PYTHONPATH", navigate to the panel called "External Libraries" and add the whole site-packages folder of your virtual environment, usually /path/to/virtualenv/lib/site-packages/. This way you won't add the Eggs one by one for each packages. When you install a new package, just go to the Eclipse window and if the import is still not resolved, select the project and press the F5 key, to refresh the project.

Hopefully, that should work for you. Good luck.

Solution 3

This is how I go about setting up PyDev with a virtualenv:

1. Create a virtualenv at /home/anshumanb/.virtualenvs/myvenv
2. In Eclipse, Window > Preferences > PyDev > Interpreter - Python, I add a new interpreter. For the interpreter executable, I point it to the virtualenv's Python i.e. /home/anshumanb/.virtualenvs/myenv/bin/python
3. In the "Select folders to be added to PYTHONPATH" screen, I select all the paths starting with /home/anshumanb/.virtualenvs/myvenv AND
   /usr/lib/python2.7
/usr/lib/python2.7/plat-linux2
/usr/lib/python2.7/lib-tk and nothing else.
   (See this for why I select /usr/lib/python2.7, /usr/lib/python2.7/plat-linux2 and /usr/lib/python2.7/lib-tk)
4. When I'm back at the Interpreter - Python screen in Eclipse preferences, I hit Apply then OK

After this, whenever I install any packages into the virtualenv, I go back to the Interpreter - Python screen, hit Apply, select the interpreter that points to my virtualenv (and the one I just installed packages into) and hit Apply. PyDev then picks up any new packages installed in the virtualenv.

Note: I'm using PyDev 2.7.1 and Python 2.7.3 but on an Arch Linux box with Eclipse 4.2.1. But I don't think it'll make any difference.

Comments

[Jimmy Kane]

Yes that is what is needed. Had same issue with PIL and pip

[Shengjie]

so how it didn't work for me still, I had to add /home/myname/virt-env/lib/python2.7/site-packages/happybase-‌​0.4-py2.7.egg in External Libraries to be able to resolve it. Just /home/myname/virt-env/lib/python2.7/site-packages doesn't work for me. Am I missing anything here?

[Shengjie]

@Jordan Jambazov In the External Libraries tab, I clicked "add source folder", then added in "/home/myname/virt-env/lib/python2.7/site-packages". yes, I tried refresh, also tried to close and reopen the project, didn't work still. It seems like it only works if I add the actual egg folder in, adding site-packages folder just doesn't work.

[Jordan Jambazov]

Maybe you can try to use the interpreter wrapper inside your virtualenv, as described here: michaelpollmeier.com/eclipse-pydev-and-virtualenv

[Shengjie]

when I use virutal env as the interpreter, it seems like missing some basic python lib, eg. copy,json are not resolved anymore. But I did get way with it by using default interpreter:/usr/bin/python2.7, and add my virtual env site-packages in PYTHONPATH as you screenshot.

[jinghli]

Could you share with us about your settings via screenshot? If your settings is as mine, I believe it should work, it is werid.

[Joran Beasley]

I dont think you have to do virtualenv ... he just is in this example

[Matthew Purdon]

It worked for me after I did a project clean and rebuild rather than just a refresh.

[PJ_Finnegan]

I had "unresolved import" errors in the source file, but I could run my program with no error anyway. After adding the the /path/to/virtualenv/lib/site-packages/ folder as source, the import error were gone, but only after restarting Eclipse Oxygen (File -> Restart).

[PJ_Finnegan]

Not recommended for Eclipse Oxygen. It gives a warning not to add user-specific path to the Eclipse-wide PYTHONPATH, but to configure them on a per-project basis like in @Jordan Jambazov answer.