Broken references in Virtualenvs
Solution 1
I found the solution to the problem here, so all credit goes to the author.
The gist is that when you create a virtualenv, many symlinks are created to the Homebrew installed Python.
Here is one example:
$ ls -la ~/.virtualenvs/my-virtual-env
...
lrwxr-xr-x 1 ryan staff 78 Jun 25 13:21 .Python -> /usr/local/Cellar/python/2.7.7/Frameworks/Python.framework/Versions/2.7/Python
...
When you upgrade Python using Homebrew and then run brew cleanup
, the symlinks in the virtualenv point to paths that no longer exist (because Homebrew deleted them).
The symlinks needs to point to the newly installed Python:
lrwxr-xr-x 1 ryan staff 78 Jun 25 13:21 .Python -> /usr/local/Cellar/python/2.7.8_1/Frameworks/Python.framework/Versions/2.7/Python
The solution is to remove the symlinks in the virtualenv and then recreate them:
find ~/.virtualenvs/my-virtual-env/ -type l -delete
virtualenv ~/.virtualenvs/my-virtual-env
It's probably best to check what links will be deleted first before deleting them:
find ~/.virtualenvs/my-virtual-env/ -type l
In my opinion, it's even better to only delete broken symlinks. You can do this using GNU find
:
gfind ~/.virtualenvs/my-virtual-env/ -type l -xtype l -delete
You can install GNU find
with Homebrew if you don't already have it:
brew install findutils
Notice that by default, GNU programs installed with Homebrew tend to be prefixed with the letter g
. This is to avoid shadowing the find
binary that ships with OS X.
Solution 2
After trying a few things, this worked for me:
go to your virtualenv directory (but don't run workon):
cd ~/.virtualenv/name_of_broken_venv
Now delete these files:
rm -rf .Python bin/python* lib/python2.7/* include/python2.7
Then to rebuild your venv, run:
virtualenv .
workon name_of_broken_venv
pip freeze
You should now see a list of your installed packages again.
Solution 3
This occurred when I updated to Mac OS X Mavericks from Snow Leopard. I had to re-install brew beforehand too. Hopefully you ran the freeze command for your project with pip.
To resolve, you have to update the paths that the virtual environment points to.
- Install a version of python with brew:
brew install python
- Re-install virtualenvwrapper.
pip install --upgrade virtualenvwrapper
- Removed the old virtual environment:
rmvirtualenv old_project
- Create a new virtual environment:
mkvirtualenv new_project
- Work on new virtual environment
workon new_project
- Use pip to install the requirements for the new project.
pip install -r requirements.txt
This should leave the project as it was before.
Solution 4
A update version @Chris Wedgwood
's answer for keeping site-packages
(keeping packages installed)
cd ~/.virtualenv/name_of_broken_venv
mv lib/python2.7/site-packages ./
rm -rf .Python bin lib include
virtualenv .
rm -rf lib/python2.7/site-packages
mv ./site-packages lib/python2.7/
Solution 5
It appears the proper way to resolve this issue is to run
pip install --upgrade virtualenv
after you have upgraded python with Homebrew.
This should be a general procedure for any formula that installs something like python, which has it's own package management system. When you install brew install python
, you install python
and pip
and easy_install
and virtualenv
and so on. So, if those tools can be self-updated, it's best to try to do so before looking to Homebrew as the source of problems.
Related videos on Youtube
oxtay
Okhtay is an EE engineer with a PhD degree, with experience in the areas of software-defined radios and cognitive radios. He has a new found passion in web app development and data sciences and is developing his skills in those areas. #SOreadytohelp
Updated on July 12, 2022Comments
-
oxtay almost 2 years
I recently installed a bunch of dotfiles on my Mac along with some other applications (I changed to iTerm instead of Terminal, and Sublime as my default text editor) but ever since, all my virtual environments have stopped working, although their folders inside .virtualenvs are still there and they give the following error whenever I try to run anything in them:
dyld: Library not loaded: @executable_path/../.Python Referenced from: /Users/[user]/.virtualenvs/modclass/bin/python Reason: image not found Trace/BPT trap: 5
I have removed all the files related to dotfiles and have restored my .bash_profile to what it was before, but the problem persists. Is there any way to diagnose the problem or solve it in an easy way (e.g. not requiring to create all the virtualenvs all over again)?
-
unutbu about 10 yearsPossibly related: debugfix.com/2011/11/dyld-library-loaded-executable_path-python
-
oxtay about 10 yearsThank you for the comment, @unubtu. This certainly is helpful. But I am also not able to make any new virtualenvs. My
rmvirtualenv
still works but when trying to runmkvirtualenv
, I get the following error:-bash: /usr/local/bin/virtualenv: /usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/Resour: bad interpreter: No such file or directory
So, it seems a problem with my python paths but I can't see where the problem is, since I can run python and it seems fine. -
oxtay about 10 years[update] I may have found the problem but I am not sure and I am actually not sure how to fix it. It seems that all
virtualenv
commands are working now in theory, but since there is a problem with python, they don't do anything. So the real problem is with brew's python. And I have a suspicion that the reason is because of a name change in python directories. For some reason, all these commands are looking for python in folder/usr/local/Cellar/python/2.7.6
but the folder's name is actually/usr/local/Cellar/python/2.7.6_1
. -
oxtay about 10 yearsSince I am a novice, I don't know how risky it is to manually change the name from 2.7.6_1 to 2.7.6 and see what happens.
-
unutbu about 10 yearsYou should be able to rename
2.7.6_1
to2.7.6
. If worse came to worst, you could rename it back. -
oxtay about 10 yearsYup. I actually went ahead and tried it. That actually made me notice another problem. The problem wasn't the name, but rather it seemed it was broken links, somehow caused in brew. I tried a bunch of things and the problem seems to be gone now. What seems to have worked was uninstalling Python in homebrew all together and installing it again and then running
brew unlink python && brew link python
followed bybrew linkapps
. Although I think the second part wasn't even necessary since I had tried it before to no avail. Thanks again for the help! -
Patrick Williams about 8 yearsBroken links didn't work for me, unfortunately. I was able to work around the 'bad interpreter' problem by @oxtay's solution, by making a new directory (
mkdir
) called/usr/local/Cellar/python/2.7.6
and copying (cp -r
) all of the files from/usr/local/Cellar/python/2.7.9
into that folder. I was too nervous to just rename the folder!
-
-
Robert Brisita almost 10 yearsThis worked for an issue with setuptools, specifically: Warning: cannot find svn location for setuptools==0.6c12dev-r88846
-
oxtay over 9 yearsThis was a while ago and I believe I eventually did something along these lines, but since I hadn't run 'pip freeze > requirements.txt' back then, it wasn't the most efficient solution. Lesson learned.
-
2Toad about 8 years+1
gfind
was perfect, since I had a lot of unbroken symlinks (e.g., nodeenv) that I didn't want to delete -
Ryan about 8 yearsFWIW, I just tried this approach after upgrading to El Capitan and re-installing homebrew, and my package list was not preserved.
-
RafazZ over 7 yearsI guess it is because this solution is not obsolete -- I have just tried it and it fixed my issue. Also, I think if you don't have symlinks, you won't see the error described here, so this comment becomes not a solution but a distraction -- Just because you have a newer version, doesn't mean everybody does. That's my guess why the downvote :)
-
sds over 7 years@RafazZ: I hope it is now better. However, I wonder why it is still a symlink for you. And yes, I do get that error because the virtualenv python is linked against the stock python libs.
-
RafazZ over 7 yearsI think the default behaviour is still to create symlinks and you need an
--always-copy
argument to override it. At least that what I got from the User Guide -
sds over 7 years@RafazZ: I never used
--always-copy
and I have regular files :-( -
vdboor over 7 yearsAnother way to remove broken symlinks is using the standard find:
find -L ~/.virtualenvs/my-virtual-env/ -type l | xargs rm
-
christang almost 7 yearsI applied this solution, followed by running:
virtualenv .
in my broken virtual environment. The updated version ofvirtualenv
then recreated the necessary dependencies and I was good to go. This process was more self-managed and robust than the accepted answer for me. -
Harry Moreno almost 6 yearswith pipenv you can remove by doing
pipenv --rm
and recreate,pipenv shell
,pipenv install
-
Aseem almost 6 yearsI deleted my entire virtualenv dir. now i cannot remove symlinks. Non of the solutions mentioned on this page work for me on mac. i still get same error "image not found . Abort trap: 6"
-
deed02392 over 5 yearsThese steps didn't quite work for me:
pip3 freeze dyld: lazy symbol binding failed: Symbol not found: __Py_UnixMain
-
Bohumir Zamecnik about 5 yearsJust to add, if the env was with Python 2, run it with argument:
virtualenv ~/.virtualenvs/foo -p python2
, otherwise it will use Python 3. -
Francisco Peters almost 5 yearsIf you're complementing another user's answers you should leave a comment for them so they can edit! Nice contribution.
-
Tyler Smith almost 5 yearsHe doesn't have enough reputation points to comment on an answer.
-
Harish Prasanna almost 5 yearsThis is beyond perfection. Helps migrate python version while retaining all the packages. If you are following this, do not execute @Chris Wedgewood's instructions.
-
user2233949 over 4 yearsThank you very much. Deleting the symlinks and recreating the virtualenv did the trick. Well done
-
dhanshreeA over 4 yearsThank you so much for this answer! Another very hacky solution that I tried was to unlink brew from the upgraded version of Python, and reinstalling the previous version.
-
Handfeger over 4 yearsyou also can use
pipenv --rm
in the folder of your env and thenpipenv install --dev
-
lukik over 4 yearsThis also worked to fix broken links after installing Python 3.7 on a system that had Python3.6
-
David Buck over 4 yearsWelcome to SO. Though we thank you for your answer, it would be better if it provided additional value on top of the other answers. In this case, your answer does not provide additional value, since another user already posted that solution. If a previous answer was helpful to you, you should vote it up once you have enough reputation
-
scubabuddha about 4 yearsIn 2020, this is still the answer.
-
cybertoast over 3 yearsIf you get errors related to
virtualenvwrapper.sh: There was a problem running the initialization hooks
you will need topip install virtualenv virtualenvwrapper
orpip3 install virtualenv virtualenvwrapper
just to get everything back in sync. -
Ran Halprin over 3 yearsThis didn't work for me, but Robert Brisita's suggestion of reinstalling virtualenvwrapper did