pip installing in global site-packages instead of virtualenv

Question

Using pip3 to install a package in a virtualenv causes the package to be installed in the global site-packages folder instead of the one in the virtualenv folder. Here's how I set up Python3 and virtualenv on OS X Mavericks (10.9.1):

I installed Python3 using Homebrew:

ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"
brew install python3 --with-brewed-openssl

Changed the $PATH variable in .bash_profile; added the following line:

export PATH=/usr/local/bin:$PATH

Running which python3 returns /usr/local/bin/python3 (after restarting the shell).

Note: which python3 still returns /usr/bin/python though.

Installed virtualenv using pip3:

pip3 install virtualenv

Next, create a new virtualenv and activate it:

virtualenv testpy3 -p python3
cd testpy3
source bin/activate

Note: if I don't specify -p python3, pip will be missing from the bin folder in the virtualenv.

Running which pip and which pip3 both return the virtualenv folder:

/Users/kristof/VirtualEnvs/testpy3/bin/pip3

Now, when I try to install e.g. Markdown using pip in the activated virtualenv, pip will install in the global site-packages folder instead of the site-packages folder of the virtualenv.

pip install markdown

Running pip list returns:

Markdown (2.3.1)
pip (1.4.1)
setuptools (2.0.1)
virtualenv (1.11)

Contents of /Users/kristof/VirtualEnvs/testpy3/lib/python3.3/site-packages:

__pycache__/
_markerlib/
easy_install.py
pip/
pip-1.5.dist-info/
pkg_resources.py
setuptools/
setuptools-2.0.2.dist-info/

Contents of /usr/local/lib/python3.3/site-packages:

Markdown-2.3.1-py3.3.egg-info/
__pycache__/
easy-install.pth
markdown/
pip-1.4.1-py3.3.egg/
setuptools-2.0.1-py3.3.egg
setuptools.pth
virtualenv-1.11-py3.3.egg-info/
virtualenv.py
virtualenv_support/

As you can see, the global site-packages folder contains Markdown, the virtualenv folder doesn't.

Note: I had Python2 and Python3 installed before on a different VM (followed these instructions) and had the same issue with Python3; installing packages in a Python2 based virtualenv worked flawlessly though.

Any tips, hints, … would be very much appreciated.

Solution 1

Funny you brought this up, I just had the exact same problem. I solved it eventually, but I'm still unsure as to what caused it.

Try checking your bin/pip and bin/activate scripts. In bin/pip, look at the shebang. Is it correct? If not, correct it. Then on line ~42 in your bin/activate, check to see if your virtualenv path is right. It'll look something like this

VIRTUAL_ENV="/Users/me/path/to/virtual/environment"

If it's wrong, correct it, deactivate, then . bin/activate, and if our mutual problem had the same cause, it should work. If it still doesn't, you're on the right track, anyway. I went through the same problem solving routine as you did, which piping over and over, following the stack trace, etc.

Make absolutely sure that

/Users/kristof/VirtualEnvs/testpy3/bin/pip3

is what you want, and not referring to another similarly-named test project (I had that problem, and have no idea how it started. My suspicion is running multiple virtualenvs at the same time).

If none of this works, a temporary solution may be to, as Joe Holloway said,

Just run the virtualenv's pip with its full path (i.e. don't rely on searching the executable path) and you don't even need to activate the environment. It will do the right thing.

Perhaps not ideal, but it ought to work in a pinch.

Link to my original question:

VirtualEnv/Pip trying to install packages globally

Solution 2

For me this was not a pip or virtualenv problem. It was a python problem. I had set my $PYTHONPATH manually in ~/.bash_profile (or ~/.bashrc) after following some tutorial online. This manually set $PYTHONPATH was available in the virtualenv as it probably should be allowed.

Additionally add2virtualenv was not adding my project path to my $PYTHONPATH for some reason within the virtualenv.

Just some forking paths for those who might still be stuck! Cheers!

Solution 3

I had the same problem, I solved it by removing venv directory and recreating it!

deactivate (if venv is activated first deactivate it)
rm -rf venv
virtualenv -p python3 venv
. ENV/bin/activate
pip3 install -r requirements.txt

Now everything works like a charm.

Solution 4

The first thing to check is which location pip is resolving to:

which pip

if you are in a virtualenv you would expect this to give you something like:

/path/to/virtualenv/.name_of_virtualenv/bin/pip

However it may be the case that it's resolving to your system pip for some reason. For example you may see this from within your virtualenv (this is bad):

/usr/local/bin/pip (or anything that isn't in your virtualenv path).

To solve this check your pipconfig in:

~/.pipconf
~/.conf/pip
/etc/pip.conf

and make sure that there is nothing that is coercing your Python path or your pip path (this fixed it for me).

Then try starting a new terminal and rebuild your virtualenv (delete then create it again)

Solution 5

I had the same issue on macos with python 2 and 3 installed.

Also, I had aliases to point to python3 and pip3 in my .bash_profile.

alias python=/usr/local/bin/python3
alias pip=/usr/local/bin/pip3

Removing aliases and recreating virtual env using python3 -m venv venv fixed the issue.

View more solutions

Comments

[DocZerø]

Thanks Chase. I came by your question before posting mine, but it seems I skipped the very last line which mentioned the shebang. And indeed, it was set to #!/usr/local/bin/python3.3 instead of #!/Users/kristof/VirtualEnvs/testpy3/bin/python3.3. I changed it, activated the virtualenv and installed the Markdown package. Pip now installs in the virtualenv site-packages folder instead of the global one.

[Will]

I ran into this too, thanks so much for the answer. I noticed the shebangs and then immediately afterward found this question, confirming my suspicions. Does anyone know why the shebang was wrong? It would be nice to find a permanent fix so we don't have to check it every time we make a new virtual environment.

[Neil Traft]

I had the same problem. My activate script was fine, but beware, all the pip* scripts and easy_install* scripts have the wrong shebang. They all have to be fixed manually. I was unable to fix them via reinstalling pip or anything like that. Also, a clarification to Joe Holloway's workaround: the problem is not with the shell searching for pip, it's the fact that pip explicitly specifies the wrong python. Therefore you would need to specify the python yourself, like so: $ ~/.virtualenvs/venv/bin/python ~/.virtualenvs/venv/bin/pip --version

[Dave]

Or if you use sudo, you also have to activate the virtual environment. sudo su followed by <venv>/bin/activate followed by pip install.

[shellbye]

I met this problem after a --relocatable my env, and the line 42 is wrong, Looks like the --relocatable did not do it job right.

[foslock]

This my issue as well. My file looked like this, no idea how it got there: [install]\nprefix=

[Aman]

@foslock yep, that's what mine looked like too. bad news haha!

[joarleymoraes]

This happened to me when I renamed an intermediate directory, so I had to edit activate and pip scripts in '/bin'

[ptevans]

For me (Mac OS Sierra, python 3.5 installed by brew) I fixed this using the first line of your advice: edit the pip in the virtualenv. The shebang was hardcoded, instead it should be: #!/usr/bin/env python

[t.animal]

Also check /etc/pip.conf! I had a similar issue and after a lot of debugging figured someone misconfigured the system I was working on by messing around with this file.

[citynorman]

I got this because I moved the directory to another path and it needed virtualenv to be run again

[Mr. Unnormalized Posterior]

I had moved my virtual environment to different path. This caused the all packages to be installed globally. this helped me thanks!

[subtleseeker]

I was using pip3 while virtualenv, by default, used python2 thus using pip instead of pip3. I checked the bin to find no pip3. Using virtualenv -p python3 venv solved the problem.

[sid]

I also had this problem on Manjaro, and resolved it the same way. After resolving I reinstalled python-pip via pamac and the virtualenv pip continued to function correctly. Not sure exactly what is going on, but I agree with your assessment of a double install issue.

[Q. Qiao]

I am using arch Linux. I think the /etc/pip.conf is set up by the OS.

[MewX]

Thanks! You saved my day! Those configs were like ghosts hidden in the file system

[Alvaro Rodriguez Scelza]

This solved my problem. Pycharm's automatic virtualenv creation was not working properly. Manual installation did the trick. Thanks.

[Matteo]

I had a terminal session defined alias that overriden pip, for some reasons which pip was still giving me the correct path!

[Reboot_87]

This fixed it for me. My issue was that I changed the folder name so the path in activate was no longer correct. Thank you!

[iomv]

macos python installation is needlessly painful imho

[Colin]

I was pulling my hair out, and this is what finally fixed things for me: "which pip" revealed the problem, "unalias pip" fixed it.

[Aakash Solanki]

How to solve this in windows? I am getting same path in both virtual environment and global: /c/Users/UserName/AppData/Local/Programs/Python/Python39/Scr‌​ipts/pip

[Lincoln]

Thanks! For me this was caused by a ros setup script: source /opt/ros/melodic/setup.bash

[Preshma Linet Pereira]

How did you edit a .exe file?

[Dysio]

I opened it with Notepad++

[questionto42standswithUkraine]

For me, it helped removing the venv and installing it again. virtualenv is deprecated from python3.6 on, use for example 'python3.6 -m venv'` instead.

[Matin Sasan]

@AakashSolanki just delete the venv folder and recreate it. Then reinstall the packages within the new venv.