Pip problems - Could not install packages due to an EnvironmentError

Question

I think I have some issues with either Python and/or pip on my Mac. I have Python 2.7 installed globally and then I normally setup virtualenvs and install Python3.6.4 but in the last day or so Ive been getting problems with packages such as Fabric and SSH2 where I have either not been able to install them with various errors or with Fabric it throws when I try to import the package.

Im now trying to remove Fabric and install Fabric3 and its throwing errors like this:

Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/Users/david/Documents/projects/uptimeapp/env/lib/python3.6/site-packages/Fabric3-1.14.post1.dist-info'
Consider using the `--user` option or check the permissions.

(env) Davids-MacBook-Air:uptimeapp david$ pip install fabric3 --user
Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

If I do sudo pip install fabric then it installs but with this warning:

The directory '/Users/david/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/david/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

But I thought it was not advised to pip install with sudo?

These are the errors I get when I try to pip install ssh2-python

ssh2/agent.c:569:10: fatal error: 'libssh2.h' file not found
    #include "libssh2.h"
             ^~~~~~~~~~~
    1 error generated.
    error: command 'clang' failed with exit status 1

    ----------------------------------------
Command "/Users/david/Documents/projects/uptimeapp/env/bin/python3.6 -u  -c "import setuptools,   tokenize;__file__='/private/var/folders/bl/97vt48j97zd2sj05zmt4xst00000gn/T  /pip-install-mpyq41q4/ssh2-python/setup.py';f=getattr(tokenize, 'open',   open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record   /private/var/folders/bl/97vt48j97zd2sj05zmt4xst00000gn/T/pip-record-qul_k3kq/install-record.txt --single-version-externally-managed --compile -  -install-headers /Users/david/Documents/projects/uptimeapp/env/bin/../include/site/python3.6  /ssh2-python" failed with error code 1 in /private/var/folders/bl/97vt48j97zd2sj05zmt4xst00000gn/T/pip-install-mpyq41q4/ssh2-python/

I have managed to remove Fabric and install Fabric3 with the sudo command but I would rather not do that.

I should add that Ive not had any other problems with installing other packages either globally in Python2.7 or in envs.

Solution 1

The permission denied error is raised because you've already borked your virtual environment by installing with sudo. Run

$ sudo chown -R david:staff /Users/david/Documents/projects/uptimeapp/env

to fix the permissions. Maybe it's even wise to fix the permissions for the whole home dir, should you have other permission issues:

$ sudo chown -R david:staff /Users/david/

Now reinstalling packages should work again:

$ source /Users/david/Documents/projects/uptimeapp/env/bin/activate
$ (env) pip uninstall -y fabric
$ (env) pip install fabric

'libssh2.h' file not found

means that before installing ssh-python, you need to install the according lib first:

$ brew install libssh2

Solution 2

You can make pip to install the package within the virtualenv library location:

sudo -H venv/bin/pip install fabric

Comments

[goblin_rocket]

thanks, that was it for the env permissions errors but still getting the same error after brew installing libssh2, I will probably open a separate question for that though

[hoefling]

Hmm, looking at the libssh2 formula, I see a test that explicitly checks for the required header file to be installed. So the file must be somewhere in /usr/Cellar/: run brew list libssh2 | grep libssh2.h` to check that.

[hoefling]

If the file is present, it means the header file's include path is not passed to clang in compile phase, this indicates some problems with Homebrew. Maybe brew doctor will tell something useful.

[hoefling]

f the header file is present, you may try passing the header file explicitly as a workaround: CPPFLAGS="-I/path/to/libssh2.h/parent/dir" CFLAGS="-I/path/to/libssh2.h/parent/dir" pip install ssh2-python, however, a proper solution would be fixing the default include paths in your env.