What is the purpose of "pip install --user ..."?

Question

From pip install --help:

--user  Install to the Python user install directory for your platform. 
        Typically ~/.local/, or %APPDATA%\Python on Windows. 
        (See the Python documentation for site.USER_BASE for full details.)

The documentation for site.USER_BASE is a terrifying wormhole of interesting *NIX subject matter that I don't understand.

What is the purpose of --user in plain english? Why would installing the package to ~/.local/ matter? Why not just put an executable somewhere in my $PATH?

Solution 1

pip defaults to installing Python packages to a system directory (such as /usr/local/lib/python3.4). This requires root access.

--user makes pip install packages in your home directory instead, which doesn't require any special privileges.

Solution 2

--user installs in site.USER_SITE.

For my case, it was /Users/.../Library/Python/2.7/bin. So I have added that to my PATH (in ~/.bash_profile file):

export PATH=$PATH:/Users/.../Library/Python/2.7/bin

Solution 3

Just a warning:

According to this issue, --user is currently not valid inside a virtual env's pip, since a user location doesn't really make sense for a virtual environment.

So do not use pip install --user some_pkg inside a virtual environment, otherwise, virtual environment's pip will be confused. See this answer for more details.

Solution 4

Without Virtual Environments

pip <command> --user changes the scope of the current pip command to work on the current user account's local python package install location, rather than the system-wide package install location, which is the default.

• See User Installs in the PIP User Guide.

This only really matters on a multi-user machine. Anything installed to the system location will be visible to all users, so installing to the user location will keep that package installation separate from other users (they will not see it, and would have to install it themselves separately to use it). Because there can be version conflicts, installing a package with dependencies needed by other packages can cause problems, so it's best not to push all packages a given user uses to the system install location.

• If it is a single-user machine, there is little or no difference to installing to the --user location. It will be installed to a different folder, that may or may not need to be added to the path, depending on the package and how it's used (many packages install command-line tools that must be on the path to run from a shell).
• If it is a multi-user machine, --user is preferred to using root/sudo or requiring administrator installation and affecting the Python environment of every user, except in cases of general packages that the administrator wants to make available to all users by default.
  • Note: Per comments, on most Unix/Linux installs it has been pointed out that system installs should use the general package manager, such as apt, rather than pip.

With Virtual Environments

• See more about installing packages with virtual environments in the Python Packaging documentation.
• Read about how to create and use virtual environments, and the venv command in the Python VENV docs.

The --user option in an active venv/virtualenv environment will install to the local user python location (same as without a virtual environment).

Packages are installed to the virtual environment by default, but if you use --user it will force it to install outside the virtual environments, in the users python script directory (in Windows, this currently is c:\users\<username>\appdata\roaming\python\python37\scripts for me with Python 3.7).

However, you won't be able to access a system or user install from within virtual environment (even if you used --user while in a virtual environment).

If you install a virtual environment with the --system-site-packages argument, you will have access to the system script folder for python. I believe this included the user python script folder as well, but I'm unsure. However, there may be unintended consequences for this and it is not the intended way to use virtual environments.

Location of the Python System and Local User Install Folders

You can find the location of the user install folder for python with python -m site --user-base. I'm finding conflicting information in Q&A's, the documentation and actually using this command on my PC as to what the defaults are, but they are underneath the user home directory (~ shortcut in *nix, and c:\users\<username> typically for Windows).

Other Details

The --user option is not a valid for every command. For example pip uninstall will find and uninstall packages wherever they were installed (in the user folder, virtual environment folder, etc.) and the --user option is not valid.

Things installed with pip install --user will be installed in a local location that will only be seen by the current user account, and will not require root access (on *nix) or administrator access (on Windows).

The --user option modifies all pip commands that accept it to see/operate on the user install folder, so if you use pip list --user it will only show you packages installed with pip install --user.

Solution 5

Other answers mention site.USER_SITE as where Python packages get placed. If you're looking for binaries, these go in {site.USER_BASE}/bin.

If you want to add this directory to your shell's search path, use:

export PATH="${PATH}:$(python3 -c 'import site; print(site.USER_BASE)')/bin"

View more solutions

Comments

[Trevor Boyd Smith]

you can import site; print site.USER_SITE to print the install location. For me I got /${HOME}/.local/lib/python${PY_MAJOR}.${PY_MINOR}/site-packa‌​ges.

[noobninja]

On a host machine, /usr/local/lib/pythonX.X/dist-packages is the default directory for packages installed by pip. But if one user wants to install user-specific packages, they can use $ sudo pip3 --user install some_package. That package will remain unavailable to groups and others who access that host.

[Rob Truxal]

Thanks; that makes sense. But is the point of --user to make sure that one does not run the package as root? (I'm imagining something similar to like Wireshark/kismet/burpsuite options to set up group-access policies, thereby not allowing all the program-features to run as root. Is that on the right track?) or is the --user option just meant to allow the installation without root privileges? If that's the case, why don't I ever use sudo pip install foo_package? I've never needed root-privelages to install via pip before.

[NDEthos]

@Rob Truxal. I think the point is that the package is not going to be seen by other users. Maybe you want an older/newer version of a package but if you install it on the system you are going to muck up your work mates.

[Rob Truxal]

oh! The --user param is about user-isolation! That makes like a rediculous ammt of sense. Thanks @NDEthos!

[abbood]

ok here is a (noobish) question: suppose that i logged in as a user foo, and then I ran this command pip install --user -r requirements.txt.. and everything installed just fine. Then i logged in as user bar, and ran the python program like so: sudo -u foo ./odoo-bin.. will it read from the python packages that was installed for the foo user? or how does that work?

[abbood]

also is there a way to list only the packages that are installed for the current user? ie something like pip freeze --user?

[abbood]

you can see the context of what i'm saying in this question

[Martin Kosicky]

would be great if there was a way to configure the environment that the --user is implicit

[variable]

Ok so its like a target environment. What what if the package is install both with and without the --user, then which version will be used?

[sinoroc]

Would you consider rephrasing the first part? Outside of a Python virtual environment it really is best to avoid using pip install without the --user entirely. This would install Python packages in places that really should be left to the system's package manager (for example apt in Debian/Ubuntu). It's better not to mess with this, this leads to so many issues. If a Python package needs to be available to all users, then use the operating system's package manager, but do not do sudo pip install .... An alternative is sudo pip install --target .... On Windows it is less of an issue.

[LightCC]

Okay - you mean the second half of the final bullet point in the 'without virtual environments' section? If not, which specific parts? (feel free to edit it directly for this update, I'm not primarily a *nix user)

[sinoroc]

I see, if you don't use Windows, then it's much less of a concern indeed since it doesn't really have a centralized package manager, unless you start using something like nuget. I'll see if I come around to edit your answer.

[LightCC]

@sinoroc I added a Note to that paragraph. Feel free to update it to be more precise, etc., or edit elsewhere if I have similar wording.

[aspiring1]

@NDEthos : How do we know which python version and pip packages, we are using, the /usr/local ones or ~/.local ones? Does that depend on $PATH variable? Also, using virtualenvironments using virtualenvwrapper, accomplishes this task of saving the muck to other users and hiding stuff too, pretty well.

[aerijman]

Just to be more specific, ~/.local/lib/pythonX.X/site-packages/.. instead of /usr/local/lib/pythonX.X/site-packages where X.X is the version

[BrendanSimon]

or you want to install a module/package on a system where you do not have root/admin privleges

[E Gow]

Is the best practice to place ~/.local/bin in the PATH before system dirs like /bin and /usr/bin or after the system dirs? If the local bin is earlier in the path then a user could end up being spoofed by replacements for commands that take a password such as passwd, ssh, scp, etc.

[Timo]

I use the python app in windows 10 and pip install pkg and pip install pkg --user install both in c:\users\user\appdata\local\packages\pythonsoftwarefoundatio‌​n.python.3.9_qbz5n2k‌​fra8p0\localcache\lo‌​cal-packages\python3‌​9\site-packages. I wonder why because there is also C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.15‌​20.0_x64__qbz5n2kfra‌​8p0\Lib\site-package‌​s that is non-user? python -m site puts c:/progr.. in sys.path and c:/users.. in user_base and in sys.path. So c:/users.. is both user and non-user?

[LightCC]

@Timo If the Python install itself is a user install rather than a system insall, then there might not be a difference in where --user installs to.

[Francis Cagney]

Shouldn't be your decision. My .profile puts this directory at the head of the path if it exists. It means that anything I personally install has precedence over standard installs which is right.