Installing python module within code

Question

I need to install a package from PyPi straight within my script. Maybe there's some module or distutils (distribute, pip etc.) feature which allows me to just execute something like pypi.install('requests') and requests will be installed into my virtualenv.

Solution 1

The officially recommended way to install packages from a script is by calling pip's command-line interface via a subprocess. Most other answers presented here are not supported by pip. Furthermore since pip v10, all code has been moved to pip._internal precisely in order to make it clear to users that programmatic use of pip is not allowed.

Use sys.executable to ensure that you will call the same pip associated with the current runtime.

import subprocess
import sys

def install(package):
    subprocess.check_call([sys.executable, "-m", "pip", "install", package])

Solution 2

You can also use something like:

import pip

def install(package):
    if hasattr(pip, 'main'):
        pip.main(['install', package])
    else:
        pip._internal.main(['install', package])

# Example
if __name__ == '__main__':
    install('argh')

Solution 3

If you want to use pip to install required package and import it after installation, you can use this code:

def install_and_import(package):
    import importlib
    try:
        importlib.import_module(package)
    except ImportError:
        import pip
        pip.main(['install', package])
    finally:
        globals()[package] = importlib.import_module(package)


install_and_import('transliterate')

If you installed a package as a user you can encounter the problem that you cannot just import the package. See How to refresh sys.path? for additional information.

Solution 4

This should work:

import subprocess

def install(name):
    subprocess.call(['pip', 'install', name])

Solution 5

i added some exception handling to @Aaron's answer.

import subprocess
import sys

try:
    import pandas as pd
except ImportError:
    subprocess.check_call([sys.executable, "-m", "pip", "install", 'pandas'])
finally:
    import pandas as pd

View more solutions

Comments

[chuwy]

Yes it's definitely should work. But I thought there is more elegant way;) I'll be waiting a little bit, may be there is.

[quantum]

@Downvoter: What exactly is wrong with my answer? This answer has all the OP wanted. It doesn't even use a shell.

[Lukas Graf]

This is the correct answer and the only sensible way to manage a Python projects' dependencies. It will work with virtualenv, Fabric, buildout, you name it. The method described by @xiaomao, even though answering exactly what the OP asked, is pure madness.

[fiatjaf]

Here this exits after installing.

[GaryBishop]

It depends on the right version of pip being first on the path. If the user is running an alternate python installation, pip will install into the first one instead of the current one. The import approach above will install in the right place. I upvoted anyway to counter the down vote.

[jason]

@Rikard Anglerud. Is there an option to upgrade the packages in a batch? something like 'install --upgrade'?

[Patrick]

@RikAng What to change in your code if i want to install multiple packages

[nbro]

Is there a way to pass options when installing, for example the version of the package? If yes, could you please add it to your answer just for completeness.

[Kaos]

@nbro you pass options to pip.main() as you would on the command line (but with each option as a separate entry in the list, instead of a single string). And to specify which version of the package you want, do the same as on the command line.. ex: pip.main(['install', 'pip==7.1.0'])

[Myer]

See also stackoverflow.com/questions/6457794/…, which shows how to handle the case where an install fails.

[kgadek]

Any idea how to do that on Python 3? imp.reload(site) gets me RuntimeError: dictionary changed size during iteration

[Smit Johnth]

Windows even don't have pip on PATH by default.

[Jared]

This is also a great option for installing new python libraries on Windows via the Python console if you don't have a machine with powershell on it.

[Natim]

Depending on how the script is running you wont call the right pip.

[Xabs]

How can I have this working for a particular virtualenv? I'm trying to automate generating wheels for a matrix of packages and python versions. What I want is to create a virtualenv, install some dependencies, and create the wheel.

[Xabs]

Replying to myself: to install in a specific virtualenv: pip install --target=my-virtualenv/lib/python2.7/site-packages.

[kaycee]

I'm getting ValueError: Unable to configure handler 'console': 'OutStream' object has no attribute 'closed' when using this function. Anyone know why ?

[Ishan Khare]

where does this install the package, after i did this, i was not able to do pip uninstall <package_name>. I can still uninstall it using pip.main but just wanted to know where does it install the package?

[tribbloid]

Doesn't work and gave me the error: Traceback (most recent call last): File "/home/***/git/pip.py", line 9, in <module> install('arg') File "/home/***/git/pip.py", line 4, in install pip.main(['install', package]) AttributeError: 'module' object has no attribute 'main'

[running.t]

There is a lot of situations when this approach will not work. e.g. when you use several versions of python on one machine and you interpret current script with not default python.

[Corey Goldberg]

while this is proper advice, it doesn't answer the question asked

[hoefling]

While packaging is a topic, there are a lot of other use cases, for example a deployment script written in python.

[Fallenreaper]

Was curious. would this work properly if i do: pip install requests[security] ? I wasnt sure if it would properly define the globals correctly.

[pitchblack408]

How do you do this with pip3?

[uchuugaka]

Apparently, this approach is preferred by the pip team at this time.

[Kelvin Ng]

Sadly, this works well with virtualenv github.com/explosion/spaCy/commit/…

[3pitt]

from pip._internal import main as pipmain then you can use pipmain() just like the deprecated pip.main() see stackoverflow.com/questions/43398961/…

[bli]

For some reason, after compiling and installing a local version of python 2.7.15, the command-line pip was not working (it tried to import stuff from the system-installed python and complained about missing pip distribution). The above solution worked for me, but after upgrading pip (pip.main(["install", "--upgrade", "pip"])), I had to use the modification suggested by @MikePalmice to further install packages.

[red888]

how do you use a requirements file and install packages to a different directory?

[Morse]

its depreceated now.

[Jordan Mackie]

"Sadly, this works well with..." @KelvinNg did you mean to say it doesn't work well with virtualenv?

[Kelvin Ng]

May not work with virtualenv github.com/explosion/spaCy/commit/…

[Kelvin Ng]

@JordanMackie No, I just feel sad call pip programatically doesn't work but this one does.

[nbk]

It's deprecated for a reason & not recommended anymore. see pip.pypa.io/en/latest/user_guide/#using-pip-from-your-progra‌​m

[cowlinator]

This fails with AttributeError: module 'pip' has no attribute '_internal'. However, it does work if you import pip._internal

[Shantanu Bedajna]

nice implementation of subprocess and pip, better than most solutions here

[Basj]

This works for me on Python 3.6.7: import pip._internal.main ; pip._internal.main.main(['install', 'MODULENAMEHERE']).

[wim]

Does not work, update please. pip.pypa.io/en/latest/user_guide/#using-pip-from-your-progra‌​m

[wim]

Outdated. pip.main no longer works. pip.pypa.io/en/latest/user_guide/#using-pip-from-your-progra‌​m

[wim]

Calling [sys.executable, '-m', 'pip', 'install', name] is making sure to get the "right" pip here.

[Antti Haapala -- Слава Україні]

You're not checking the retunr value of subprocess.call so the code might fail.

[dotbit]

the official python docs explicitly recommend AGAINST doing such a thing. Whyever was this upvoted?

[jdpipe]

One issue with this is that, for novice users on Windows, python and pip are not always on their PATH, and so a .py file that could be double-clicked would be quite convenient, whereas a "pip install xxx" comment can be quite tricky.

[Borislav Aymaliev]

Because in specific cases (e.g. when your cannot update your PATH on a corporate network) this is the only way to install a module. Generally, it is hard to take the docs seriously when it recommends "against something" without providing an alternative solution.

[parvij]

CalledProcessError: Command '['C:\\ProgramData\\Anaconda3\\pythonw.exe', '-m', 'pip', 'install', 'googleapiclient']' returned non-zero exit status 1.

[user441669]

I'm trying to use this approach, but my python is embedded/started from another executable, so "sys.executable" doesn't return the right path. Is there an alternative that would work for python that's started by some other process?

[Coddy]

why subprocess.check_call and not subprocess.call?

[sh37211]

Does importing within a function really import into the main namespace, or just the namespace of that install_and_import function?

[Glen Thompson]

This is just a copy of Tanmay Shrivastava's answer

[Daniel Bandeira]

Ok, running "import pandas as pd" brings no problem, but... logically isn't it ugly?

[Shubham Garg]

this is so easy and simple to understand for beginners as compared to all the other answers. Thank you very much.

[CIsForCookies]

Why doesn't this work if the script that installs and reloads calls another script that does the import?

[Serge Stroobandt]

The asterisk in front of *missing serves to unpack the set represented by the variable name missing. See also: Understanding the asterisk(*) of Python

[BND]

strange thing I need to rerun the script the module was installed from so that it will be recognized

[Aaron de Windt]

@BND You could automate this using os.execv(...) or it's variants. Something like os.execv(" ".join((sys.excecutable, *sys.argv))) might work.

[Aaron de Windt]

@BND Another solution would be to use imp.reload(...) to reload the module if you already imported it before reinstalling.