Is there a way to list pip dependencies/requirements?

Question

Without going through with the installation, I want to quickly see all the packages that pip install would install.

Solution 1

This was tested with pip versions 8.1.2, 9.0.1, 10.0.1, and 18.1.

To get the output without cluttering your current directory on Linux use

pip download [package] -d /tmp --no-binary :all: -v

-d tells pip the directory that download should put files in.

Better, just use this script with the argument being the package name to get only the dependencies as output:

#!/bin/sh

PACKAGE=$1
pip download $PACKAGE -d /tmp --no-binary :all:-v 2>&1 \
| grep Collecting \
| cut -d' ' -f2 \
| grep -Ev "$PACKAGE(~|=|\!|>|<|$)"

Also available here.

Solution 2

Check out my project johnnydep!

Installation:

pip install johnnydep

Usage example:

$ johnnydep requests
name                       summary
-------------------------  ----------------------------------------------------------------------
requests                   Python HTTP for Humans.
├── certifi>=2017.4.17     Python package for providing Mozilla's CA Bundle.
├── chardet<3.1.0,>=3.0.2  Universal encoding detector for Python 2 and 3
├── idna<2.7,>=2.5         Internationalized Domain Names in Applications (IDNA)
└── urllib3<1.23,>=1.21.1  HTTP library with thread-safe connection pooling, file post, and more.

A more complex tree:

$ johnnydep ipython 
name                              summary
--------------------------------  -----------------------------------------------------------------------------
ipython                           IPython: Productive Interactive Computing
├── appnope                       Disable App Nap on OS X 10.9
├── decorator                     Better living through Python with decorators
├── jedi>=0.10                    An autocompletion tool for Python that can be used for text editors.
│   └── parso==0.1.1              A Python Parser
├── pexpect                       Pexpect allows easy control of interactive console applications.
│   └── ptyprocess>=0.5           Run a subprocess in a pseudo terminal
├── pickleshare                   Tiny 'shelve'-like database with concurrency support
├── prompt-toolkit<2.0.0,>=1.0.4  Library for building powerful interactive command lines in Python
│   ├── six>=1.9.0                Python 2 and 3 compatibility utilities
│   └── wcwidth                   Measures number of Terminal column cells of wide-character codes
├── pygments                      Pygments is a syntax highlighting package written in Python.
├── setuptools>=18.5              Easily download, build, install, upgrade, and uninstall Python packages
├── simplegeneric>0.8             Simple generic functions (similar to Python's own len(), pickle.dump(), etc.)
└── traitlets>=4.2                Traitlets Python config system
    ├── decorator                 Better living through Python with decorators
    ├── ipython-genutils          Vestigial utilities from IPython
    └── six                       Python 2 and 3 compatibility utilities

Solution 3

If and only if the package is install, you can use pip show <package>. Look for the Requires: filed at the end of the output. Clearly, this breaks your requirement but might be useful nonetheless.

For example:

$ pip --version
pip 7.1.0 [...]
$ pip show pytest
---
Metadata-Version: 2.0
Name: pytest
Version: 2.7.2
Summary: pytest: simple powerful testing with Python
Home-page: http://pytest.org
Author: Holger Krekel, Benjamin Peterson, Ronny Pfannschmidt, Floris Bruynooghe and others
Author-email: holger at merlinux.eu
License: MIT license
Location: /home/usr/.tox/develop/lib/python2.7/site-packages
Requires: py

Solution 4

Note: the feature used in this answer was deprecated in 2014 and removed in 2015. Please see other answers that apply to modern pip.

The closest you can get with pip directly is by using the --no-install argument:

pip install --no-install <package>

For example, this is the output when installing celery:

Downloading/unpacking celery                                                                                   
  Downloading celery-2.5.5.tar.gz (945Kb): 945Kb downloaded
  Running setup.py egg_info for package celery

    no previously-included directories found matching 'tests/*.pyc'
    no previously-included directories found matching 'docs/*.pyc'
    no previously-included directories found matching 'contrib/*.pyc'
    no previously-included directories found matching 'celery/*.pyc'
    no previously-included directories found matching 'examples/*.pyc'
    no previously-included directories found matching 'bin/*.pyc'
    no previously-included directories found matching 'docs/.build'
    no previously-included directories found matching 'docs/graffles'
    no previously-included directories found matching '.tox/*'
Downloading/unpacking anyjson>=0.3.1 (from celery)
  Downloading anyjson-0.3.3.tar.gz
  Running setup.py egg_info for package anyjson

Downloading/unpacking kombu>=2.1.8,<2.2.0 (from celery)
  Downloading kombu-2.1.8.tar.gz (273Kb): 273Kb downloaded
  Running setup.py egg_info for package kombu

Downloading/unpacking python-dateutil>=1.5,<2.0 (from celery)
  Downloading python-dateutil-1.5.tar.gz (233Kb): 233Kb downloaded
  Running setup.py egg_info for package python-dateutil

Downloading/unpacking amqplib>=1.0 (from kombu>=2.1.8,<2.2.0->celery)
  Downloading amqplib-1.0.2.tgz (58Kb): 58Kb downloaded
  Running setup.py egg_info for package amqplib

Successfully downloaded celery anyjson kombu python-dateutil amqplib

Admittedly, this does leave some cruft around in the form of temporary files, but it does accomplish the goal. If you're doing this with virtualenv (which you should be), the cleanup is as easy as removing the <virtualenv root>/build directory.

Solution 5

I quote an alternative solution from @onnovalkering:

PyPi provides a JSON endpoint with package metadata:

>>> import requests
>>> url = 'https://pypi.org/pypi/{}/json'
>>> json = requests.get(url.format('pandas')).json()
>>> json['info']['requires_dist']
['numpy (>=1.9.0)', 'pytz (>=2011k)', 'python-dateutil (>=2.5.0)']
>>> json['info']['requires_python']
'>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*'

For a specific package version, add an additional version segment to the URL:

https://pypi.org/pypi/pandas/0.22.0/json

Also if you are using conda (as suggested by @ShpielMeister), you can use:

conda info package==X.X.X

to display information, including dependencies for a particular version or:

conda info package

to display information, including dependencies about all supported versions of that package.

View more solutions

Comments

[dusan]

Related: stackoverflow.com/q/9232568/183791

[pgmank]

Another question including alternative answers for this problem: stackoverflow.com/questions/41816693/…

[Admin]

The reason for this is that the metadata doesn't exist outside of setup.py so unlike say with rpm or dpkg where you build a metadata index on top and query that pip and pypi don't work that way. So we have to pass over each requirement.

[lastoneisbearfood]

Thanks @paul, your comment makes the answer complete

[Colonel Panic]

I tried pip --no-install celery but I receive the error no such option: --no-install (pip 1.2.1)

[entropy]

I think he meant pip install --no-install celery

[Jian]

On my pip version (1.5.4) the --no-install flag is deprecated.

[radtek]

For 1.5.4, use pip install --download=. --no-use-wheel celery

[Alison S]

pip now displays a warning deprecating --download and --no-use-wheel as well.

[Merlyn Morgan-Graham]

Pip now says to use pip download <packagename>. They sure are deprecation happy!

[Cecil Curry]

pip download <packagename> technically works, but has the obvious disadvantage of polluting the current working directory with package cruft. See The Card Cheat's one-liner and related shell script for clever alternatives.

[Cecil Curry]

Ridiculously, --download has been deprecated as well. The canonical command now appears to be pip download <package> -d /tmp --no-binary :all: as suggested by The Card Cheat.

[Alexander Fradiani]

isn't pip freeze supposed to do this?

[wim]

I'm glad a command ridiculous sounding as "pip install --no-install" was deprecated .. :)

[Ian Clark]

A very (very) crude reading of requirements.txt using this: < requirements.txt egrep -v "^#" | egrep -v "^$" | xargs -L 1 -I % sh -c 'echo %; echo "======"; ./deps.sh %; echo "";

[Jmills]

@hans-musgrave made a good point in another answer that I hadn't noticed previously, so updated the bash script to only exclude lines that match the package along with end of line or the start of a valid version specifier rather than any line that contains the package name.

[wim]

Some packages only provide binary, so --no-binary :all: is not a good idea. A project which only shipped wheel and not sdist would fail.

[wim]

This only shows the direct requirements, all the transitive dependencies would be missing. And it requires an installation. So, it doesn't really answer the question.

[so860]

I downloaded this and use it, it's a great package. BUT doesn't it require packages to be installed? The OP is specifically requesting an approach that doesn't require installation. Important to caveat.

[wim]

@so860 No, it does not require the packages to be installed. That's the whole point, it works in an isolated environment.

[Louis Yang]

This end up download and compile for all the dependence packages which can be very slow....

[wim]

I downvoted because this json endpoint is not reliable. For an example look at boto3, the requires_dist is null but that is a project which certainly has dependencies in the metadata.

[jenniwren]

For those wanting a quick one-liner, here's an example for pandas, with files being stored in ~/pip-tmp: pip download pandas -d ~/pip-tmp -v 2>&1 | grep "Collecting". This gives you a quick overview of what the requirements are. Obviously you'd then want to delete the files from the ~/pip-tmp directory. Also I guess it could take a while if there are a plethora of requirements.

[wim]

Since recent versions of pip can be using PEP 517 / PEP 518, this may also start to turn up false positives because of the build-system requirements. Unreliable.

[GPHemsley]

Note that this does not list dependencies that are already installed (which is fine for OP).

[GPHemsley]

To be clear: installing johnnydep itself installs dependencies.

[Jonathan DEKHTIAR]

@wim: this project is pure brillance ! Love it !

[Ben Farmer]

This seems like a nice package, but it does seem to install the packages. In a separate env, sure, but it still has to actually install the packages there, which is very slow when there are a lot of packages, and it fails if the installation of a package fails. So I do not think it fulfils the OP requirement to see the dependencies without going through the installation process.

[wim]

@BenFarmer That's incorrect. It doesn't install packages, it reads the metadata from wheel files, which does require a package download- but not an installation. In the rare case that a project has published a source distribution but no compatible wheel is available, then pip will attempt to generate a wheel from the sdist (that is the only reliable way to get package metadata from sdist). In the most common case, there is no installation process.

[wim]

It is a slow process for large dep trees, unfortunately, because the PyPI json APIs don't provide the necessary information to do this without a package download. That's not something specific to johnnydep though, poetry and other projects all have the same slowness due to the package download requirements.

[Ben Farmer]

Ah my mistake, I just saw output about setup.py, and "Failed to build scipy, Cleaning up..." and assumed it was doing an install. It's a bit absurd that this is so hard in Python. Or I guess pip. I guess maybe conda can do it more easily, if you only need to use that.

[Ben Farmer]

What if I want to do this for a local package I am working on? e.g. that I would install with "pip install -e ./mypackage"? Sometimes I just want to know what the full dependency tree is for a project that I am working on.

[yooloobooy]

lovely utilizy. Thank you!

[Timo]

@GPHemsley .. and it has quite some dependencies, see pip show: Requires: structlog, wheel, setuptools, wimpy, cachetools, anytree, distlib, tabulate, colorama, pip, packaging, toml, pkginfo, oyaml