pip: install dependencies of dependencies

Question

I would like to manually install all the requirements of a Python package I'm writing. To this end, I created the file requirements.txt and added the dependencies, line by line:

$ cat requirements.txt
meshio
numpy

When running

$ pip install -r requirements.txt

those two packages are installed alright, but I noticed that the dependencies of meshio aren't (i.e., whatever is listed in its requirements.txt). Not surprising, how is pip supposed to know?

Is there a mechanism for installing the entire dependency tree with pip?

Solution 1

Turns out for the dependencies to be installed, the packages needs to list its dependencies as

install_requires=[
    'numpy',
    'pyyaml'
],

as part of setup() in setup.py, not in requirements.txt.

Solution 2

You may be interested by pip-tools, a python package that can be used to build a requirements.txt file that takes into account all underlying dependencies. It can be installed via pip:

pip install --upgrade pip  # pip-tools needs pip>=6.
pip install pip-tools

Once installed, you can use the pip-compile command to generate your requirements file. For example, suppose you work on a Flask project. You would have to do the following:

Write the following line to a file:

Flask

Run pip-compile <your-file>. It will produce your requirements.txt, with all the dependencies pinned. You can re-run pip-compile to update the packages. Your output file will look like this:

#
# This file is autogenerated by pip-compile
# Make changes in requirements.in, then run this to update:
#
#    pip-compile <your-file>
#
flask==0.10.1
itsdangerous==0.24        # via flask
jinja2==2.7.3             # via flask
markupsafe==0.23          # via jinja2
werkzeug==0.10.4          # via flask

Comments

[JR Utily]

I was already using pip-tools for development, I didn't think of it to install things in production. But it comes in handy ! Thanks a lot.

[JR Utily]

if there are transitive dependencies, to ensure pip get the right versions for all of them, you need to analyse all the setup.py files, and pip-tools can do it for you