Optional dependencies in a pip requirements file

Question

How can I specify optional dependencies in a pip requirements file?

According to the pip documentation this is possible, but the documentation doesn't explain how to do it, and I can't find any examples on the web.

Solution 1

Instead of specifying optional dependencies in the same file as the hard requirements, you can create a optional-requirements.txt and a requirements.txt.

To export your current environment's packages into a text file, you can do this:

pip freeze > requirements.txt

If necessary, modify the contents of the requirements.txt to accurately represent your project's dependencies. Then, to install all the packages in this file, run:

pip install -U -r requirements.txt

-U tells pip to upgrade packages to the latest version, and -r tells it to install all packages in requirements.txt.

Solution 2

In 2015 PEP-0508 defined a way to specify optional dependencies in requirements.txt:

requests[security]

That means that yourpackage needs requests for its security option. You can install it as:

pip install yourpackage[security]

Comments

[Michael Scott Asato Cuthbert]

Great Q. -- a related q. would be how to define a set of requirements that should be attempted to be installed but not to consider installation a failure if they cannot be installed. I have a package that works better if numpy is installed but has a fallback if numpy can't compile for some reason. I would love to make it a default-installed but optional if failed requirement.

[del]

I think you've misunderstood the question. 'pip freeze' will just print out all of the dependencies. What I want to know is how I can specify which dependencies are required and which are optional within the pip requirements file.

[Daniel Naab]

I see the reference in the docs I think you're referring to, but I'm not sure it's possible in one requirements file... although you could have two dependency files, one which lists optional dependencies. I'll modify my answer

[del]

Thanks - this is the approach I was already taking, but reading the bit about optional dependencies in the doc made me think there might be a better way to do it.

[del]

You aren't really free to include both the required and optional dependencies in a requirements file because 'pip install' will immediately abort if any of the packages in the file fail to install. It seems that using two separate requirements files is the only correct solution.

[gene_wood]

You can configure one of these "extras" using the extras_require argument for the setup function in setuptools. You can see in the requests setup.py how the security "extra" is configured.

[Jon-Eric]

Related: How to install python module extras with pip requirements.txt file.

[endolith]

When I try this I just get a lot of errors like Collecting pytest-cov[tests] ... WARNING: pytest-cov 2.10.0 does not provide the extra 'tests' , Collecting numba[fast]<0.47 ... WARNING: numba 0.46.0 does not provide the extra 'fast'"

[endolith]

Oh I think the problem is that in the requirements.txt, the brackets indicate an "extra" of the dependency, while I was interpreting it to mean "this is only a dependency of my package if my package is installed with that extra"

[Flair]

If you are installing from github, you can use something like pip install "requests[security] @ git+ssh://[email protected]/psf/requests.git" instead