How to install PyPi packages using anaconda conda command

Question

When using the Anacoda Python distribution, what is the best way to install a PyPi package that isn't available directly through Anaconda? For now I'm using:

conda pipbuild [pypi_name]
conda install --use-local [package_spec]

But I'm unclear if this is the best way and if conda update --all will update these packages when updates are made available. I'm also unclear what the point of binstar is when PyPi already exists.

Solution 1

If you want to build conda packages for PyPI packages, the recommended way is to use conda skeleton pypi package and use conda build package on the recipe that it creates. To install the package, use conda install --use-local package (here and elsewhere, package is the name of the PyPI package you wish to install).

You will need to update the recipe each time the package is updated.

You can also use pip to install these packages. There are two disadvantages: firstly, these packages won't be managed by conda at all. Secondly, these packages will not work if your default python version is different from the python version you are using in conda.

Solution 2

I will disagree with the accepted response and note that pip install [some-pypi-package] is often the best way to install PyPi packages in Conda environments.

While the packages won't be managed by the Conda package manager, they will still be managed by the Anaconda environment. It will download the correct version of the package for the active Python install and update it correctly using the pip package manager.

When using Anaconda, you should turn to conda before pip when you can, but you don't lose any of the replicability benefits of using Anaconda when you use pip.

Anaconda recently published a doc that supports this: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#using-pip-in-an-environment

Solution 3

Since version 4.6.0, Conda has improved its interoperability with pip:

Conda and pip have historically had difficulties getting along. Pip hasn’t respected Conda’s environment constraints, while Conda has been all too happy to clobber pip-installed software. It’s a mess. Conda 4.6.0 adds preview support for better interoperability. With this interoperability, Conda can use pip-installed packages to satisfy dependencies, and can even remove pip-installed software cleanly and replace them with Conda packages when appropriate. There’s still room for improvement before pip and Conda are hunky-dory BFFs, but we hope this is a good start. This feature is disabled by default right now because it can significantly impact Conda’s performance. If you’d like to try it, you can set this condarc setting:

conda config --set pip_interop_enabled True

So, the way to get PyPI packages into conda (at the time of writing this) seems to be:

pip install <package>

If you want conda to replace the PyPI packages with its own (where possible), just run:

conda update --all

Given that the above setting is made. Conda marks its own channels as higher priority than pip, thus packages will be replaced.

Comments

[user1507844]

Thanks! Dumb question: what does condo managing the packages get me? Dependency management mostly? Automatic updates? Also, why is the method you described preferable to what I described above?

[asmeurer]

conda managing things gets you the benefits of dependency management. It also makes it easier to use that package with conda environments. Updates won't happen "automatically" (in the sense that you do need to type conda update for that to happen). As to why it's preferable, conda skeleton is much more stable than conda pipbuild.

[endolith]

It doesn't say this in conda.pydata.org/docs/using/… it just says pip install x. should those instructions be updated?

[endolith]

and is there a way to have conda manage binary packages like lfd.uci.edu/~gohlke/pythonlibs/#scikits.audiolab ?

[faph]

conda convert can convert Gohlke's packages to conda packages conda.pydata.org/docs/commands/build/conda-convert.html

[Nathan Lloyd]

Final command to install the package: conda install --use-local package

[Bryce Guinta]

Note that you may need to run conda install conda-build to get the conda skeleton command (I'm using miniconda).

[ws_e_c421]

One issue to be aware of with this: If a pip package requires a different version of a dependency already installed by conda, pip will replace the installed package, and conda won't realize its package has been removed. For a one-shot, scripted environment setup, this is probably fine. For an environment that you install into by hand over time, you will likely end up confused after a while.

[ws_e_c421]

One way to mitigate the issue I mentioned above: save the output of pip freeze to constraints.txt and then install with pip using pip install -c constraints.txt <package>. This will prevent pip from removing conda packages. It will also pin pip packages. The pip packages could be filtered out of constraints.txt by looking at the output of conda list for the items marked as <pip>.

[Chris Conlan]

For things like Django, I’ll set up a virtual environment using Anaconda then do installations purely through pip for reasons mentioned above. Pure conda works best for getting a scientific computing stack together that uses conda channels.

[Himanshuman]

CondaValueError: Key 'pip_interop_enabled' is not a known primitive parameter.

[NichtJens]

What's your conda version? You can check via conda --version. I would presume your version is simply older than 4.6.0...

[asmeurer]

@astabada, the package manager is called "conda" and the packages are called "conda packages". "Anaconda" is a distribution of conda packages, and not the only way to use conda.

[li.SQ]

I am having this error pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out when I tried doing conda skeleton pypi package.