How to download cross-platform wheels via pip?

Question

I'm accustomed to pre-downloading packages using Pip, then copying them over to a target machine for deployment. With the newly introduced Python Wheels, I'm forced to "pip ... --no-use-wheel", as some of the downloaded packages are platform specific (I'm developing on OSX and deploying to Debian) and will not install on the target machine. Is there a way to download Wheels for target platforms (or platform independent)?

Solution 1

The pip download command now has the --platform argument, which you can use to specify the desired platform:

pip download --platform=manylinux1_x86_64 --only-binary=:all: lxml

• the --platform=manylinux1_x86_64 option indicates that you want wheels for this specific platform. manylinux1_x86_64 means roughly "compatible with most distributions and with an intel CPU architecture". This answer links to some PEPs that describe which platforms exist and what OS/CPU they are compatible with.
• the --only-binary=:all: forces the use of binary distribution packages (ie. wheels, as opposed to sdist "source distribution packages") for ":all:" the things that will be installed in this command. Instead of :all:, one can pass a comma-separated list of specific distribution packages; see pip install --help for more info.

Note: I use the term "distribution package" to avoid confusion with the other kind of "package" (the ones one can import in a python script).

Solution 2

The easiest way to achieve that is IMO to use a custom script.

You can access the whole of the PyPI index via the simple interface, if the package of interest offers one or more wheels, they will be listed at the same address + /<package-name>.

For example: if you were to install setuptools all wheels would be listed at: https://pypi.python.org/simple/setuptools/

In your script, remember to implement the recommended tag priority as specified by PEP-425. Essentially that boils down to download the most specific (as opposed to the most general) version of the package as this normally translate into performance advantages, with for example C extensions replacing pure python implementations of some algorithm.

Comments

[Cloud]

Why there is no matching distribution for some packages? Such as pip download --only-binary=:all: --platform=manylinux1_x86_64 tornado. How can I download those packages?

[sitaktif]

@SiminJie not all the packages have all distributions available as wheels, but I'm sure the package maintainer would accept a contribution to address that ;-)

[Alexander Stohr]

whats the individual options for? (i want to learn something from this entry.)