how to download pip dependencies locally?

Question

I am running my python application that has requirements.txt file which contains various dependencies. I am deploying this application in a Pivotal Cloud Foundry environment. However, the environment I am deploying in it airgapped. Therefore I can't seem to get the dependencies.

The Git repo for python CF buildpack suggests that if an application has a vendor directory then it might get the dependencies from there: https://github.com/cloudfoundry/python-buildpack/blob/master/bin/steps/pip-install#L18

My question is, how can I download the dependencies mentioned in my requirements.txt file locally into a vendor folder?

Solution 1

You can fetch all dependencies with the following command (an Internet connection is of course required)

pip download -r requirements.txt

Then, you can install offline those dependencies with the following command

pip install -r requirements.txt --no-index --find-links file:///tmp/packages

--no-index : Ignore package index (only looking at --find-links URLs instead).

-f, --find-links <URL> : If a URL or path to an html file, then parse for links to archives. If a local path or file:// URL that's a directory, then look for archives in the directory listing.

This answer is taken from this post

Solution 2

Look at: http://docs.cloudfoundry.org/buildpacks/python/index.html#vendor-app-dependencies

tl;dr pip install --download vendor -r requirements.txt

Just use pip to install on the dev box and then check that in.

Comments

[Anthony]

So my requirements.txt has numpy in it. I run the above command using docker container for CF: docker run -v $PWD:/myapp -it cloudfoundry/cflinuxfs2 bash then I install pip in that and then run pip install --download vendor -r requirements.txt. This downloads the files in my vendor directory. and one of the files is called numpy-1.13.0-cp27-cp27mu-manylinux1_x86_64.whl. However, when I try to push my application, I get an error numpy-1.13.0-cp27-cp27mu-manylinux1_x86_64

[jonrsharpe]

If you find a duplicate, please close as such rather than copying over the answer.