Using an extra python package index url with setup.py

Question

Is there a way to use an extra Python package index (ala pip --extra-index-url pypi.example.org mypackage) with setup.py so that running python setup.py install can find the packages hosted on pypi.example.org?

Solution 1

If you're the package maintainer, and you want to host one or more dependencies for your package somewhere other than PyPi, you can use the dependency_links option of setuptools in your distribution's setup.py file. This allows you to provide an explicit location where your package can be located.

For example:

from setuptools import setup

setup(
    name='somepackage',
    install_requires=[
        'somedep'
    ],
    dependency_links=[
        'https://pypi.example.org/pypi/somedep/'
    ]
    # ...
)

If you host your own index server, you'll need to provide links to the pages containing the actual download links for each egg, not the page listing all of the packages (e.g. https://pypi.example.org/pypi/somedep/, not https://pypi.example.org/)

Solution 2

setuptools uses easy_install under the hood.

It relies on either setup.cfg or ~/.pydistutils.cfg as documented here.

Extra paths to packages can be defined in either of these files with the find_links. You can override the registry url with index_url but cannot supply an extra-index-url. Example below inspired by the docs:

[easy_install]
find_links = http://mypackages.example.com/somedir/
             http://turbogears.org/download/
             http://peak.telecommunity.com/dist/
index-url = https://mypi.example.com

Solution 3

I wanted to post a latest answer to this since both the top answers are obsolete; use of easy_install has been deprecated by setuptools.

https://setuptools.pypa.io/en/latest/deprecated/easy_install.html

Easy Install is deprecated. Do not use it. Instead use pip. If you think you need Easy Install, please reach out to the PyPA team (a ticket to pip or setuptools is fine), describing your use-case.

Please use pip moving forward. You can do one of the following:

1. provide --index-url flag to pip command
2. define index-url in pip.conf file
3. define PIP_INDEX_URL environment variable

https://pip.pypa.io/en/stable/topics/configuration/

Solution 4

The following worked for me (develop, not install):

$ python setup.py develop --index-url https://x.com/n/r/pypi-proxy/simple

Where https://x.com/n/r/pypi-proxy/simple is a local PyPI repository.

Solution 5

Found solution when using Dockerfile:

RUN cd flask-mongoengine-0.9.5 && \
    /bin/echo -e [easy_install]\\nindex-url = https://pypi.tuna.tsinghua.edu.cn/simple >> setup.cfg && \
    python setup.py install

Which /bin/echo -e [easy_install]\\nindex-url = https://pypi.tuna.tsinghua.edu.cn/simple will exists in file setup.cfg:

[easy_install]
index-url = https://pypi.tuna.tsinghua.edu.cn/simple

View more solutions

Comments

[Tommy]

this answer appears to be wrong. see the last paragraph in the answer here: stackoverflow.com/questions/13353869/…

[SpoonMeiser]

I don't think this is correct. The question asks specifically about controlling what setup.py does (which we can assume uses setuptools) and, IIUC requirements.txt is only honoured by pip

[Roman]

I ended up ditching the setup.py and using this method.

[Optimus Prime]

Would be more helpful to include location of this setup.py as well.

[Alex-Bogdanov]

setuptools unable to render --extra-index-urls in requirements.txt. The only things it expects is a list of strings with deps version details, etc. requests>=2.19

[NOZUONOHIGH]

Not working, and python setup.py install --help doesn't have any params relate to --index-url

[miku]

@NOZUONOHIGH, thanks, I corrected my answer - it was the "develop", not "install", that accepts an index-url flag.

[con-f-use]

How does that handle a http index? For instance pip will want --trusted-host <index-hostname> for http indexes.

[con-f-use]

as far as I know, dependency links have been deprecated, see e.g.: github.com/pypa/setuptools/issues/987 and github.com/pypa/pip/issues/4187

[Troy Daniels]

The link in the answer says that pip now ignored dependency_links but does not say what to use instead.

[Jason Harrison]

why not include flask-mogoengin-0.9.5/setup.cfg in your source repository? Why create it at docker build time?

[NOZUONOHIGH]

@JasonHarrison It's not create but append. By doing so, we don't need to ADD/COPY a additional modified setup.cfg file when build the docker image, one Dockerfile and everything works!

[Matthias Lohr]

Did anybody find a replacement for that?

[tony_tiger]

This is unfortunately not a robust solution and does not utilize benefits of a dependency manager. The dependency_links list needs to list all of the dependencies, including transitive ones. Example, you have a dependency on pandas and need to use a private mirror of the PyPI. Then you not only need to list pandas, but also numpy, python-dateutil, pytz

[Tad Bumcrot]

As of Aug 2021, 'index-url' should become 'index_url', at least that's the recommendation I'm getting from setuptools ` UserWarning: Usage of dash-separated 'index-url' will not be supported in future versions. Please use the underscore name 'index_url' instead`

[rtindru]

This does not seem to be working anymore as pypi dropped support fro --process-dependency-links as of v19.0

[Nerxis]

thanks for setup.cfg example, worked quite well, unfortunately easy install is deprecated now: setuptools.pypa.io/en/latest/deprecated/easy_install.html

[Lawrence I. Siden]

See the warning at the top of setuptools.pypa.io/en/latest/userguide/…