python pip specify a library directory and an include directory

69,022

Solution 1

pip has a --global-option flag

You can use it to pass additional flags to build_ext.
For instance, to add a --library-dirs (-L) flag:
pip install --global-option=build_ext --global-option="-L/path/to/local" pyodbc

gcc supports also environment variables: http://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html

I couldn't find any build_ext documentation, so here is the command line help

Options for 'build_ext' command:
  --build-lib (-b)     directory for compiled extension modules
  --build-temp (-t)    directory for temporary files (build by-products)
  --plat-name (-p)     platform name to cross-compile for, if supported
                       (default: linux-x86_64)
  --inplace (-i)       ignore build-lib and put compiled extensions into the
                       source directory alongside your pure Python modules
  --include-dirs (-I)  list of directories to search for header files
                       (separated by ':')
  --define (-D)        C preprocessor macros to define
  --undef (-U)         C preprocessor macros to undefine
  --libraries (-l)     external C libraries to link with
  --library-dirs (-L)  directories to search for external C libraries
                       (separated by ':')
  --rpath (-R)         directories to search for shared C libraries at runtime
  --link-objects (-O)  extra explicit link objects to include in the link
  --debug (-g)         compile/link with debugging information
  --force (-f)         forcibly build everything (ignore file timestamps)
  --compiler (-c)      specify the compiler type
  --swig-cpp           make SWIG create C++ files (default is C)
  --swig-opts          list of SWIG command line options
  --swig               path to the SWIG executable
  --user               add user include, library and rpath
  --help-compiler      list available compilers

Solution 2

Building on Thorfin's answer and assuming that your desired include and library locations are in /usr/local, you can pass both in like so:

sudo pip install --global-option=build_ext --global-option="-I/usr/local/include/" --global-option="-L/usr/local/lib"  <you package name>

Solution 3

Another way to indicate the location of include files and libraries are set relevant environment variables before running pip e.g.

export LDFLAGS=-L/usr/local/opt/openssl/lib
export CPPFLAGS=-I/usr/local/opt/openssl/include
pip install cryptography

Solution 4

Just FYI... If you are having trouble installing a package with pip, then you can use the

--no-clean option to see what is exactly going on (that is, why the build did not work). For instance, if numpy is not installing properly, you could try

pip install --no-clean numpy

then look at the Temporary folder to see how far the build got. On a Windows machine, this should be located at something like:

C:\Users\Bob\AppData\Local\Temp\pip_build_Bob\numpy

Just to be clear, the --no-clean option tries to install the package, but does not clean up after itself, letting you see what pip was trying to do.

Otherwise, if you just want to download the source code, then I would use the -d flag. For instance, to download the Numpy source code .tar file to the current directory, use:

pip install -d %cd% numpy

Solution 5

I was also helped by Thorfin's answer; I was building GTK3+ on windows and installing pygobject, I was having difficulties on how to include multiple folders with pip install.

I tried creating pip config file as per pip documentation. but failed. the one working is with the command line:

pip install --global-option=build_ext --global-option="-IlistOfDirectories" 
# and/or with:  --global-option="-LlistofDirectories"

the separator that works with multiple folders in windows is ';' semicolon, NOT colon ':' it might be different in other OS.

sample working command line:

pip install --global-option=build_ext --global-option="-Ic:/gtk-build/gtk/x64/release/include;d:/gtk-build/gtk/x64/release/include/gobject-introspection-1.0" --global-option="-Lc:\gtk-build\gtk\x64\release\lib" pygobject==3.27.1

you can use '' or '/' for path, but make sure do not type backslash next to "

this below will fail because there is backslash next to double quote

pip install --global-option=build_ext --global-option="-Ic:\willFail\" --global-option="-Lc:\willFail\" pygobject==3.27.1
Share:
69,022

Related videos on Youtube

Cricri
Author by

Cricri

Updated on November 25, 2020

Comments

  • Cricri
    Cricri over 3 years

    I am using pip and trying to install a python module called pyodbc which has some dependencies on non-python libraries like unixodbc-dev, unixodbc-bin, unixodbc. I cannot install these dependencies system wide at the moment, as I am only playing, so I have installed them in a non-standard location. How do I tell pip where to look for these dependencies ? More exactly, how do I pass information through pip of include dirs (gcc -I) and library dirs (gcc -L -l) to be used when building the pyodbc extension ?

    • Bryan
      Bryan over 10 years
      Was python installed with the --user option?
  • Cricri
    Cricri over 10 years
    James, indeed. In the answer I have given I should have specified that I have used virtualenv as well.
  • gred
    gred about 10 years
    This doesn't really help, since the OP clearly stated that they need to install non-python libraries (i.e. pass args to the compiler/linker). I'm actually in the same boat, and I'm a little frustrated that every third answer to this question is "use virtualenv", because (while I agree that virtualenv is a great tool) it doesn't really address this problem.
  • nicholsonjf
    nicholsonjf about 10 years
    how did you point setup.py to the dependencies you installed in the non-standard location?
  • nicholsonjf
    nicholsonjf about 10 years
    @gred, after a closer look at the question, I agree...virtualenv isn't the right solution here. Looks like the OP found a workaround with by using setup.py.
  • Praveen
    Praveen about 8 years
    I wish this were better documented. This was pretty much the only thing that worked for me, installing scikit-learn against a statically compiled ATLAS, combined with option-specification as described here
  • yota
    yota over 7 years
    like this python setup.py build_ext --rpath=/usr/local/lib ?
  • vivekv
    vivekv almost 5 years
    I am not sure how this remotely helps the OP. This looks like a pitch for using virtualenv
  • nicholsonjf
    nicholsonjf almost 5 years
    @vivekv If you had read the comments previous to yours, you'd see that this point was already made and subsequently acknowledged by me
  • snark
    snark over 4 years
    I found --install-option also worked instead of --global-option for specifying paths to libraries. I'm not sure when one should be used rather than the other though. Maybe --install-option only applies to install and --global-option has a wider scope. See also pip.pypa.io/en/stable/reference/pip_install/…
  • Totomobile
    Totomobile over 3 years
    If you want to include multiple libraries or include directories, they are separated by a semi-colon, not a colon.
  • not2qubit
    not2qubit over 3 years
    What about windows paths? What is default and how to write them?
  • Kentzo
    Kentzo about 3 years
    I wonder if there is a way to specify a path that is relative to the build directory that PIP temporary creates while it's running.
  • Tomasz Gandor
    Tomasz Gandor about 2 years
    Wow, this is really great; it also helps when doing python setup.py build and python setup.py bdist_wheel!
  • Robert Lujo
    Robert Lujo over 1 year
    In my case I needed to add CFLAGS=-I/usr/local/opt/...../include too.