Python Build Error: failed to build modules _ssl and _hashlib

Question

I am adding python 2.7.13 as an altinstall by installing the source code to my RHEL4 box with

wget --no-check-certificate https://www.python.org/ftp/python/2.7.13/Python-2.7.13.tar.xz
tar -xvzf Python2.7.13.tar.xz
cd Python2.7.13
./configure --with-ensurepip=install
make
make test
make altinstall

so that I do not overwrite the default python that is required for other use. Python 2.7.13 would successfully install but was missing several essential modules for the project I am working on.

Originally the _ssl and _haslib modules would error in this section.

Python build finished, but the necessary bits to build these modules were not found:
_bsddb             _sqlite3           _tkinter
bsddb185           dbm                dl
gdbm               imageop            sunaudiodev

To find the necessary bits, look in setup.py in detect_modules() for the module's name.

I installed openssl and ensured that they were in the default location that python was looking for them, so now I have the necessary bits but then it ends with this message instead

Failed to build these modules:
_hashlib           _ssl

Below is the entire output of the python2.7 setup.py build from the unzipped python package. I have been scouring google and anywhere I can find but I have been unsuccessful in anything so far

running build
running build_ext
INFO: Can't locate Tcl/Tk libs and/or headers
building '_ssl' extension
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/ssl/include -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/include/python2.7 -c /Python/Modules/_ssl.c -o build/temp.linux-x86_64-2.7/Python/Modules/_ssl.o
/Python/Modules/_ssl.c:57: warning: ignoring #pragma GCC diagnostic
/Python/Modules/_ssl.c: In function ‘_setup_ssl_threads’:
/Python/Modules/_ssl.c:4012: warning: comparison is always false due to limited range of data type
gcc -pthread -shared build/temp.linux-x86_64-2.7/Python/Modules/_ssl.o -L/usr/local/ssl/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.linux-x86_64-2.7/_ssl.so
/usr/bin/ld: /usr/local/ssl/lib/libssl.a(s3_meth.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/usr/local/ssl/lib/libssl.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
building '_hashlib' extension
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/ssl/include -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/include/python2.7 -c /Python/Modules/_hashopenssl.c -o build/temp.linux-x86_64-2.7/Python/Modules/_hashopenssl.o
gcc -pthread -shared build/temp.linux-x86_64-2.7/Python/Modules/_hashopenssl.o -L/usr/local/ssl/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.linux-x86_64-2.7/_hashlib.so
/usr/bin/ld: /usr/local/ssl/lib/libcrypto.a(o_names.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/usr/local/ssl/lib/libcrypto.a: could not read symbols: Bad value
collect2: ld returned 1 exit status

Python build finished, but the necessary bits to build these modules were not found:
_bsddb             _sqlite3           _tkinter
bsddb185           dbm                dl
gdbm               imageop            sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


Failed to build these modules:
_hashlib           _ssl

 running build_scripts

When I attempt to use pip that is installed with my python 2.7.13 I get an SSL error so I have been installing all my packages and other modules from the source like cx_Oracle and CherryPy.

pip2.7 install cffi
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting cffi
  Could not fetch URL https://pypi.python.org/simple/cffi/: There was a problem confirming the ssl certificate: Can't connect to HTTPS URL because the SSL module is not available. - skipping
  Could not find a version that satisfies the requirement cffi (from versions: )
No matching distribution found for cffi

I also tried to add the ssl module manually with

wget --no-check-certificate https://pypi.python.org/packages/83/21/f469c9923235f8c36d5fd5334ed11e2681abad7e0032c5aba964dcaf9bbb/ssl-1.16.tar.gz#md5=fb12d335d56f3c8c7c1fefc1c06c4bfb
tar -xvzf ssl-1.16.tar.gz
cd ssl-1.16
python2.7 setup.py build

But I get an error that it should not be used with python past 2.6

Traceback (most recent call last):
  File "setup.py", line 12, in <module>
    + "or earlier.")
ValueError: This extension should not be used with Python 2.6 or later (already built in), and has not been tested with Python 2.3.4 or earlier.

EDIT

I was looking around for solutions and after combing over the outputs of the setup.py build and found someone with a similar-ish problem that seems to be related to openssl here so I rebuilt my openssl with

./config enable-shared
make
make test
make install

and now I get a slightly different error about the ssl module, am I just screwing up my environment more and more?

python2.7 setup.py build
running build
running build_ext
INFO: Can't locate Tcl/Tk libs and/or headers
building '_ssl' extension
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/ssl/include -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/include/python2.7 -c /Python/Modules/_ssl.c -o build/temp.linux-x86_64-2.7/Python/Modules/_ssl.o
/Python/Modules/_ssl.c:57: warning: ignoring #pragma GCC diagnostic
/Python/Modules/_ssl.c: In function ‘_setup_ssl_threads’:
/Python/Modules/_ssl.c:4012: warning: comparison is always false due to limited range of data type
gcc -pthread -shared build/temp.linux-x86_64-2.7/Python/Modules/_ssl.o -L/usr/local/ssl/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.linux-x86_64-2.7/_ssl.so
*** WARNING: renaming "_ssl" since importing it failed: libssl.so.1.0.0: cannot open shared object file: No such file or directory
building '_hashlib' extension
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/ssl/include -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/include/python2.7 -c /Python/Modules/_hashopenssl.c -o build/temp.linux-x86_64-2.7/Python/Modules/_hashopenssl.o
gcc -pthread -shared build/temp.linux-x86_64-2.7/Python/Modules/_hashopenssl.o -L/usr/local/ssl/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.linux-x86_64-2.7/_hashlib.so
*** WARNING: renaming "_hashlib" since importing it failed: libssl.so.1.0.0: cannot open shared object file: No such file or directory

Python build finished, but the necessary bits to build these modules were not found:
_bsddb             _sqlite3           _tkinter
bsddb185           dbm                dl
gdbm               imageop            sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


Failed to build these modules:
_hashlib           _ssl

running build_scripts

The new warning message in the output *** WARNING: renaming "_ssl" since importing it failed: libssl.so.1.0.0: cannot open shared object file: No such file or directory indicates that the file does not exist but I see it in /usr/local/ssl/lib/ as libssl.so.1.0.0 and can find it with a search

find / -name libssl.so.1.0.0
/usr/local/ssl/lib/libssl.so.1.0.0
/tmp/openssl-1.0.2l/libssl.so.1.0.0

Solution

Well, I don't know whether to feel stupid and this is just a common thing I completely overlooked doing since I am fairly new to linux CLI. But, I was able to resolve my build error for the _hashlib and _ssl modules with a couple simple steps in https://stackoverflow.com/a/28460293/8222554 amo's response.

I first attempted the ldconfig but that did not resolve my issue so I simply looked at what was in my LD_LIBRARY_PATH with

echo $LD_LIBRARY_PATH

and then got back these locations which did not include where the libssl.so.1.0.0 file was located

/usr/lib/oracle/11.2/client64/lib:/usr/lib/oracle/11.2/client/lib

Then I followed the steps I had done previously when installing the Oracle instant client and cx_Oracle module which were easily laid out on on this blog post: https://ubuntugeeknerd.blogspot.com/2013/08/how-to-install-oxoracle-in-rhel-64-bit.html

Those steps I used for the ssl module were as follow

export LD_LIBRARY_PATH=/usr/local/ssl/lib/:$LD_LIBRARY_PATH >> ~/.bashrc
echo $LD_LIBRARY_PATH
/usr/local/ssl/lib/:/usr/lib/oracle/11.2/client64/lib:/usr/lib/oracle/11.2/client/lib

After this I did python2.7 setup.py clean followed by python2.7 setup.py build and last python2.7 setup.py install. The output didn't return any failed to build besides the ones which I did not have the necessary bits for (and was not looking to build anyways).

I can verify the ssl module is there now when opening up python and importing the module successfully

Python 2.7.13 (default, Jun 26 2017, 15:21:33)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>>

Hopefully this will be handy to someone who is running into a similar issue

Comments

[Christopher Jones]

FYI cx_Oracle 6 (which is currently at RC 1 release status) has Wheels so any older install instructions are likely to be obsolete. Also if building it from source, you no longer need Oracle header files.

[João Paulo]

In my case, when building openssl, I needed to config it with the shared option (-fPIC).