Failed to build these modules: _hashlib

Python Help
1

I’m trying to build Python 3.7.1 rc2 from source on Centos 6. I’ve built a local OpenSSL 1.0.2p in /usr/local/ssl, and configured Modules/Setup.dist to point to that directory.

 ./configure --prefix=/usr \
            --enable-optimizations \
            --with-lto 
            --enable-ipv6 \
            --with-ensurepip \
            --with-openssl=/usr/local/ssl \
            --with-computed-gotos \
            --enable-loadable-sqlite-extensions && \

However, _hashlib fails to build.

Failed to build these modules:
_hashlib

The custom installation of SSL is recognized.

[root@system bin]# ./python3
Python 3.7.1rc2 (default, Oct 15 2018, 17:16:47) 
[GCC 5.3.1 20160406 (Red Hat 5.3.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.0.2p  14 Aug 2018'

Interestingly, the hashlib module seems to work.

[root@system bin]# ./python3
Python 3.7.1rc2 (default, Oct 15 2018, 17:16:47) 
[GCC 5.3.1 20160406 (Red Hat 5.3.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> hashlib.sha256('the _hashlib module didnt build!'.encode('ascii')).hexdigest()
'4f61f9875aaac7e19e4a08f6bae49128cbdb8c9586ff22d41c5c6c9916fa6a97'

But I really wish I wouldn’t get that build error. Any tips to resolve it?

2

_hashlib is the OpenSSL implementations behind the hashlib module. In absense of it, slower built-in implementations will be used.

You’ll need to look in the build logs to see why compilation of _hashlib.so failed.

1 Like
3

I rebuilt and separated stderr output, which is:

/opt/rh/devtoolset-4/root/usr/libexec/gcc/x86_64-redhat-linux/5.3.1/ld: /usr/local/ssl/lib/libcrypto.a(o_names.o): relocation R_X86_64_32 against `.text' can not be used when making a shared object; recompile with -fPIC
/usr/local/ssl/lib/libcrypto.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
/opt/rh/devtoolset-4/root/usr/libexec/gcc/x86_64-redhat-linux/5.3.1/ld: /usr/local/ssl/lib/libcrypto.a(o_names.o): relocation R_X86_64_32 against `.text' can not be used when making a shared object; recompile with -fPIC
/usr/local/ssl/lib/libcrypto.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status

stdout log apropo _hashlib looks normal as far as I can tell (which isn’t very far!)

building '_hashlib' extension
gcc -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I/usr/local/ssl/include -I./Include -I. -I/usr/local/include -I/tmp/Python-3.7.1/Include -I/tmp/Python-3.7.1 -c /tmp/Python-3.7.1/Modules/_hashopenssl.c -o build/temp.linux-x86_64-3.7/tmp/Python-3.7.1/Modules/_hashopenssl.o
gcc -pthread -shared build/temp.linux-x86_64-3.7/tmp/Python-3.7.1/Modules/_hashopenssl.o -L/usr/local/ssl/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.linux-x86_64-3.7/_hashlib.cpython-37m-x86_64-linux-gnu.so

That was with openssl configured like so:

./config && make depend && make all && make install

Following the clue about recompiling with fPIC, if I build openssl with shared and link ldconfig to /usr/local/ssl/lib, then _hashlib builds successfully.

export CFLAGS=-fPIC
./config shared && make depend && make all && make install

Is that the correct approach?

4

Likely… That’s a very old OS and unique config as you are building your own openssl (wise). If it worked, linked, and _hashlib imports for you, yay!

1 Like
5

Yes, it did work, so I’m all set. Thanks!

6

This was the main thread containing the exact error message I encountered. After spending a full day on this issue, here is how it got resolved:

Use rpath to bake the library path into the executable so that it is not necessary to set the library path each time a Python executable is needed; e.g., pip, virtualenv, etc. Here is an example: LDFLAGS="${LDFLAGS} -Wl,-rpath=/usr/local/ssl/lib" CFLAGS="${CFLAGS} -I/usr/local/ssl/include" ./configure --prefix=/home/lawlist/opt/python_3_9_13 --with-openssl=/usr/local/ssl The key ingredients to a successful Python build and to avoid a _hashlib failure when building same were to use the shared flag when building openssl and use the --with-openssl flag when building Python.

1 Like
7

The key ingredients to a successful Python build and to avoid a _hashlib failure when building same were to use the shared flag when building openssl and use the --with-openssl flag when building Python.

This resolved my issue. Specifically, I was already using --with-openssl when building Python, and I just needed to set the shared flag when building OpenSSL. Thanks!

1 Like