libboost_python3.so.1.56.0: undefined symbol: PyClass_Type

Question

I'm trying to create a helloWorld module for Python3 in C++ using boost::python library.

Here is a CmakeList.txt:

set(Python_ADDITIONAL_VERSIONS 3.4)
find_package( PythonLibs 3.4 REQUIRED )
include_directories( ${PYTHON_INCLUDE_DIRS} )

find_package( Boost 1.56.0 EXACT COMPONENTS python3 REQUIRED )
include_directories( ${Boost_INCLUDE_DIR} )

# Define the wrapper library that wraps our library
add_library( hello SHARED main.cpp )
target_link_libraries( hello ${Boost_LIBRARIES} ${PythonLibs_LIBRARIES} )

# don't prepend wrapper library name with lib
set_target_properties( hello PROPERTIES PREFIX "" OUTPUT_NAME hello)

main.cpp

#include <boost/python.hpp>

char const* greet( )
{
    return "Hello world";
}    
BOOST_PYTHON_MODULE(mymodule)
{
    using namespace boost::python;
    def( "greet", greet );
}

I installed boost libraries from source like described here, but it does not allow me to use boost-python3 library (have an error in Cmake). For this purpose I used

./bootstrap.sh --with-python-version=3.4 --prefix=/usr/local

instead of

./bootstrap.sh --prefix=/usr/local

to explicitly specify version of python;

As an output we get a shared library hello.so. All seems to be ok. But...

When I try to import the library to python script sript.py with content:

import hello

in terminal using command ...$ python3 script.py

I receive an error

Traceback (most recent call last):
  File "script.py", line 1, in <module>
    import hello 
ImportError: /usr/local/lib/libboost_python3.so.1.56.0: undefined symbol: PyClass_Type

The question is: How to make boost library compatible with python3? There are no problems with python2. But I need python3. I also saw the page when the same error happens but it didn't help me.

My software:

• boost version 1.56.0
• pyhton 3.4
• cmake version 2.8.12.2
• gcc 4.8.2
• OS: Ubuntu 14.04 LTS, 64 bit

Solution

As noted in this answer:

PyClass_Type is is part of the Python 2 C API and not part of the Python 3 C API. Hence, the Boost.Python library was likely built against Python 2. However, it is being loaded by a Python 3 interpreter, where the PyClass_Type is not available.

The exact procedure used to produce libboost_python3.so is not presented, so I can only speculate a non clean build, such as building Boost.Python with Python2, then reconfiguring bootstrap with Python3, and then building Boost.Python with the Python2 object files. Regardless, verify a clean build of Boost.Python with Python3.

$ ./bootstrap.sh --with-python=/usr/bin/python2
...
Detecting Python version... 2.7
$ ./b2 --with-python --buildid=2 # produces libboost_python-2.so
$ ./bootstrap.sh --with-python=/usr/bin/python3 --with-python-root=/usr
...
Detecting Python version... 3.3
$ ./b2 --with-python --buildid=3noclean # produces libboost_python-3noclean.so
$ ./b2 --with-python --clean
$ ./b2 --with-python --buildid=3 # produces libboost_python-3.so

$ nm -D stage/lib/libboost_python-2.so | grep PyClass_Type
                 U PyClass_Type
$ nm -D stage/lib/libboost_python-3noclean.so | grep PyClass_Type
                 U PyClass_Type
$ nm -D stage/lib/libboost_python-3.so | grep PyClass_Type

As expected, libboost_python-2.so references the PyClass_Type symbol. Additionally, the libboost_python-3noclean.so contains a reference to PyClass_Type as it was built with libboost_python-2.so's object files. With a clean build, libboost_python-3.so should not contain a reference to PyClass_Type.

Comments

[Fraser]

Shouldn't ${PythonLibs_LIBRARIES} be ${PYTHON_LIBRARIES}?

[Ivan_Bereziuk]

Error is the same after I replaced ${PythonLibs_LIBRARIES} into ${PYTHON_LIBRARIES}

[Alex Constantin]

Have you tried the solution mentioned here ? stackoverflow.com/questions/19865757/…

[Ivan_Bereziuk]

Yes, I did. (See the last 2 sentences, and link attached to the word page)

[Ivan_Bereziuk]

After your build, how should I modify CmakeList.txt for usage this libraries placed in these certain directories? Because the error is the same, after I rebuild project with including additional directories to this libraries. Nothing was changed in Cmake output.

[Ivan_Bereziuk]

I also tried to include *.so file manually: specifying path to new libboost_python-3.so instead of ${Boost_LIBRARIES} in CMakeList.txt file: target_link_libraries( hello "/home/john/soft/boost/boost_1_56_0/stage/lib/libboost_pytho‌​n-3.so" ${PythonLibs_LIBRARIES} ). yield is new another error: ImportError: dynamic module does not define init function (PyInit_hello)

[Tanner Sansbury]

@IvanBereziuk The library and module names must match. Either build the library as mymodule.so or name the module hello by using BOOST_PYTHON_MODULE(hello).

[Ivan_Bereziuk]

Yay!!! It works(in case I include *.so manually). But this is not the best solution to the problem (to prescribe the path to the libraries by hand). What to do when you need to use multiple boost libraries? How to integrate our library into common ${Boost_LIBRARIES}?

[Ivan_Bereziuk]

Yes, I found the solution. Somehow the new boos::python3 library have name libboost_python-3.so(for comparison to the old libboost_python3.so. So I changed python3 on python-3 and module began importing without errors.

[Tanner Sansbury]

@IvanBereziuk The naming difference is the result of buildid argument provided to b2. I was using those in the example to force different versions of the library so that I could demonstrate the contaminated build.

[ara.hayrabedian]

@TannerSansbury - thank you so much almost 3 years later! your example of producing a clean build worked so much better than anything else i found online. i think a lot of sources don't differentiate between config at the bootstrap and b2 stages.

[bartolo-otrit]

I have to remove bin.v2/libs/python directory in order to force making new *.so files.