Hello world with boost python and python 3.2

Question

So I'm trying to interface python 3.2 and c++ using boost python, and have come across many many issues. I've finally gotten it to compile using the 2.7 libraries and it works, but I can't seem to make it work with python 3.2.

Here's the c++ code

#include <iostream>

using namespace std;

void say_hello(const char* name) {
    cout << "Hello " <<  name << "!\n";
}

int main(){return 0;}

#include <boost/python/module.hpp>
#include <boost/python/def.hpp>
using namespace boost::python;

BOOST_PYTHON_MODULE(hello)
{
    def("say_hello", say_hello);
}

If I compile it using the 2.7 libraries it works just fine, but when I use the 3.2 libraries I get tons of undefined references from libboost_python.so

Otherwise I wrote a little bit of python to make it work:

from distutils.core import setup
from distutils.extension import Extension

setup(name="PackageName",
    ext_modules=[
        Extension("hello", ["testBoost.cpp"],
        libraries = ["boost_python"])
    ])

and this will create an so using python 3.2 or 2.7 build, but when I open the python 3 interpreter and attempt to import the so it give me the error undefined symbol PyClass_Type from libboost_python.so again. Any ideas? Is boost python compatible with python 3.x?

If the information is useful, here is my attempted compile using 3.2:

   $ g++ testBoost.cpp -I/usr/include/python3.2 -I/usr/local/include/boost/python -lboost_python -lpython3.2mu 
    /tmp/ccdmU1Yu.o: In function `PyInit_hello':
    testBoost.cpp:(.text+0xc2): undefined reference to `boost::python::detail::init_module(PyModuleDef&, void (*)())'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_Size'
    /usr/local/lib/libboost_python.so: undefined reference to `PyFile_FromString'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_Type'
    /usr/local/lib/libboost_python.so: undefined reference to `PyInt_Type'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_FromString'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_FromStringAndSize'
    /usr/local/lib/libboost_python.so: undefined reference to `Py_InitModule4_64'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_FromFormat'
    /usr/local/lib/libboost_python.so: undefined reference to `PyNumber_Divide'
    /usr/local/lib/libboost_python.so: undefined reference to `PyNumber_InPlaceDivide'
    /usr/local/lib/libboost_python.so: undefined reference to `PyInt_AsLong'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_InternFromString'
    /usr/local/lib/libboost_python.so: undefined reference to `PyClass_Type'
    /usr/local/lib/libboost_python.so: undefined reference to `PyString_AsString'
    /usr/local/lib/libboost_python.so: undefined reference to `PyInt_FromLong'
    /usr/local/lib/libboost_python.so: undefined reference to `PyFile_AsFile'
    collect2: ld returned 1 exit status

And the error from the python 3 interpreter is

File "<stdin>", line 1, in <module>
ImportError: /usr/local/lib/libboost_python.so.1.47.0: undefined symbol: PyClass_Type

Thanks for any help!

Solution 1

Although this discussion old, just for the record: Modify project-config.jam to change the python version to your setup

# Python configuration
using python : 3.4 : /usr ;

Then build boost:

./b2 clean
./b2 --with-python link=static cxxflags="-std=c++11 -fPIC" variant=release stage
./b2 --with-python link=static cxxflags="-std=c++11 -fPIC" variant=release install

The later command requires super user privileges. Then move to the folder containing C++ code for the extension:

g++ -std=c++11 hellopy.cpp -I/usr/include/python3.4 -I/usr/local/include/boost/python -lboost_python3  -o hello.so -shared -fPIC

You can then import hello into your python environment.

Solution 2

I had the exact same problem, with Ubuntu 12.04. I installed the 1.48 version of the library and had to link with libboost_python-py32.so instead of libboost_python.so After this the linker errors was gone.

Solution 3

The above c++ code compiles into a module with

$ g++ testBoost.cpp -I/usr/include/python3.2 -I/usr/local/include/boost/python -lboost_python3 -lpython3.2mu -o hello.so -shared

This compile command adds -lboost_python3, and -shared, and also the naming convention for python extension modules. You should also install the python3-dev package, and configure/build/install boost with python3, if you haven't already.

In python 3, I can then do the following:

$ python3
Python 3.2 (r32:88445, Mar 25 2011, 19:28:28) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import hello
>>> hello.say_hello('bill')
Hello bill!
>>>

You should be off to the races at that point.

Solution 4

Linking in the python libraries (for e.g. -L/usr/lib/x86_64-linux-gnu -lpython2.7 on linux or find_package(PythonLibs) in CMake¹) will make this linker issue go away.

Here is a more detailed explanation of the issue below. On the command line,

$ nm --dynamic <path-to>/libboost_python.so | grep PyString_Size

If you are feeling lazy and assuming your libboost_python is linking to python2.7, just run this

$ nm --dynamic `locate libboost_python27.so | awk 'NR==1'` | grep PyString_Size

You should see something like

U PyString_Size

So PyString_Size is undefined (U) in libboost_python27.so. This is what the linker was complaining about. We've confirmed that. Now let's look for this symbol in libpython.

$ nm --dynamic `locate libpython2.7.so | awk 'NR==1'` | grep PyString_Size

On my machine, I saw something like this:

00000000000f0530 T PyString_Size

The T indicates that the text for this symbol is at the address indicated. So this is proof that we were not linking in libpython in addition to libboost_python.

¹ Why aren't you using CMake? :)

View more solutions

Comments

[Dwight]

I tried using -lboost_python3 but still get the same error, and I've already changed my jam file to point to python 3

[Dwight]

Okay so I compiled with your instructions, and it told me I needed to recompile with -fpic, which worked, and I ended up with the .so. Now however, when I try to import in python, I get an error that says: ImportError: /usr/local/lib/libboost_python3.so.1.47.0: undefined symbol: PyClass_Type

[droptop]

Have any idea on how to get libboost_python to link statically with libpython during a boost build ??