What does "Symbol not found / Expected in: flat namespace" actually mean?

Question

When I import a module I built, I get this boost-python related error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: dlopen(./myMod.so, 2): Symbol not found: __ZN5boost6python7objects15function_objectERKNS1_11py_functionERKSt4pairIPKNS0_6detail7keywordES9_E
  Referenced from: ./myMod.so
  Expected in: flat namespace
 in ./myMod.so

What does this actually mean? Why was this error raised?

Solution 1

Description

The problem was caused by mixing objects that compiled with libc++ and object that compiled with libstdc++.

In our case, the library myMod.so (compiled with libstdc++) need boost-python that compiled with libstdc++ (boost-python-libstdc++ from now). When boost-python is boost-python-libstdc++, it will work fine. Otherwise - on computer that its boost-python has compiled with libc++ (or another c++ library), it will have a problem loading and running it.

In our case, it happens because that libc++ developers intentionally changed the name of all of their symbols to prevent you (and save you) from mixing code from their library and code from a different one: myMod.so need a function that take an argument from the type. In libc++, this type's name is std::__1::pair. Therefore, this symbol was not found.

To understand why mixing two version of the same API is bad, consider this situation: There are two libraries: Foo and Bar. They both have a function that takes a std::string and uses it for something but they use a different c++ library. When a std::string that has been created by Foo will be passed to Bar, Bar will think that this is an instance of its c++ library's std::string and then bad things can happen (they are a completely different objects).

Note: In some cases, there would be no problem with two or more different versions of the same API in a completely different parts of a program. There will be a problem if they will pass this API's objects between them. However, checking that can be very hard, especially if they pass the API object only as a member of another object. Also, a library's initialization function can do things that should not happen twice. Another version may do these things again.

How to solve that?

• You can always recompile your libraries and make them match each other.

• You can link boost-python to your library as a static library. Then, it will work on almost every computer (even one that doesn't has boost-python installed). See more about that here.

Summary

myMod.so need another version of boost-python, one that compiled with a specific c++ library. Therefore, It would not work with any another version.

Solution 2

Symbol not found means the definition of the declared function or variable was not found. When a header file of a shared object is compiled with your program, linker adds symbols of declared functions and objects to your compiled program. When your program is loaded by the OS's loader, the symbols are resolved so that their definition will be loaded. It is only at this time where if the implementation is missing, loader complains it couldn't find the definition due to may be failing to resolve the actual path to the library or the library itself wasn't compiled with the implementation/source file where the definition of the function or object resides. There is a good article on this on the linux journal http://www.linuxjournal.com/article/6463.

Solution 3

In my case I was receiving:

ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/xmlsec.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '_xmlSecDSigNs'

BACKGROUND:

M1 MacBook Pro with Montery

I was working with a python virtualenv (using pyenv) to use an earlier version of python3.8 (3.8.2), while my system had 3.8.10 installed natively.

While I was in the activated 3.8.2 virtualenv I noticed the path in dlopen() was pointing to the package in the native python install NOT the virtualenv install.

SOLUTION:

In my case, I did not need the native 3.8 version at all so I simply removed it and this solved the problem.

Solution 4

I encounter the same problem.

Expected in: flat namespace

Add the linker flag fixes the problem

-lboost_python37

change the dynamic library name to the one installed on the os.

By the way, my os is macOS High Sierra and I use brew to install boost_python3.

Solution 5

In my case I was just failing to import all the required sources (c++ files) when compiling with Cython.

From the string after "Symbol not found" you can understand which library you are missing.

View more solutions

Comments

[sorin]

I am facing a similar issue with pyenv compiled pythons on Mojave and I really want to make pyenv to work instead of having to deal with py version manangement in a different way. See github.com/pyenv/pyenv/issues/1242 -- Any hints would be appreciated!

[Yu Tao]

it is boost-python3 not boost_python3

[ComputerScientist]

Can you describe how the linker flag should be added?