Symbol visibility and namespace

Question

I'm experimenting with C++ symbol visibility on Linux and gcc. It seems that the preferred way is to use -fvisibility=hidden, and export used symbols one-by-one according to Visibility gcc wiki page (http://gcc.gnu.org/wiki/Visibility). My problem is that many libraries does not handle this well, they forget to explicitly export symbols, which is a serious problem. After several fixed bugs even some parts of boost may still be affected. Of course those bugs should be fixed, but till that I would like to use a "safe" way to hide as much as symbols as possible.

I came up with a solution: I place all the symbols in a namespace and I use symbol hide attribute on that and export the public interface, this way only my symbols can be affected.

The problem is that I got a warning message when I compile something against that library for every class that I haven't exported and I use in the application as class field.

namespace MyDSO __attribute__ ((visibility ("hidden"))) {
  struct Foo {
    void bar() __attribute__ ((visibility ("default"))) {}
  };
}

struct Bar {
  MyDSO::Foo foo;
};

int main() {}

The warning message can be reproduced in this small example, but of course the namespace should be in a library the other class in the application.

$ gcc-4.7.1 namespace.cpp -o namespace
namespace.cpp:7:8: warning: ‘Bar’ declared with greater visibility than the type of its field ‘Bar::foo’ [-Wattributes]

As I understand symbol visibility, hiding namespace should have quite similar effect to using -fvisibility=hidden, but I never got similar warnings using the latter. I see that when I pass -fvisibility=hidden to the application the class in the application will also be hidden, so I won't get a warning. But when I don't pass the option none of the symbols in the headers will seem hidden to the compiler, so I won't get a warning again.

What is the propose of this warning message? Is it a serious problem? In which situations can this cause any problem? How hiding namespace is different to fvisibility=hidden?

Solution 1

Before I answer your specific question, I should mention for others reading that applying symbol visibility attributes per namespace is a GCC-specific feature. MSVC only supports dllexport on classes, functions and variables, and if you want your code to be portable you must match MSVC there. As my original GCC symbol visibility guide (the one you linked to on the GCC website) points out, MSVC's macro based dllexport machinery can be easily reused to achieve something similar on GCC, so porting to MSVC will get you symbol visibility handling "for free".

Regarding your specific problem, GCC is correct to warn you. If an external user tried to use public type Bar they almost certainly need to use everything inside Bar, including Bar::foo. For the exact same reason all private member functions, despite being private, need to be visible. A lot of people are surprised at this, reasoning that private member function symbols are by definition inaccessible to anyone, but they forget that just because the programmer doesn't have access doesn't mean that the compiler doesn't need access. In other words, private member functions are private to you, but not the compiler. If they appear in a header file, that generally means the compiler needs access, even in an anonymous namespace (which are only anonymous to programmers, not to compilers which tend to use a hash of the contents as the "real" namespace name).

Hiding a namespace has very different effects to -fvisibility=hidden. This is because GCC spews out many symbols above and beyond those for a specific type e.g. for vtables, for type_info etc. -fvisibility=hidden hides stuff you can't hide by any compiler instructed way, and it's stuff absolutely essential to loading two binaries into the same process with colliding symbols e.g. two shared objects built using different versions of Boost.

I appreciate your attempts to fix the problems caused by broken symbol visibility in ELF and the consequences on broken C++ binaries and much lost programmer productivity. However you can't fix them - they are faults in ELF itself, which was designed for C and not C++. If it's any consolation, I wrote an internal BlackBerry white paper a few months ago on this topic as ELF symbol visibility problems are just as much a problem for us in BB10 as they are for any large corporation with a significant C++ codebase. So, maybe you might see some solutions proposed for C++17, especially if Doug Gregor's C++ Modules implementation makes good progress.

Solution 2

Your use of the visibility attributes seems backwards to me; I think you would have better results using -fvisibility=hidden and adding visibility "default" to the library declaration namespace, since the interface of the library presumably has default visibility or you couldn't use it from your application. If you don't want to modify the library headers, you could use #pragma GCC visibility push/pop around your #includes.

Also, as Niall says, marking individual member functions as default doesn't work, the whole Foo type needs to have default visibility if it's part of the interface of the library.

Comments

[VargaD]

Thanks for your detailed answer. Just for curiosity, I'm interested in what symbols does the compiler need that cannot be generated? As far as I know in trivial classes even typeinfo can be generated. When using vis=hidden you won't get warning even when you hide symbols that shouldn't be hidden, you just get undefined symbol error from linker. Using hidden namespace gcc can detect the problem. Maybe there are legitimate use to only export some symbols in a class, but gcc emits warning anyway. Doug Gregor's C++ Modules is very interesting, I liked his presentation, thank you for sharing it.

[Niall Douglas]

In short, typeinfo for any class with a virtual is always emitted, whereas typeinfo for classes without is only emitted when typeid() or something which uses it (exception catches, dynamic_cast<> etc) is used. Also, most compilers emit two or more constructor implementations per program specified constructor, and there is some magic in destructor generation too. In short, -fvisibility=hidden hides the lot, and your method would only hide the programmer specified stuff and not the magic internals. Much of this begins to get easier with C++ Modules, though not at all solved. Niall

[VargaD]

Thanks for your reply. I don't see why the whole class needs to be exported. Most of the classes don't even have a vtable. I many a few test applications, even tried to throw not exported exception, but all of them worked pretty well. I try to understand why it needs to be exported. Do you know in which circumstances it fails? Can you show an example?

[Jason Merrill]

The basic issue is that in C++, classes have linkage; if a class has hidden visibility, that means that you aren't planning to use it outside of its own DSO. And so giving a member function greater visibility than its class doesn't make sense, since you need to use the class in order to use the member function. Why don't you want to export the class itself?

[VargaD]

My problem is that exporting the whole class isn't maintainable, symbols easily get exported. In C using -fvisibility-hidden you just export symbol you want to export, but in C++ you have to export the whole class and difficult to hide all the symbols you don't want to export. I see that classes that have vtable must be exported. I have read many times that in C++ symbol visibility can cause a lot of problem, and I shouldn't hide classes that have exported methods, but I can't see why. I tried hard to produce runtime errors because of visibility, but I couldn't.

[Admin]

@VargaD maybe late but I think you should try the pimpl method, it is described well in the book API Design for C++ by Martin Reddy.