What are the differences between .so and .dylib on macOS?

Question

.dylib is the dynamic library extension on macOS, but it's never been clear to me when I can't / shouldn't use a traditional unix .so shared object.

Some of the questions I have:

• At a conceptual level, what are the main differences between .so and .dylib?
• When can/should I use one over the other?
• Compilation tricks & tips (For example, the replacement for gcc -shared -fPIC, since that doesn't work on osx)

Solution 1

The Mach-O object file format used by Mac OS X for executables and libraries distinguishes between shared libraries and dynamically loaded modules. Use otool -hv some_file to see the filetype of some_file.

Mach-O shared libraries have the file type MH_DYLIB and carry the extension .dylib. They can be linked against with the usual static linker flags, e.g. -lfoo for libfoo.dylib. They can be created by passing the -dynamiclib flag to the compiler. (-fPIC is the default and needn't be specified.)

Loadable modules are called "bundles" in Mach-O speak. They have the file type MH_BUNDLE. They can carry any extension; the extension .bundle is recommended by Apple, but most ported software uses .so for the sake of compatibility. Typically, you'll use bundles for plug-ins that extend an application; in such situations, the bundle will link against the application binary to gain access to the application’s exported API. They can be created by passing the -bundle flag to the compiler.

Both dylibs and bundles can be dynamically loaded using the dl APIs (e.g. dlopen, dlclose). It is not possible to link against bundles as if they were shared libraries. However, it is possible that a bundle is linked against real shared libraries; those will be loaded automatically when the bundle is loaded.

Historically, the differences were more significant. In Mac OS X 10.0, there was no way to dynamically load libraries. A set of dyld APIs (e.g. NSCreateObjectFileImageFromFile, NSLinkModule) were introduced with 10.1 to load and unload bundles, but they didn't work for dylibs. A dlopen compatibility library that worked with bundles was added in 10.3; in 10.4, dlopen was rewritten to be a native part of dyld and added support for loading (but not unloading) dylibs. Finally, 10.5 added support for using dlclose with dylibs and deprecated the dyld APIs.

On ELF systems like Linux, both use the same file format; any piece of shared code can be used as a library and for dynamic loading.

Finally, be aware that in Mac OS X, "bundle" can also refer to directories with a standardized structure that holds executable code and the resources used by that code. There is some conceptual overlap (particularly with "loadable bundles" like plugins, which generally contain executable code in the form of a Mach-O bundle), but they shouldn't be confused with Mach-O bundles discussed above.

Additional references:

• Fink Porting Guide, the basis for this answer (though pretty out of date, as it was written for Mac OS X 10.3).
• ld(1) and dlopen(3)
• Dynamic Library Programming Topics
• Mach-O Programming Topics

Solution 2

The difference between .dylib and .so on mac os x is how they are compiled. For .so files you use -shared and for .dylib you use -dynamiclib. Both .so and .dylib are interchangeable as dynamic library files and either have a type as DYLIB or BUNDLE. Heres the readout for different files showing this.

libtriangle.dylib:
Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64  X86_64        ALL  0x00       DYLIB    17       1368   NOUNDEFS DYLDLINK TWOLEVEL NO_REEXPORTED_DYLIBS



libtriangle.so:
Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64  X86_64        ALL  0x00       DYLIB    17       1256   NOUNDEFS DYLDLINK TWOLEVEL NO_REEXPORTED_DYLIBS

triangle.so:
Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64  X86_64        ALL  0x00      BUNDLE    16       1696   NOUNDEFS DYLDLINK TWOLEVEL

The reason the two are equivalent on Mac OS X is for backwards compatibility with other UNIX OS programs that compile to the .so file type.

Compilation notes: whether you compile a .so file or a .dylib file you need to insert the correct path into the dynamic library during the linking step. You do this by adding -install_name and the file path to the linking command. If you dont do this you will run into the problem seen in this post: Mac Dynamic Library Craziness (May be Fortran Only).

Solution 3

The file .so is not a UNIX file extension for shared library.

It just happens to be a common one.

Check line 3b at ArnaudRecipes sharedlib page

Basically .dylib is the mac file extension used to indicate a shared lib.

Solution 4

Just an observation I just made while building naive code on OSX with cmake:

cmake ... -DBUILD_SHARED_LIBS=OFF ...

creates .so files

while

cmake ... -DBUILD_SHARED_LIBS=ON ...

creates .dynlib files.

Perhaps this helps anyone.

View more solutions

Comments

[Martin York]

@ninefingers. Correct. But some of the tools will use default values unless something is very explicit. e.g. Compilers will use there platform specific shared libray extension when the -l<lib> flag is used (actual flag may very across compilers).

[Mikhail Edoshin]

Thank you for this extensive comment :) Do I understand it correctly, that if I load one bundle from another bundle (i.e. the path is app -> bundle A -> bundle B), then bundle B won't be able to see any symbols in bundle A? And if yes, are there any ways to somehow solve this? I've just hit it, I think: stackoverflow.com/questions/4193539/…

[Miles]

@noloader: -dynamiclib is a GCC flag. It makes the compiler pass -dylib to ld.

[netpoetica]

Updated URL for man page for ld on Mac OSX: manpages.info/macosx/ld.1.html

[Admia]

how can I make the ./configure to generate .dylib files rather than bundle files .so? ./configure --enable-shared does not do this task.

[Zachary Kraus]

from my experience most configuration files on mac will either build a .so file or a static library file because the configuration files are using standard unix / linux file names.

[KcFnMi]

Does this all mean a .so, no matter if was build on Mac or Linux, can be used in both Mac and Linux?