Why can't I install multiple versions of a shared library?

Question

There are often instances where a certain program will depend on library version x.y and another on x.z but, as far as I'm aware, no package manager will allow me to install both x.y and x.z. Sometimes they will allow both major versions (such as qt4 and qt5, which can be installed at the same time), but (seemingly) never minor versions.

Why is this? As in, what is the limiting factor that prevents it? I assume there must be a good reason for not allowing this seemingly useful functionality. E.g., is there not a field to indicate what version to load when loading a shared object and thus no way for Linux to know how to decide which to load? Or is there really no reason for it? Like all minor versions are supposed to be compatible anyway or something?

Solution 1

Actually, you can install multiple versions of a shared library if it's done properly.

Shared libraries are usually named as follows:

lib<name>.so.<api-version>.<minor>

Next, there are symlinks to the library under the following names:

lib<name>.so
lib<name>.so.<api-version>

When a developer links against the library to produce a binary, it is the filename that ends in .so that the linker finds. There can indeed be only one of those installed at a time for any given <name> but that only means that a developer cannot target multiple different versions of a library at the same time. With package managers, this .so symlink is part of a separate -dev package which only developers need install.

When the linker finds a file with a name ending in .so and uses it, it looks inside that library for a field called soname. The soname advises the linker what filename to embed into the resulting binary and thus what filename will be sought at runtime. The soname is supposed to be set to lib<name>.so.<api-version>.

Therefore, at run time, the dynamic linker will seek lib<name>.so.<api-version> and use that.

The intention is that:

• <minor> upgrades don't change the API of the library and when the <minor> gets bumped to a higher version, it's safe to let all the binaries upgrade to the new version. Since the binaries are all seeking the library under the lib<name>.so.<api-version> name, which is a symlink to the latest installed lib<name>.so.<api-version>.<minor>, they get the upgrade.
• <api-version> upgrades change the API of the library, and it is not safe to let existing binary applications use the new version. In the case that the <api-version> is changed, since those applications are looking for the name lib<name>.so.<api-version> but with a different value for <api-version>, they will not pick up the new version.

Package managers don't often package more than one version of the same library within the same distribution version because the whole distribution, including all binaries that make use of the library, is usually compiled to use a consistent version of every library before the distribution is released. Making sure that everything is consistent and that everything in a distribution is compatible with everything else is a big part of the workload for distributors.

But you can easily end up with multiple versions of a library if you've upgraded your system from one version of your distritution to another and still have some older packages requiring older library versions. Example:

• libmysqlclient16 from an older Debian, contains libmysqlclient.so.16.0.0 and symlink libmysqlclient.so.16.
• libmysqlclient18 from current Debian, contains libmysqlclient.so.18.0.0 and symlink libmysqlclient.so.18.

Solution 2

This functionality is not disallowed, it is just not very common as a result of the way most libraries numbering work and because of the inconvenience of package name changes.

If the use a dotted version number scheme X.Y.Z. The "micro" version Z often changes on bugfixes, the "minor" number Y changes on backward compatible changes and the "major" version number X has to change on API changes (and sometimes does on major extra functionality).

There should never be a reason that you do not want to have the latest bugs fixed, and backward compatible changes should not break your software either.

If the library is developed that way you should always be able to replace X.Y.Z by X.(Y+m).(Z+n). for any given m and n. I.e. you should always be able to replace your library with the latest in the same major number series. And if the library developers are careful and the next major number is compatible (e.g. by the announcement to deprecate things, but not remove them yet) you can even use the next major number.

For package developers this means they can use the name with just one, or even no number name to give you the latest version by just updating the package. If they ship a library in a package abc2 then they have to go through hoops to move their own software that relied on abc2 to upgrade to use abc3, sometimes with transition packages. It is more convenient to leave out the major version number from a library if that works for most depending packages. So even if both abc2 and abc3 should be available at some point available in a distribution, abc3 is often called abc (just like abc2 was called when there was not abc3 yet), and as soon as no packages depend on abc2 within the distribution it becomes possible to drop abc2 altogether.

The numbering scheme is not uniformly followed, but I can only imagine that with the advent of the internet disseminating information on how to use such a scheme, and the pressure from library users (including distribution developers) to make important things like backwards compatibility clear without having to read through a CHANGES file included in the library, have contributed that this has become more common.

One counter example, but not of a library is the python intpreter, which is not compatible on its shared objects and pickling format on a minor number change. Therefore you will see packages for python (the latest in the 2.7 series), and python3 (the latest in currently the python3.4 series) as well as explicit packages for python 2.6 (not getting less common) as well as python 3.3.