Get ld to pick the correct library

Question

I'm trying to compile a program prog and link it against OpenSSL's 1.0.2 beta, built from source and installed in /usr/local/ssl-1.0.2. On an older system using 0.9.8, this works without too much trouble. On a more recent system with 1.0.1 installed, this requires a bit more work. I'm wondering why.

1) On Ubuntu 10.04, with OpenSSL 0.9.8:

Here are the steps I follow to compile and link against 1.0.2.

$ ./config shared --openssldir=/usr/local/ssl-1.0.2 && make && make install
$ ldconfig
$ ldconfig -p | grep libcrypto

=> Only 0.9.8 files show up, so I add the path to the 1.0.2 files...

$ ldconfig /usr/local/ssl-1.0.2/lib
$ ldconfig -p | grep libcrypto

=>

 libcrypto.so.1.0.0 (libc6) => /usr/local/ssl-1.0.2/lib/libcrypto.so.1.0.0
 libcrypto.so.0.9.8 (libc6, hwcap: 0x0008000000008000) => /lib/i686/cmov/libcrypto.so.0.9.8
 libcrypto.so.0.9.8 (libc6, hwcap: 0x0004000000000000) => /lib/i586/libcrypto.so.0.9.8
 libcrypto.so.0.9.8 (libc6, hwcap: 0x0002000000000000) => /lib/i486/libcrypto.so.0.9.8
 libcrypto.so.0.9.8 (libc6) => /lib/libcrypto.so.0.9.8
 libcrypto.so.0.9.8 (libc6) => /usr/lib/libcrypto.so.0.9.8
 libcrypto.so (libc6) => /usr/local/ssl-1.0.2/lib/libcrypto.so

And so I can compile prog...

$ gcc -o prog ... -L/usr/local/ssl-1.0.2/lib -lcrypto
$ ldd prog

=>

    libcrypto.so.1.0.0 => /usr/local/ssl-1.0.2/lib/libcrypto.so.1.0.0 (0x0083b000)

... and it is correctly linked against 1.0.2.

2) On Debian Wheezy, with OpenSSL 1.0.1:

Same steps, different result.

$ ./config shared --openssldir=/usr/local/ssl-1.0.2 && make && make install
$ ldconfig
$ ldconfig -p | grep libcrypto

=>

 libcrypto.so.1.0.0 (libc6, hwcap: 0x0008000000008000) => /usr/lib/i386-linux-gnu/i686/cmov/libcrypto.so.1.0.0
 libcrypto.so.1.0.0 (libc6, hwcap: 0x0004000000000000) => /usr/lib/i386-linux-gnu/i586/libcrypto.so.1.0.0
 libcrypto.so.1.0.0 (libc6) => /usr/lib/i386-linux-gnu/libcrypto.so.1.0.0

Likewise, I add the path to 1.0.2...

$ ldconfig /usr/local/ssl-1.0.2/lib
$ ldconfig -p | grep libcrypto

=>

 libcrypto.so.1.0.0 (libc6, hwcap: 0x0008000000008000) => /usr/lib/i386-linux-gnu/i686/cmov/libcrypto.so.1.0.0
 libcrypto.so.1.0.0 (libc6, hwcap: 0x0004000000000000) => /usr/lib/i386-linux-gnu/i586/libcrypto.so.1.0.0
 libcrypto.so.1.0.0 (libc6) => /usr/local/ssl-1.0.2/lib/libcrypto.so.1.0.0
 libcrypto.so.1.0.0 (libc6) => /usr/lib/i386-linux-gnu/libcrypto.so.1.0.0
 libcrypto.so (libc6) => /usr/local/ssl-1.0.2/lib/libcrypto.so

Then I try to compile...

$ gcc -o prog ... -L/usr/local/ssl-1.0.2/lib -lcrypto
$ ldd prog

=>

    libcrypto.so.1.0.0 => /usr/lib/i386-linux-gnu/i686/cmov/libcrypto.so.1.0.0 (0xb7591000)

But here it is not linked against 1.0.2. The compile-time library path is correct (specified with -L, gcc would fail otherwise since some functions used in prog are specific to 1.0.2), but not the run-time one.

3) How to get it working on Wheezy

With or without running ldconfig /usr/local/ssl-1.0.2/lib:

$ gcc -o prog ... -Wl,--rpath=/usr/local/ssl-1.0.2/lib -L/usr/local/ssl-1.0.2/lib -lcrypto
$ ldd prog

=>

    libcrypto.so.1.0.0 => /usr/local/ssl-1.0.2/lib/libcrypto.so.1.0.0 (0xb7592000)

Alternatively, run export LD_LIBRARY_PATH=/usr/local/ssl-1.0.2/lib before running gcc.

What I'd like to know

Using LD_DEBUG=libs ./prog as suggested by mr.spuratic, I found that the paths were looked up in /etc/ld.so.cache. I opened that file and found that the order in which .so are looked up corresponds to the output of ldconfig -p.

So the actual question is:

• Why does the 1.0.2 file gets on top of ldconfig's list in 1) but not in 2) ? Pure randomness? Confusion due to 1.0.1 and 1.0.2 files having the same suffix? ("1.0.0")

Or, said differently,

• Why are the flags added in 3) not necessary in 1) ?

Solution

There are three things you need to take care of when compiling/linking against a non-default package:

• headers (usually CFLAGS)
• compile-time library path (usually LDFLAGS)
• run-time library path (rpath via LDFLAGS, LD_RUN_PATH, LD_LIBRARY_PATH or ld.so.conf)

You haven't said what prog is, so I can't how well-behaved its configuration might be (or if it uses autoconf?), I've seen many that only perform the first two steps reliably.

During the link stage the library path order is relevant, assuming you're using the GNU toolchain (gcc & binutils) you can probably see what's going on by setting CFLAGS before configure (or possible in the Makefile directly):

export CFLAGS="-Wl,-t"

This passes the -t trace option to the linker. You may need to add V=1 or VERBOSE=1 to the make command if you only get terse "CC" and "LD" lines output during the make.)

At run time you can see what ld.so tries by carefully setting LD_DEBUG, e.g.

LD_DEBUG=libs ./myprog

(or try values of files or symbols for more detail)

To specify all three parameters correctly at build time you should be able to do:

• export CFLAGS="-I/usr/local/ssl-1.0.2/include"
• export LDFLAGS="-L/usr/local/ssl-1.0.2/lib -R/usr/local/ssl-1.0.2/lib"

then reconfigure/recompile.

You're using --openssldir rather than the more conventional --prefix (I recommend the latter and also using just make install_sw if you don't need the 1000 or so man pages & symlinks a default install gives you). This may be part of the problem. For some reason the .so libraries that you show are known to ld.so do not have a version suffix (e.g. .so.1.0.2), a proper "make install" should have set that up for you (via the link-shared target in the main Makefile).

The -R option instructs the linker to embed an RPATH in the executable output for the specific OpenSSL library so that it does not need to rely on the default that the runtime linker (ld.so) would normally provide. You can modify existing binaries with chrpath instead.

This is more or less equivalent to exporting LD_LIBRARY_PATH=/usr/local/ssl-1.0.2/lib. You can read more about RPATH and the related RUNPATH here: http://blog.tremily.us/posts/rpath/

As a last resort you could possibly build OpenSSL without "shared" or with "noshared", this will give you static libraries which won't have this problem (but may well have other problems, e.g. for use within ELF .so, causing PIC/PIE problems)

---

Based on the updated details, I believe the problem is that 1.0.1 and 1.0.2beta both set the .so version suffix (SONAME) to 1.0.0. On the first system with only 0.9.8 this causes no problems; on the second with 1.0.1 and 1.0.2 both versioned as 1.0.0, it's "first match wins" based on the ld.so.{conf,d} ordering. Remember, ld the compile time linker is a different program to ld.so the run-time linker, and can have different behaviour (usually resulting in symbol errors or worse, as you have seen).

$ cd /usr/local/src/openssl/openssl/1.0.2beta1
$ readelf -a libssl.so | grep SONAME
0x0000000e (SONAME)                     Library soname: [libssl.so.1.0.0]

$ cat verchk.c
int main(int argc, char *argv[]) {
    printf("build: %s\n",OPENSSL_VERSION_TEXT);
    printf("run  : %s\n",SSLeay_version(SSLEAY_VERSION));
    return 0;
}

$ gcc -Wall  -I/usr/local/src/openssl/openssl-1.0.2-beta1/include \
    -Wl,-rpath /usr/local/src/openssl/openssl-1.0.2-beta1/ \
    -o verchk /usr/local/src/openssl/openssl-1.0.2-beta1/libcrypto.so verchk.c

$ ./verchk
build: OpenSSL 1.0.2-beta1 24 Feb 2014
run  : OpenSSL 1.0.2-beta1 24 Feb 2014

$ grep SHLIB_M...R= Makefile
SHLIB_MAJOR=1
SHLIB_MINOR=0.0

---

Update OpenSSL-1.1 has made some API level changes, the above code will fail to compile with v1.1 headers and older libraries (undefined reference to `OpenSSL_version').

SSLeay_version() is now deprecated and (depending on OPENSSL_API_COMPAT) may be #define-d to the proper API function OpenSSL_version().

Comments

[Peniblec]

Thanks! Your post helped me dig a bit deeper. To answer your questions: prog is a small program built in two steps (gcc -c $(CCFLAGS), then gcc $(LDFLAGS)). The actual question is less about getting it to work and more about why 1.0.2 is not automatically picked up since the same steps are followed, i.e. why is it necessary to specify rpath in one case and not in the other. I've substantially edited the question to provide more info (well, at least more organized info).

[Peniblec]

@update: Thanks again, that was insightful. I'll try to set SHLIB_MINOR to 0.2, configure/make/install, and see if that changes things, just for kicks.