Set LD_LIBRARY_PATH correctly

environment-variables libraries matlab boost

32,189

Solution 1

The only way it worked for me is by defining LD_PRELOAD and point it to system library. So my current environment would be:

LD_LIBRARY_PATH=/usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/extern/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/opengl/lib/glnxa64
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libexpat.so

So in docker, I have created a .matlab file that contains the mentioned env variables. Then, in dockerfile I added the following:

services:
  django: &django
    build:
      context: .
      dockerfile: ./compose/local/django/Dockerfile
    depends_on:
      - postgres
    volumes:
      - .:/app
      - shared_upload_tmp:/app_temp
      - media:/app/computation_server/media
    env_file:
      - ./.envs/.local/.django
      - ./.envs/.local/.postgres
      - ./.envs/.production/.matlab
    ports:
      - "8800:8000"
    command: /start

Solution 2

You have a couple of options:

Prefix the path to your shared libraries in the export statement above (more than likely $LD_LIBRARY_PATH is not set on your system and so the only paths that are getting set are the ones explicitly set in your export statement.)
```
 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/:/usr/lib/:/usr/lib64/:/usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/extern/bin/glnxa64
```
This export statement will not be a permanent part of your shell sessions unless you add it to .bashrc fyi.
Add the MATLAB library path to /etc/ld.so.conf.d/ and run ldconfig to identify the new paths for the linker on your system.

(BTW I am guessing at the MATLAB library name. You might need to make the name of the file something other than MATLAB like all lower case or something else. Basically we need the name of the library as it is referenced during the linking process. You could inspect a make file to see how this is referenced.)
```
 # Add the file '/etc/ld.so.conf.d/MATLAB' with the following lines 
 /usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64
 /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64
 /usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64
 /usr/local/MATLAB/MATLAB_Runtime/v94/extern/bin/glnxa64
```
Then run ldconfig.
```
 sudo ldconfig
```
You may need to unset the $LD_LIBRARY_PATH variable to make the linker use the normal system search path and pickup the new search path for MATLAB.
```
 unset LD_LIBRARY_PATH
```

32,189

Coderji

Updated on September 18, 2022

Comments

Coderji over 1 year
I am using Docker with image of Ubuntu 16.04. I am using this docker to run:
- Import C++ classes in python with the help of boost.python
- Import matlab's python package in python with the help Matlab Runtime (MCR).
Before I have installed MCR in the docker, I have installed boost using:
```
$ sudo apt-get install -y libboost-all-dev
```
and I was able to wrap C++ classes and call them in python. Then I have installed MCR and one of the requirement is to set LD_LIBRARY_PATH to Matlab's library.
```
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/extern/bin/glnxa64
```
It worked, I was able to run matlab's python package in python but testing the C++ classes wrappers failed due to change of LD_LIBRARY_PATH. For example, I get the following error when I try to import C++ class in python:
```
 /usr/lib/x86_64-linux-gnu/libpython3.4m.so.1.0: undefined symbol: XML_SetHashSalt
```
If I unset LD_LIBRARY_PATH C++ import works but MCR fails. So the question what LD_LIBRARY_PATH should be in order to run both C++ classes and Matlab's python package successfully?

I have tried to add path to boost libraries to LD_LIBRARY_PATH but didn't work, so the env value would be:
```
/usr/lib/x86_64-linux-gnu:/usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/extern/bin/glnxa64
```
EDIT

the dependencies of C++ library file before setting the LD_LIBRARY_PATH:
```
    linux-vdso.so.1 (0x00007ffcee0dc000)
    libpython3.4m.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython3.4m.so.1.0 (0x00007f9d69e59000)
    libboost_python-py34.so.1.55.0 => /usr/lib/x86_64-linux-gnu/libboost_python-py34.so.1.55.0 (0x00007f9d69c09000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f9d698fe000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9d695fd000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9d693e7000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9d6903c000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f9d68e34000)
    libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f9d68c0b000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f9d689f0000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9d687d3000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9d685cf000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f9d683cc000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f9d6a6c3000)
```
After setting the LD_LIBRARY_PATH:
```
    linux-vdso.so.1 (0x00007ffc42e9b000)
    libpython3.4m.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython3.4m.so.1.0 (0x00007fad9635b000)
    libboost_python-py34.so.1.55.0 => /usr/lib/x86_64-linux-gnu/libboost_python-py34.so.1.55.0 (0x00007fad9610b000)
    libstdc++.so.6 => /usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64/libstdc++.so.6 (0x00007fad95d8a000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fad95a89000)
    libgcc_s.so.1 => /usr/local/MATLAB/MATLAB_Runtime/v94/sys/os/glnxa64/libgcc_s.so.1 (0x00007fad95873000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fad954c8000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fad952c0000)
    libexpat.so.1 => /usr/local/MATLAB/MATLAB_Runtime/v94/bin/glnxa64/libexpat.so.1 (0x00007fad95095000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fad94e7a000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fad94c5d000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fad94a59000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fad94856000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fad96bc5000)
```
- schily over 5 years
  
  LD_LIBRARY_PATH should be avoided in favor of binaries that set the RUNPATH internally in the ELF header.
- Coderji over 5 years
  
  @schily I am sorry, can you elaborate more on this, I am very confused with RUNPATH and LD_LIBRARY_PATH. Is there a way to set RUNPATH for matlab MCR without the need to set LD_LIBRARY_PATH?
- schily over 5 years
  
  If you have the standard ELF utilities installed, you could run dump -Lv binaryfile to get the related information. If you only have the GNU tools, you need to check the related man pages. RUNPATH is created in the ELF header with the -R option from the linker and tells the runtime linker where to look for shared libraries that are specific to that binary.
- smw over 5 years
  
  I'm not sure why this is getting downvoted - aside from the fact that it's arguably up to Mathworks to distribute their application in a way that doesn't require such hackery. You may find some of the discussion here relevant: MATLAB bindings, when installed, do not work
- Coderji over 5 years
  
  @steeldriver thanks for your support. I really hate it when people downvote without pointing out what is the problem. Anyway, thanks for the link I will have a look at it
- 3D1T0R over 5 years
  
  I'm sorry if this is a weird question, but do you need both of these things to work at the same time (i.e. you run one command and both dependencies are needed) or are you able to run these things separately (i.e. you could run var=whatever command to launch a command with a certain var set without setting it for the whole session) ... If the latter, then you could make a .sh script that runs the command that needs the variable with var=whatever in front of it and not have it set the rest of the time.
- mikeserv over 5 years
  
  unset LD_LIBRARY_PATH
- Coderji over 5 years
  
  @mikserve I have tried to set the env for each run but the web framework (python's Django) would still capture the environments pre to the change.
Coderji over 5 years

Thanks for your answer.. I will test what have you suggested and let you know.
Coderji over 5 years

Thanks for your answer.. Your answer was the closest answer I didn't get it to work yet but I think I am getting there. Thanks again.