Conda set LD_LIBRARY_PATH for env only

Question

I have an installation of miniconda3 where I have created a virtual environment called py35. I have some libraries that I only want to use from within this environment. hence they are under

 /.../miniconda3/envs/py35/libs

However they are not found from within the environment as LD_LIBRARY_PATH does not contain said folder. I now want to set LD_LIBRARY_PATH to include the /lib only when I am in the virtual environment.

I was thinking of modifying the activate script miniconda uses to start the environment but am not quite sure if this is standard practice or if there is an easier way to achieve this.

Solution 1

You can set environment variables when an environment is activated by editing the activate.d/env_vars.sh script. See here: https://conda.io/docs/user-guide/tasks/manage-environments.html#macos-and-linux

The key portions from that link are:

1. Locate the directory for the conda environment in your Terminal window, such as /home/jsmith/anaconda3/envs/analytics.

2. Enter that directory and create these subdirectories and files:

   cd /home/jsmith/anaconda3/envs/analytics
   mkdir -p ./etc/conda/activate.d
   mkdir -p ./etc/conda/deactivate.d
   touch ./etc/conda/activate.d/env_vars.sh
   touch ./etc/conda/deactivate.d/env_vars.sh
   

3. Edit ./etc/conda/activate.d/env_vars.sh as follows:

   #!/bin/sh
   
   export MY_KEY='secret-key-value'
   export MY_FILE=/path/to/my/file/
   

4. Edit ./etc/conda/deactivate.d/env_vars.sh as follows::

   #!/bin/sh
   
   unset MY_KEY
   unset MY_FILE
   

When you run conda activate analytics, the environment variables MY_KEY and MY_FILE are set to the values you wrote into the file. When you run conda deactivate, those variables are erased.

Solution 2

I just wanted to add that you could declare 2 variables in the activate.d/env_vars.sh like, it makes it easier to reset the variable to the pre-activation state:

export OLD_LD_LIBRARY_PATH=${LD_LIBRARY_PATH}
export LD_LIBRARY_PATH=/your/path:${LD_LIBRARY_PATH}

and then in deactivate.d/env_vars.sh:

export LD_LIBRARY_PATH=${OLD_LD_LIBRARY_PATH}
unset OLD_LD_LIBRARY_PATH

Comments

[FlyingTeller]

Exactly what I was looking for. Thank you. Only small alteration since I add something to LD_LIBRARY_PATH I cannot use unset. Instead I do export LD_LIBRARY_PATH=${LD_LIBRARY_PATH/'PATH_I_ADDED/""}` in the deactivation script

[jlhw]

I've been trying to do this but the .sh files in the activate.d folder do not seem to be executing when I activate the conda environment. The new environment variables I create do not exist. Any pointers on this? Thank you!

[darthbith]

If the procedure from the documentation doesn't work, I'd suggest posting a new question including your recipe, the scripts, and any error output

[E.Serra]

Upvoted, useful stuff. Is there a way of doing this when creating the environment? does not seem the right thing to do with automated deployment

[darthbith]

@E.Serra There are pre-link, post-link, and post-unlink scripts that are used when a package is installed, which occurs on environment creation.

[E.Serra]

Should have read the docs before asking, TL;DR, exactly what I needed though thanks!

[Kelsius]

Highly recommend this if you need to keep the original values around. Been using this solution and works well.

[Vineet Bansal]

Is there a good reason why this is not done automatically by conda? After all, the user of an environment is expected to be able to use mutually-dependent libraries by activating the environment. A lot of these environment-specific packages are shared libraries and python wrappers for them, so when would you NOT want to do this? Shouldn't env-specific .so files ALWAYS take precedence over other locations?

[ameerosein]

Awesome, worked well!

[Gustavo Seabra]

This works well, but should be combined with @Oleksandr answer here. However, it does not seem to have an effect on Jupyter notebooks :-(

[igorkf]

Awesome! Thanks

[darthbith]

You're very welcome @igorkf!