How to switch environment on Jupyter notebook for new notebook?

Question

I have an instance with various environment and some notebooks are compatible with different environment for e.g. notebook1 is for MXNet and notebook2 is for Tensorflow.

How do I jump to new environment when I move from notebook1 to notebook2? I tried doing that but this doesn't quite work with Jupyter notebook? Any suggestion?

So I need to do it from conda environment but looks like jupyter notebook UI doesn't respect (calls right activation function) to set the path.

Solution 1

You could use the nb_conda_kernels package, which provides a separate jupyter kernel for each conda environment, along with the appropriate code to handle their setup. This makes switching conda environment as simple as switching jupyter kernel (e.g. from the kernel menu), which I find very convenient. You can get it from the conda-forge channel, using

conda install -c conda-forge nb_conda_kernels

Solution 2

When you start a jupyter notebook within an environment, it will only have access to the modules installed in that particular enviroment. If you need two specific environments for two different notebooks, you will need to start a jupyter notebook within the two environments separately.

If you are using anaconda, you would do the following in a terminal:

source activate MXNET
jupyter notebook

Then in a second terminal:

 source activate Tensorflow
 jupyter notebook

Solution 3

Installing the nb_conda_kernels package as suggested by user @jcb91 worked for me. I did have to install it both in my root (base) environment as well as the virtual environment with which I wanted to use it. To do this, run the following in the Anaconda prompt (in your base environment):

conda install nb_conda_kernels

Then, activate your virtual environment (in the code below named 'myenv') and install the nb_conda_kernels package there as well. You can do this by running in the Anaconda prompt:

conda activate myenv
conda install nb_conda_kernels

You now should be able to switch to your different environment using:

Kernel -> Change Kernel

Solution 4

I don't really know the definition and relationship between kernel and conda env, but I beleive we can set a kernel for each env.

I got a problem similar to you: I clean my data in data_cleaning.ipynb under env_without_lgb. However, I have lightgbm installed under env_lightgbm. So, I want to switch kernel/env from env_without_lgb to env_lightgbm, which we can use Kernel -> Change Kernel:

If there is not a kernel you want, create a kernel under the right env according to this guide. For me, I only have a Pure Python 3 kernel and R kernel at first and then I create a kernel under env_lightgbm. At last, we can switch kernel when we editing our code.

However, I still can not figure out the relationship between kernel and env.

View more solutions

Comments

[rgaut]

Yes without using these command by just going through the UI, I want to switch the env

[rgaut]

Yes along with this package I think also need to change the ~/.jupyter/jupyter_notebook_config.py file with following config c.NotebookApp.kernel_spec_manager_class = 'environment_kernels.EnvironmentKernelSpecManager'

[jcb91]

This (setting the kernel_spec_manager_class value in config) is done automatically by the nb_conda_kernels.install module (see github.com/Anaconda-Platform/nb_conda_kernels/blob/…), which is called automatically as part of the conda package installation (see the post-link script in the conda recipe at github.com/Anaconda-Platform/nb_conda_kernels/blob/…)

[Amade]

not sure if this is what you're looking for, but you can try changing the environment in Jupyter Notebook menu, in Kernel -> Change Kernel -> [... available kernels ...]

[Shailen]

the conda-forge channel uses really old packages. Using this channel, you may also be forced to downgrade your python version. To be on the safe side, use the default anaconda channel, i.e use -c defaults

[Shailen]

@Amade - that does not work for me. Changing the kernels always uses the python of a certain activated python. The kernel.json for each of the kernels in the list point to different python paths in their respective environments. Still, !which python always uses the one in the currently activated env where jupyter notebook was started.

[Shailen]

Also, as of 01.2019, the comment from @rgaut does not seem to to work. You need to edit ~/.jupyter/jupyter_notebook_config.py and set: c.NotebookApp.kernel_spec_manager_class = 'nb_conda_kernels.CondaKernelSpecManager'