Share data between IPython Notebooks

Question

If I have several IPython notebooks running on the same server. Is there any way to share data between them? For example, importing a variable from another notebook? Thanks!

Solution 1

This works for me :

The %store command lets you pass variables between two different notebooks.

data = 'this is the string I want to pass to different notebook' %store data

Now, in a new notebook… %store -r data print(data) this is the string I want to pass to different notebook

I've successfully tested with sklearn dataset :

from sklearn import datasets

dataset = datasets.load_iris()

%store dataset

in notebook to read data :

%store -r dataset

src : https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/

Solution 2

Notebooks in Jupyter Lab can share the same kernel. In your notebook you can choose the kernel of another notebook and variables from the other notebook will be available in both notebooks.

1. Click on the button that describes your current kernel.
2. Choose the kernel of the other notebook, whose variables you want to access.

Solution 3

IPython supports the %store magic (here is the documentation). It seems to have the same constraints of pickle: if the file can be pickled it will also be storable.

Anyway, it will work for sure with common Python types. Here's a basic example:

var_1 = [1,2,3,4] #list
var_2 = {'a':1,'b':2,'c':3} #dict
var_3 = (6,7,8) #tuple
var_4 = {'d','e','f'} #set
%store var_1
%store var_2
%store var_3
%store var_4

 Stored 'var_1' (list)
 Stored 'var_2' (dict)
 Stored 'var_3' (tuple)
 Stored 'var_4' (set)

Then on a different IPython notebook it will be sufficient to type:

%store -r var_1 
%store -r var_2 
%store -r var_3 
%store -r var_4

Solution 4

If your data is in a single variable then have a try at saving it to a file using the %save magic in one notebook and then reading it back in another.

The one difficulty is that the text file will contain the data but no variable definition so I usually contatenate it with a variable definition and then exec the result.

View more solutions

Comments

[Kyle Siegel]

Messaging looks like it could potentially work. Essentially, I am standing up a server for a group of people to do some data analysis with a package that I am writing. Each notebook is doing a specific analysis and can be pretty long and involved. Occasionally, some data will need to be shared between notebooks, and there no way to really predict what data will need to be shared. Does that make sense?

[Kyle Siegel]

Definitely a possible solution. Would there be any way to do it without modifying the notebook with the variable I want? So essentially making the variable scope of one notebook the same as another -- giving notebook 1 access to notebook 2's variables, without modifying notebook 2 at all?

[Bruno Feroleto]

This does not "share data" but writes previous IPython inputs to a text files. The %store solution from blue-sky does save and reads back data between notebooks (and sessions, for that matter).

[BillyJo_rambler]

Any idea how you can suppress the the %store 'var1' (list) message from occuring?

[gibbone]

@BillyJo_rambler you can do it by using the %%capture cell magic: just add it on top of the cell and it should suppress the message.

[Wayne]

Related but with some other features that folks may find handy, there is IPython data-vault described as, "IPython magic for simple, organized, compressed and encrypted: storage & transfer of files between notebooks."