Plot dynamically changing graph using matplotlib in Jupyter Notebook

Question

I have a M x N 2D array: ith row represents that value of N points at time i.

I want to visualize the points [1 row of the array] in the form of a graph where the values get updated after a small interval. Thus the graph shows 1 row at a time, then update the values to next row, so on and so forth.

I want to do this in a jupyter notebook. Looking for reference codes.

I tried following things but no success:

• http://community.plot.ly/t/updating-graph-with-new-data-every-100-ms-or-so/812

• https://pythonprogramming.net/live-graphs-matplotlib-tutorial/

• Create dynamic updated graph with Python

• Update Lines in matplotlib

Solution 1

Here's an alternative, possibly simpler solution:

%matplotlib notebook
import numpy as np
import matplotlib.pyplot as plt

m = 100
n = 100
matrix = np.random.normal(0,1,m*n).reshape(m,n)

fig = plt.figure()
ax = fig.add_subplot(111)
plt.ion()

fig.show()
fig.canvas.draw()

for i in range(0,100):
    ax.clear()
    ax.plot(matrix[i,:])
    fig.canvas.draw()

Solution 2

I had been particularly looking for a good answer for the scenario where one thread is pumping data and we want Jupyter notebook to keep updating graph without blocking anything. After looking through about dozen or so related answers, here are some of the findings:

Caution

Do not use below magic if you want a live graph. The graph update does not work if the notebook uses below:

%load_ext autoreload
%autoreload 2

You need below magic in your notebook before you import matplotlib:

%matplotlib notebook

Method 1: Using FuncAnimation

This has a disadvantage that graph update occurs even if your data hasn't been updated yet. Below example shows another thread updating data while Jupyter notebook updating graph through FuncAnimation.

%matplotlib notebook

from matplotlib import pyplot as plt
from matplotlib.animation import FuncAnimation
from random import randrange
from threading import Thread
import time

class LiveGraph:
    def __init__(self):
        self.x_data, self.y_data = [], []
        self.figure = plt.figure()
        self.line, = plt.plot(self.x_data, self.y_data)
        self.animation = FuncAnimation(self.figure, self.update, interval=1000)
        self.th = Thread(target=self.thread_f, daemon=True)
        self.th.start()

    def update(self, frame):
        self.line.set_data(self.x_data, self.y_data)
        self.figure.gca().relim()
        self.figure.gca().autoscale_view()
        return self.line,

    def show(self):
        plt.show()

    def thread_f(self):
        x = 0
        while True:
            self.x_data.append(x)
            x += 1
            self.y_data.append(randrange(0, 100))   
            time.sleep(1)  

g = LiveGraph()
g.show()

Method 2: Direct Update

The second method is to update the graph as data arrives from another thread. This is risky because matplotlib is not thread safe but it does seem to work as long as there is only one thread doing updates.

%matplotlib notebook

from matplotlib import pyplot as plt
from matplotlib.animation import FuncAnimation
from random import randrange
from threading import Thread
import time

class LiveGraph:
    def __init__(self):
        self.x_data, self.y_data = [], []
        self.figure = plt.figure()
        self.line, = plt.plot(self.x_data, self.y_data)

        self.th = Thread(target=self.thread_f, daemon=True)
        self.th.start()

    def update_graph(self):
        self.line.set_data(self.x_data, self.y_data)
        self.figure.gca().relim()
        self.figure.gca().autoscale_view()

    def show(self):
        plt.show()

    def thread_f(self):
        x = 0
        while True:
            self.x_data.append(x)
            x += 1
            self.y_data.append(randrange(0, 100))  

            self.update_graph()

            time.sleep(1)  


from live_graph import LiveGraph

g = LiveGraph()
g.show()

Solution 3

I explored this and produced the following which is largely self-documenting:

import matplotlib.pyplot as plt
%matplotlib notebook

print('This text appears above the figures')
fig1 = plt.figure(num='DORMANT')
print('This text appears betweeen the figures')
fig2 = plt.figure()
print('This text appears below the figures')

fig1.canvas.set_window_title('Canvas active title')
fig1.suptitle('Figure title', fontsize=20)

# Create plots inside the figures
ax1 = fig1.add_subplot(111)
ax1.set_xlabel('x label')
ax2 = fig2.add_subplot(111)

# Loop to update figures
end = 40
for i in range(end):
    ax2.cla()  # Clear only 2nd figure's axes, figure 1 is ADDITIVE
    ax1.set_title('Axes title')  # Reset as removed by cla()

    ax1.plot(range(i,end), (i,)*(end-i))
    ax2.plot(range(i,end), range(i,end), 'rx')
    fig1.canvas.draw()
    fig2.canvas.draw()

Solution 4

Another simple solution, based on IPython.display functions display and clear_output. I found it here. Here is the code (based on @graham-s's answer):

from IPython.display import display, clear_output
import numpy as np
import matplotlib.pyplot as plt

%matplotlib inline

m = 100
n = 100
matrix = np.random.normal(0, 1, size=(m, n))

fig = plt.figure()
ax = fig.add_subplot(111)

for i in range(m):
    ax.clear()
    ax.plot(matrix[i, :])
    display(fig)
    clear_output(wait=True)
    plt.pause(0.2)

It uses %matplotlib inline instead of notebook, and does not produce small image as mentioned by @MasterScrat. Works both in Jupyter Notebook and in Jupyter Lab. Sometimes image blinks that's not very nice, but usable for quick investigations.

If you need to keep axes ranges between different frames, add ax.set_xlim/ax.set_ylim after ax.clear().

Solution 5

With a moderate modification of @Shital Shah's solution, I've created a more general framework which can simply apply to various scenario:

import matplotlib
from matplotlib import pyplot as plt

class LiveLine:
    def __init__(self, graph, fmt=''):
        # LiveGraph object
        self.graph = graph
        # instant line
        self.line, = self.graph.ax.plot([], [], fmt)
        # holder of new lines
        self.lines = []

    def update(self, x_data, y_data):
        # update the instant line
        self.line.set_data(x_data, y_data)
        self.graph.update_graph()

    def addtive_plot(self, x_data, y_data, fmt=''):
        # add new line in the same figure
        line, = self.graph.ax.plot(x_data, y_data, fmt)
        # store line in lines holder
        self.lines.append(line)
        # update figure
        self.graph.update_graph()
        # return line index
        return self.lines.index(line)

    def update_indexed_line(self, index, x_data, y_data):
        # use index to update that line
        self.lines[index].set_data(x_data, y_data)
        self.graph.update_graph()


class LiveGraph:
    def __init__(self, backend='nbAgg', figure_arg={}, window_title=None, 
                 suptitle_arg={'t':None}, ax_label={'x':'', 'y':''}, ax_title=None):

        # save current backend for later restore
        self.origin_backend = matplotlib.get_backend()

        # check if current backend meets target backend
        if self.origin_backend != backend:
            print("original backend:", self.origin_backend)
            # matplotlib.use('nbAgg',warn=False, force=True)
            plt.switch_backend(backend)
            print("switch to backend:", matplotlib.get_backend())

        # set figure
        self.figure = plt.figure(**figure_arg)
        self.figure.canvas.set_window_title(window_title)
        self.figure.suptitle(**suptitle_arg)

        # set axis
        self.ax = self.figure.add_subplot(111)
        self.ax.set_xlabel(ax_label['x'])
        self.ax.set_ylabel(ax_label['y'])
        self.ax.set_title(ax_title)

        # holder of lines
        self.lines = []

    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        self.close()

    def close(self):
        # check if current beckend meets original backend, if not, restore it
        if matplotlib.get_backend() != self.origin_backend:
            # matplotlib.use(self.origin_backend,warn=False, force=True)
            plt.switch_backend(self.origin_backend)
            print("restore to backend:", matplotlib.get_backend())

    def add_line(self, fmt=''):
        line = LiveLine(graph=self, fmt=fmt)
        self.lines.append(line)
        return line

    def update_graph(self):
        self.figure.gca().relim()
        self.figure.gca().autoscale_view()
        self.figure.canvas.draw()

With above 2 class, you can simply reproduce @Graham S's example:

import numpy as np

m = 100
n = 100
matrix = np.random.normal(0,1,m*n).reshape(m,n)

with LiveGraph(backend='nbAgg') as h:
    line1 = h.add_line()
    for i in range(0,100):
        line1.update(range(len(matrix[i,:])), matrix[i,:])

Note that, the default backend is nbAgg, you can pass other backend like qt5Agg. When it is finished, it'll restore to your original backend.

and @Tom Hale's example:

with LiveGraph(figure_arg={'num':'DORMANT2'}, window_title='Canvas active title', 
                suptitle_arg={'t':'Figure title','fontsize':20}, 
                ax_label={'x':'x label', 'y':''}, ax_title='Axes title') as g:
    with LiveGraph() as h:
        line1 = g.add_line()
        line2 = h.add_line('rx')
        end = 40
        for i in range(end):
            line1.addtive_plot(range(i,end), (i,)*(end-i))
            line2.update(range(i,end), range(i,end))

Also, you can update particular line in the additive plot of @Tom Hale's example:

import numpy as np

with LiveGraph(figure_arg={'num':'DORMANT3'}, window_title='Canvas active title', 
                suptitle_arg={'t':'Figure title','fontsize':20}, 
                ax_label={'x':'x label', 'y':''}, ax_title='Axes title') as g:
        line1 = g.add_line()
        end = 40
        for i in range(end):
            line_index = line1.addtive_plot(range(i,end), (i,)*(end-i))

        for i in range(100):
            j = int(20*(1+np.cos(i)))
            # update line of index line_index
            line1.update_indexed_line(line_index, range(j,end), (line_index,)*(end-j))

Note that, the second for loop is just for updating a particular line with index line_index. you can change that index to other line's index.

In my case, I use it in machine learning training loop to progressively update learning curve.

import numpy as np
import time

# create a LiveGraph object
g = LiveGraph()

# add 2 lines
line1 = g.add_line()
line2 = g.add_line()

# create 2 list to receive training result
list1 = []
list2 = []

# training loop
for i in range(100):
    # just training
    time.sleep(0.1)

    # get training result
    list1.append(np.random.normal())
    list2.append(np.random.normal())

    # update learning curve
    line1.update(np.arange(len(list1)), list1)
    line2.update(np.arange(len(list2)), list2)


# don't forget to close
g.close()

View more solutions