Timeseries streaming in bokeh

Question

I want to plot a live time series in bokeh. I want to plot only the new data points at each update. How can I do this ?

There is an example on the bokeh website for animated plots but it involves redrawing the whole picture every time. Also I am looking for a simple example where I can do a live plot of a time series point by point.

Solution

As of Bokeh 0.11.1 there is now a streaming interface to column data sources in Bokeh server apps. You can see and easily run an example here:

https://github.com/bokeh/bokeh/tree/master/examples/app/ohlc

That example shows a live updating OHLC chart with MACD indicator (based on synthetic tick data) that only updates the plot with the most recent data points on every update.

Basically, using the streaming interface consists of two parts. First create a new dict with same "shape" as your column data source:

new_data = dict(
    time=[t],
    open=[open],
    high=[high],
    low=[low],
    close=[close],
    average=[average],
    color=[color],
)

Then pass this to the .stream method, with an optional rollover argument that specifies how big of a buffer to keep in the browser (earlier data starts to get dropped off):

source.stream(new_data, 300)

Then, just the small amount of data in new_data willbe sent to the plot, not everything.

Comments

[bigreddot]

Can you describe the difference you want a little more? The technique from the example is to update the data in the datasource, and then the plot updates automatically. Many people have used the technique from the example to stream updating time series, I am not quite sure what you want to do differently? If you want to update a time series you can append new points to the end of the data. If you want a fixed window, you can also delete points of the beginning when necessary. If yo hare using numpy arrays, slicing might help make this simpler or more efficient.

[Maxi]

Yes I am talking about partial redraw like the blitz option in the animate function of matplotlib

[bigreddot]

How many points do you have? Are you sure this is an issue? Bokeh is performant into the tens of thousands of points and beyond this you often run into over plotting issues and have to resort to other visualization techniques. That said, adding support for damage regions would be a reasonable thing to do, but it's not on our short term roadmap.

[GoFaster]

Ran the example and saw the OHLC candles streaming by across the chart. Normally the final OHLC candlestick at the hard-right-edge of the chart will 'develop' as new data arrives. A new candlestick is formed at the passing of a time interval; its 'O' value is fixed at creation, the 'HLC' values will change as new data arrives. Candle at the hard-right-edge needs to fidget up and down with incoming data. User should be able to specify the time interval using interactive controls. Anyone able to update the demo; make it convincing & realistic?

[bigreddot]

I'm certainly not an experienced finance person. I made the demo to be a simple self-contained demonstration of .stream, not to be a real-world production finance tool. If someone with the right experience would like to enhance or improve the demo, that would certainly be welcome (to an extent --- the demo needs to remain relatively simple, to be effective as a demonstration of a particular topic) As an aside the new .patch method on CDS is probably useful for updating parts of the last candle "in place".

[GoFaster]

The way I see it: 1) Give user interactive control to specify candlestick time interval . 2) Synthesise few hundred bars data into array at specified time interval (Use datetime; x-axis shows recent dates & times). 3) Display the few hundred bars. 4)Generate synthesised real-time updates, continuously manipulate & visualise the HLC values of the latest candle on chart. 5) At the passing of the specified time-interval add another candle to the chart. 6) Keep going 7)Do it all again when the user changes interval. Question is; how to make this work in Bokeh, what calls and when?

[bigreddot]

This is probably a better discussion for the public mailing list.