Python Running cumulative sum with a given window

Question

What I want to do is generate a numpy array that is the cumulative sum of another numpy array given a certain window.

For example, given an array [1,2,3,4,5,6,7,8,9,10,11,12] let's say I want a cumulative sum with a window of 3. What I want as out put would be [1,3,6,9,12,15,18,21,24,27,30,33]. I have a relatively large numpy array and would like to do a cumulative sum with a window of 400.

Solution 1

In [42]: lis=[1,2,3,4,5,6,7,8,9,10,11,12]

In [43]: w=3       #window size

In [44]: [sum(lis[i-(w-1):i+1]) if i>(w-1) else sum(lis[:i+1])  for i in range(len(lis))]
Out[44]: [1, 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33]

In [45]: w=4

In [46]: [sum(lis[i-(w-1):i+1]) if i>(w-1) else sum(lis[:i+1])  for i in range(len(lis))]
Out[46]: [1, 3, 6, 10, 14, 18, 22, 26, 30, 34, 38, 42]

for python 2.4 or less, change the ternary operator:

(falseValue, trueValue)[condition] instead of trueValue if condition else falseValue

[(sum(lis[:i+1]),sum(lis[i-(w-1):i+1]))[i>(w-1)]  for i in range(len(lis))]

Solution 2

Here is perhaps a simpler answer, based on subtracting shifted cumsums.

>>> a = np.array([1,2,3,4,5,6,7,8,9,10,11,12])
>>> b = a.cumsum()
>>> b[3:] = b[3:] - b[:-3]
>>> b
array([ 1,  3,  6,  9, 12, 15, 18, 21, 24, 27, 30, 33])

Solution 3

You should likely use numpy, unless you really don't care about speed (though I would prefere it anyways). So you could use convolve or stride_tricks based approaches (these are not obvious, but solve these things nicely).

For example given a function like this (you can find more and fancier versions too):

def embed(array, dim, lag=1):
    """Create an embedding of array given a resulting dimension and lag.
    The array will be raveled before embedding.
    """
    array = np.asarray(array)
    array = array.ravel()
    new = np.lib.stride_tricks.as_strided(array,
                                     (len(array)-dim*lag+lag, dim),
                                     (array.strides[0], array.strides[0]*lag))
    return new

You can do:

embedded = embed(array, 400)
result = embedded.sum(1)

Which is memory efficient (the embedding or whatever you call it, does only create a view) and fast. The other approach of course would be to use convolve:

np.convolve(array, np.ones(400), mode='valid')

I don't know if you want the non full windows too, this would be the same as using mode='full' (default) for the convolve. For the other approach, that would have to be handled some other way.

Solution 4

seberg's answer is better and more general than mine, but note that you need to zero-pad your samples to get the result you want.

import numpy as np
from numpy.lib.stride_tricks import as_strided as ast
samples = 100
window = 3
padding = np.zeros(window - 1)
# zero-pad your samples
a = np.concatenate([padding,np.arange(1,samples + 1)])
newshape = (len(a) - window,window)
newstrides = a.strides * 2
# this gets you a sliding window of size 3, with a step of 1
strided = ast(a,shape = newshape,strides = newstrides)
# get your moving sum
strided.sum(1)

View more solutions