How do I add rows and columns to a NUMPY array?

Question

Hello I have a 1000 data series with 1500 points in each.

They form a (1000x1500) size Numpy array created using np.zeros((1500, 1000)) and then filled with the data.

Now what if I want the array to grow to say 1600 x 1100? Do I have to add arrays using hstack and vstack or is there a better way?

I would want the data already in the 1000x1500 piece of the array not to be changed, only blank data (zeros) added to the bottom and right, basically.

Thanks.

Solution 1

If you want zeroes in the added elements, my_array.resize((1600, 1000)) should work. Note that this differs from numpy.resize(my_array, (1600, 1000)), in which previous lines are duplicated, which is probably not what you want.

Otherwise (for instance if you want to avoid initializing elements to zero, which could be unnecessary), you can indeed use hstack and vstack to add an array containing the new elements; numpy.concatenate() (see pydoc numpy.concatenate) should work too (it is just more general, as far as I understand).

In either case, I would guess that a new memory block has to be allocated in order to extend the array, and that all these methods take about the same time.

Solution 2

This should do what you want (ie, using 3x3 array and 4x4 array to represent the two arrays in the OP)

>>> import numpy as NP
>>> a = NP.random.randint(0, 10, 9).reshape(3, 3)
>>> a
>>> array([[1, 2, 2],
           [7, 0, 7],
           [0, 3, 0]])

>>> b = NP.zeros((4, 4))

mapping a on to b:

>>> b[:3,:3] = a

>>> b
    array([[ 1.,  2.,  2.,  0.],
           [ 7.,  0.,  7.,  0.],
           [ 0.,  3.,  0.,  0.],
           [ 0.,  0.,  0.,  0.]])

Solution 3

No matter what, you'll be stuck reallocating a chunk of memory, so it doesn't really matter if you use arr.resize(), np.concatenate, hstack/vstack, etc. Note that if you're accumulating a lot of data sequentially, Python lists are usually more efficient.

Comments

[mathtick]

Just a note that this doesn't appear to keep the data in place in the case when you merely want to extend the data set: >>> a = numpy.array([[1,2],[3,4]]) >>> a array([[1, 2], [3, 4]]) >>> a.resize((2,4)) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: cannot resize an array references or is referenced by another array in this way. Use the resize function >>> a = numpy.array(a) >>> a.resize((2,4)) >>> a array([[1, 2, 3, 4], [0, 0, 0, 0]])

[kηives]

I got an error from that code. Shouldn't the last line be b[:3, :3] = a ? All the same, plus one since when I did that it worked and that is what I was looking for.