The most efficient way to remove first N elements in a list?

Question

I need to remove the first n elements from a list of objects in Python 2.7. Is there an easy way, without using loops?

Solution 1

You can use list slicing to archive your goal.

Remove the first 5 elements:

n = 5
mylist = [1,2,3,4,5,6,7,8,9]
newlist = mylist[n:]
print newlist

Outputs:

[6, 7, 8, 9]

Or del if you only want to use one list:

n = 5
mylist = [1,2,3,4,5,6,7,8,9]
del mylist[:n]
print mylist

Outputs:

[6, 7, 8, 9]

Solution 2

Python lists were not made to operate on the beginning of the list and are very ineffective at this operation.

While you can write

mylist = [1, 2 ,3 ,4]
mylist.pop(0)

It's very inefficient.

---

If you only want to delete items from your list, you can do this with del:

del mylist[:n]

Which is also really fast:

In [34]: %%timeit
help=range(10000)
while help:
    del help[:1000]
   ....:
10000 loops, best of 3: 161 µs per loop

---

If you need to obtain elements from the beginning of the list, you should use collections.deque by Raymond Hettinger and its popleft() method.

from collections import deque

deque(['f', 'g', 'h', 'i', 'j'])

>>> d.pop()                          # return and remove the rightmost item
'j'
>>> d.popleft()                      # return and remove the leftmost item
'f'

A comparison:

list + pop(0)

In [30]: %%timeit
   ....: help=range(10000)
   ....: while help:
   ....:     help.pop(0)
   ....:
100 loops, best of 3: 17.9 ms per loop

deque + popleft()

In [33]: %%timeit
help=deque(range(10000))
while help:
    help.popleft()
   ....:
1000 loops, best of 3: 812 µs per loop

Solution 3

Try to run this code:

del x[:N]

Solution 4

Let's say you have this list:

mylist = [1,2,3,4,5,6,7,8,9]

And you want to remove the x last elements and store them in another list

newlist = [mylist.pop() for _ in range(x)]

You can modify the argument you pass to pop in order to remove elements from the beginning

newlist = [mylist.pop(0) for _ in range(x)]

Or leave the first element and remove x elements after

newlist = [mylist.pop(1) for _ in range(x)]

View more solutions

Comments

[RedVelvet]

using del mylist[:n] it's still inefficient ?

[Ciro Santilli OurBigBook.com]

I'm curious: how can pop be efficient if list is array backed? wiki.python.org/moin/TimeComplexity

[gies0r]

@RedVelvet: Yes. Nevertheless, regarding deque vs list keep in mind, that deque does not have all features which list have. on bigger shifts (1 million rows) I found del mylist[1000000:] is around the same execution as deque.popleft() for all the rows.. So it depends on your use case.

[Christian]

wow, is it actually in the specs that you can iterate over a deque while you're calling popleft() on it?