Strange result when removing item from a list while iterating over it

Question

I've got this piece of code:

numbers = range(1, 50)

for i in numbers:
    if i < 20:
        numbers.remove(i)

print(numbers)

but the result I'm getting is:
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]

Of course, I'm expecting the numbers below 20 to not appear in the results. Looks like I'm doing something wrong with the remove.

Solution 1

You're modifying the list while you iterate over it. That means that the first time through the loop, i == 1, so 1 is removed from the list. Then the for loop goes to the second item in the list, which is not 2, but 3! Then that's removed from the list, and then the for loop goes on to the third item in the list, which is now 5. And so on. Perhaps it's easier to visualize like so, with a ^ pointing to the value of i:

[1, 2, 3, 4, 5, 6...]
 ^

That's the state of the list initially; then 1 is removed and the loop goes to the second item in the list:

[2, 3, 4, 5, 6...]
    ^
[2, 4, 5, 6...]
       ^

And so on.

There's no good way to alter a list's length while iterating over it. The best you can do is something like this:

numbers = [n for n in numbers if n >= 20]

or this, for in-place alteration (the thing in parens is a generator expression, which is implicitly converted into a tuple before slice-assignment):

numbers[:] = (n for in in numbers if n >= 20)

If you want to perform an operation on n before removing it, one trick you could try is this:

for i, n in enumerate(numbers):
    if n < 20 :
        print("do something")
        numbers[i] = None
numbers = [n for n in numbers if n is not None]

Solution 2

Begin at the list's end and go backwards:

li = list(range(1, 15))
print(li)

for i in range(len(li) - 1, -1, -1):
    if li[i] < 6:
        del li[i]
        
print(li)

Result:

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14] 
[6, 7, 8, 9, 10, 11, 12, 13, 14]

Solution 3

@senderle's answer is the way to go!

Having said that to further illustrate even a bit more your problem, if you think about it, you will always want to remove the index 0 twenty times:

[1,2,3,4,5............50]
 ^
[2,3,4,5............50]
 ^
[3,4,5............50]
 ^

So you could actually go with something like this:

aList = list(range(50))
i = 0
while i < 20:
    aList.pop(0)
    i += 1

print(aList) #[21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]

I hope it helps.

---

The ones below are not bad practices AFAIK.

EDIT (Some more):

lis = range(50)
lis = lis[20:]

Will do the job also.

EDIT2 (I'm bored):

functional = filter(lambda x: x> 20, range(50))

Solution 4

So I found a solution but it's really clumsy...

First of all you make an index array, where you list all the index' you want to delete like in the following

numbers = range(1, 50)
index_arr = []

for i in range(len(numbers):
    if numbers[i] < 20:
        index_arr.append(i)

after that you want to delete all the entries from the numbers list with the index saved in the index_arr. The problem you will encounter is the same as before. Therefore you have to subtract 1 from every index in the index_arr after you just removed a number from the numbers arr, like in the following:

numbers = range(1, 50)
index_arr = []

for i in range(len(numbers):
    if numbers[i] < 20:
        index_arr.append(i)

for del_index in index_list:
    numbers.pop(del_index)

    #the nasty part
    for i in range(len(index_list)):
        index_list[i] -= 1

It will work, but I guess it's not the intended way to do it

View more solutions

Comments

[David Hempy]

How I wish I could +2 this answer! Elegant, easy...not entirely obfuscated.

[Gino Mempin]

Related note on for keeping an index from the Python docs docs.python.org/3.9/reference/…: "There is a subtlety when the sequence is being modified by the loop (this can only occur for mutable sequences, e.g. lists). An internal counter is used to keep track of which item is used next, and this is incremented on each iteration. ... This means that if the suite deletes the current (or a previous) item from the sequence, the next item will be skipped (since it gets the index of the current item which has already been treated)."

[mike rodent]

This is a good answer, but with the final solution, "if you want to perform an operation...", is slightly unsatisfactory because 1) in fact there is no need to include that qualification: it is just a waste of effort to try and remove elements while iterating in a single operation, so this 2-stage solution applies in all cases, and 2) because there should be a warning that setting to None won't always be appropriate (if some elements are meant to be None): some conventional "poison pill" value, appropriate to the case in hand, is needed.

[mike rodent]

This looks like a neat suggestion to a classic problem, but I'm not sure that in practice it will prove better than the 2-stage solution suggested at the end of the chosen answer: firstly, what is the cost of the copy operation? Secondly, you have to concern yourself with whether this is a deep or shallow copy. In this trivial case the question doesn't arise but that won't always be so.

[mike rodent]

This is a very specialised answer: it is not in fact clear whether we're meant to be looking for a general solution to the problem of how to remove elements while iterating, or how to do this exclusively when we just want to remove the first n elements of a list. The chosen answer provides the former, which is infinitely more helpful, but also the latter, in the shape of a one-line list comprehension solution.

[mike rodent]

When I say that about the "waste of effort", I am referring to the "general" solution by the way, i.e. when random elements may need removing, rather than the "specialist" case of removing elements only at the start (or only at the end), which lends itself to something simple, like your list comprehension solution...

[Boris Verkhovskiy]

@mikerodent no it's not. It's pretty common when you want to modify a list while iterating over it that going backwards works

[mike rodent]

@Boris you haven't understood my comment. The OP's question does not specify that we are removing contiguous elements (either from the start or end of the list).

[Boris Verkhovskiy]

I still don't understand your comment then because it doesn't matter whether the list is shuffled or ordered, this code will still work.