How to avoid "RuntimeError: dictionary changed size during iteration" error?

Question

I have checked all of the other questions with the same error yet found no helpful solution =/

I have a dictionary of lists:

d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}

in which some of the values are empty. At the end of creating these lists, I want to remove these empty lists before returning my dictionary. Current I am attempting to do this as follows:

for i in d:
    if not d[i]:
        d.pop(i)

however, this is giving me the runtime error. I am aware that you cannot add/remove elements in a dictionary while iterating through it...what would be a way around this then?

Solution 1

In Python 3.x and 2.x you can use use list to force a copy of the keys to be made:

for i in list(d):

In Python 2.x calling keys made a copy of the keys that you could iterate over while modifying the dict:

for i in d.keys():

But note that in Python 3.x this second method doesn't help with your error because keys returns an a view object instead of copynig the keys into a list.

Solution 2

You only need to use copy:

This way you iterate over the original dictionary fields and on the fly can change the desired dict d. It works on each Python version, so it's more clear.

In [1]: d = {'a': [1], 'b': [1, 2], 'c': [], 'd':[]}

In [2]: for i in d.copy():
   ...:     if not d[i]:
   ...:         d.pop(i)
   ...:         

In [3]: d
Out[3]: {'a': [1], 'b': [1, 2]}

(BTW - Generally to iterate over copy of your data structure, instead of using .copy for dictionaries or slicing [:] for lists, you can use import copy -> copy.copy (for shallow copy which is equivalent to copy that is supported by dictionaries or slicing [:] that is supported by lists) or copy.deepcopy on your data structure.)

Solution 3

Just use dictionary comprehension to copy the relevant items into a new dict:

>>> d
{'a': [1], 'c': [], 'b': [1, 2], 'd': []}
>>> d = {k: v for k, v in d.items() if v}
>>> d
{'a': [1], 'b': [1, 2]}

For this in Python 2:

>>> d
{'a': [1], 'c': [], 'b': [1, 2], 'd': []}
>>> d = {k: v for k, v in d.iteritems() if v}
>>> d
{'a': [1], 'b': [1, 2]}

Solution 4

This worked for me:

d = {1: 'a', 2: '', 3: 'b', 4: '', 5: '', 6: 'c'}
for key, value in list(d.items()):
    if (value == ''):
        del d[key]
print(d)
# {1: 'a', 3: 'b', 6: 'c'}

Casting the dictionary items to list creates a list of its items, so you can iterate over it and avoid the RuntimeError.

Solution 5

I would try to avoid inserting empty lists in the first place, but, would generally use:

d = {k: v for k,v in d.iteritems() if v} # re-bind to non-empty

If prior to 2.7:

d = dict( (k, v) for k,v in d.iteritems() if v )

or just:

empty_key_vals = list(k for k in k,v in d.iteritems() if v)
for k in empty_key_vals:
    del[k]

View more solutions

Comments

[Mark Byers]

+1: last option is interesting because it only copies the keys of those items that need deleting. This may give better performance if only a small number of items need deleting relative to the size of the dict.

[Jon Clements]

@MarkByers yup - and if a large number do, then re-binding the dict to a new one that's filtered is a better option. It's always the expectation of how the structure should work

[Mark Byers]

One danger with rebinding is if somewhere in the program there was an object that held a reference to the old dict it wouldn't see the changes. If you're certain that's not the case, then sure... that's a reasonable approach, but it's important to understand that it's not quite the same as modifying the original dict.

[Jon Clements]

@MarkByers extremely good point - You and I know that (and countless others), but it's not obvious to all. And I'll put money on the table it hasn't also bitten you in the rear :)

[HighOnMeat]

I believe you meant 'calling keys makes a copy of the keys that you can iterate over' aka the plural keys right? Otherwise how can one iterate over a single key? I'm not nit picking by the way, am genuinely interested to know if that is indeed key or keys

[wcyn]

d.iteritems() gave me an error. I used d.items() instead - using python3

[Yirkha]

This works for the problem posed in OP's question. However anyone who came here after hitting this RuntimeError in multi-threaded code, be aware that CPython's GIL can get released in the middle of the list comprehension as well and you have to fix it differently.

[Brambor]

Or tuple instead of list as it is faster.

[Michael Krebs]

To clarify the behavior of python 3.x, d.keys() returns an iterable (not an iterator) which means that it's a view on the dictionary's keys directly. Using for i in d.keys() does actually work in python 3.x in general, but because it is iterating over an iterable view of the dictionary's keys, calling d.pop() during the loop leads to the same error as you found. for i in list(d) emulates the slightly inefficient python 2 behavior of copying the keys to a list before iterating, for special circumstances like yours.

[ron_g]

Nice and concise. Worked for me in Python 2.7 also.

[Magnus Bodin]

The point of avoiding to insert the empty entries is a very good one.

[Ali dashti]

your python3 solution is not work when you want to delete an object in inner dict. for example you have a dic A and dict B in dict A. if you want to delete object in dict B it occurs error

[tsveti_iko]

for python3 you can try for i in list(d.keys()):

[Sean Breckenridge]

In python3.x, list(d.keys()) creates the same output as list(d), calling list on a dict returns the keys. The keys call (though not that expensive) is unnecessary.

[Daniel Chin]

Surprisingly, items() doc claims to "return a copy of the dictionary’s list of (key, value) pairs.", yet looping through items() while poping items still gives RuntimeError: dictionary changed size during iteration.