Python list should be empty on class instance initialisation, but it's not. Why?

Question

I would like to create instances of a class containing a list that's empty by default; instead of later setting this list to the final full list I would like to successively add items to it. Here's a piece of sample code illustrating this:

#!/usr/bin/python

class test:
    def __init__(self, lst=[], intg=0):
        self.lista   = lst
        self.integer = intg

name_dict = {}
counter   = 0

for name in ('Anne', 'Leo', 'Suzy'):
    counter += 1

    name_dict[name] = test()
    name_dict[name].integer += 1
    name_dict[name].lista.append(counter)

    print name, name_dict[name].integer, name_dict[name].lista

When I ran the above program I expected to get

Anne 1 [1]
Leo 1 [2]
Suzy 1 [3]

as I assumed lista to always be initialised to an empty list.

What I got instead was this:

Anne 1 [1]
Leo 1 [1, 2]
Suzy 1 [1, 2, 3]

If I replace self.lista = lst by self.lista = [] it works fine, just like when I add the line name_dict[name].lista = [] to the for loop.

Why is it that the contents of the previous objects' lists are retained, yet their values of integer aren't? I am rather new to Python, so it would be great if someone could point out to me where my thoughts/assumptions have gone astray.

Thanks a lot in advance for your replies.

Solution 1

It is a very bad idea to use a mutable object as a default value, as you do here:

def __init__(self, lst=[], intg=0):
     # ...

Change it to this:

def __init__(self, lst=None, intg=0):
     if lst is None:
         lst = []
     # ...

The reason that your version doesn't work is that the empty list is created just once when the function is defined, not every time the function is called.

In some Python implementations you can see the value of the default values of the function by inspecting the value of func_defaults:

print test.__init__.func_defaults
name_dict[name] = test()
# ...

Output:

([],)
Anne 1 [1]
([1],)
Leo 1 [1, 2]
([1, 2],)
Suzy 1 [1, 2, 3] 

Solution 2

The problem lies in this line:

def __init__(self, lst=[], intg=0):

You shouldn't use a list as a default argument. The first time __init__ is called without lst specified the Python interpreter will define an empty list []. Subsequent calls to the function will operate on the same list if lst is not specified, without declaring a new list. This causes weird problems.

You should instead use a default value of None and add a check at the beginning of the function:

def __init__(self, lst=None, intg=0):
    if lst is None:
        lst = []

See this post for further details. Quoting the post:

Default arguments are evaluated at function definition time, so they're persistent across calls. This has some interesting (and confusing) side effects. An example:

>>> def foo(d=[]):
...     d.append('a')
...     return d

If you've not tried this before, you probably expect foo to always return ['a']: it should start with an empty list, append 'a' to it, and return. Here's what it actually does:

>>> foo() ['a']
>>> foo() ['a', 'a']
>>> foo() ['a', 'a', 'a']

This is because the default value for d is allocated when the function is created, not when it's called. Each time the function is called, the value is still hanging around from the last call. This gets even weirder if you throw threads into the mix. If two different threads are executing the function at the same time, and one of them changes a default argument, they both will see the change.

Of course, all of this is only true if the default argument's value is a mutable type. If we change foo to be defined as

>>> def foo2(d=0):
...     d += 1
...     return d

then it will always return 1. (The difference here is that in foo2, the variable d is being reassigned, while in foo its value was being changed.)

Solution 3

The problem lies with the default constructor argument. You should read this question to find answer to your question: “Least Astonishment” in Python: The Mutable Default Argument

Comments

[Jochen Ritzel]

possible duplicate of "Least Astonishment" in Python: The Mutable Default Argument

[keegan3d]

I like to declare it like this: lst = lst or []

[Mark Byers]

@keegan3d: That's not quite the same. If the user provides an empty list your version will not use that list but instead create a new one, because an empty list is "falsy".