Python: How can I inherit from the built-in list type?

Question

I want to add some attributes to the built-in list type, so I wrote this:

class MyList(list):
    def __new__(cls, *args, **kwargs):
        obj = super(MyList, cls).__new__(cls, *args, **kwargs)
        obj.append('FirstMen')
        return obj

    def __init__(self, *args, **kwargs):
        self.name = 'Westeros'

    def king(self):
        print 'IronThrone'

if __name__ == '__main__':
    my_list = MyList([1, 2, 3, 4])
    print my_list

but my_list contains only the element 'FirstMen'. Why my __new__ doesn't work here? And how should I inherit from a built-in type like list? Is it the same for the immutable types like str?

Solution 1

The list type usually does the actual initialisation of the list inside its __init__() method, as it is the convention for mutable types. You only need to overwrite __new__() when subtyping immutable types. While you can overwrite __new__() when subclassing list, there is not much point in doing so for your use case. It's easier to just overwrite __init__():

class MyList(list):
    def __init__(self, *args):
        list.__init__(self, *args)
        self.append('FirstMen')
        self.name = 'Westeros'

Also note that I recommend against using super() in this case. You want to call list.__init__() here, and not possibly anything else.

Solution 2

First of all, are you doing this as an exercise to understand __new__? If not, there is almost certainly a better way to do what you're trying to do. Could you explain what you'd like to achieve here?

That said, here's what's happening in your example:

1. You invoke MyList([1,2,3,4])
2. This first invokes MyList.__new__(MyList,[1,2,3,4])
3. Your implementation calls list.__new__(MyList,[1,2,3,4]) This returns a new instance of MyList, with no elements. list.__new__ does not populate the list. It leaves that to list.__init__, which is never called.
4. Your __new__ method appends 'FirstMen' to the empty MyList instance.
5. Your __new__ method returns the instance of MyList.
6. MyList.__init__([1,2,3,4]) is invoked.
7. It sets the name attribute to 'Westeros'.
8. It returns.
9. The instance is assigned to my_list and printed.

See here for an explanation of __new__: http://docs.python.org/reference/datamodel.html#basic-customization

Comments

[pjhades]

Ah, I forgot to call the superclass's __init__. But how about the immutable types? Or is there a general rule about when __new__ should be overridden?

[Sven Marnach]

@PJ.Hades: Only overwrite it if you need to. This is generally the case if you need to influence the construction of the immutable data. Subclassing tuple to add further methods and attributes does not require to overwrite __new__(). If in doubt, just read the documentation.

[pjhades]

I knew the cause of this problem. Thanks for your explanation :)

[Sven Marnach]

Note that if MyList.__init__() did call list.__init__(), it would replace the content of the list by [1, 2, 3, 4], so the 'FirstMen' would be gone.

[Istiaque Ahmed]

@SvenMarnach, self.append('FirstMen') - where does FirstMen append - to list or MyList ?

[ucb]

Why would you use list.__init__() over super().__init__() ?

[Sven Marnach]

@ucb Generally, constructors of subclasses don't have necessarily the same prototype as the base class. super() may call the constructor of a random other class in a multiple inheritence scenario, which only works well when the constructors collaborate. It may not matter much, but I personally think using super() here just adds some unnecessary confusion. You know exactly what constructor you want to call, so it's best to be explicit about it.