The difference between super().method() versus super(self.__class__,self).method()

Question

Here is the code I was trying to write:

class A(object):
    def bind_foo(self):
        old_foo = self.foo
        def new_foo():
            old_foo()
            #super().foo()
            super(self.__class__,self).foo()

        self.foo = new_foo

    def __init__(self):
        print("A __init__")          

    def foo(self):
        print("A foo")

class B(A):
    def __init__(self):
        print("B __init__")
        super().__init__()

    def foo(self):
        print("B foo")
        super().foo()

class C(A):
    def __init__(self):
        print("C __init__")
        super().__init__()
        super().bind_foo()

    def foo(self):
        print("C foo")    

b  = B()
b.foo()

c = C()
c.foo()

Class B and A is the expected behavior, that is, when I call b.foo(), it calls a.foo() as well with super(). Class C is the trying to mimic the child B and parent A behavior but this time I dont want to put explicitly super().foo() in the child class but I still want the parent foo() to be called. It works as expected.

However, what I dont quite get is that, under A.bind_foo, I have to use super(self.__class__,self).foo() rather than super().foo. super().foo gives a

"SystemError: super(): no arguments". 

Can someone explain why that is so?

Solution

You should not use self.__class__ or type(self) when calling super().

In Python 3, a call to super() without arguments is equivalent to super(B, self) (within methods on class B); note the explicit naming of the class. The Python compiler adds a __class__ closure cell to methods that use super() without arguments (see Why is Python 3.x's super() magic?) that references the current class being defined.

If you use super(self.__class__, self) or super(type(self), self), you will hit an infinite recursion exception when a subclass tries to call that method; at that time self.__class__ is the derived class, not the original. See When calling super() in a derived class, can I pass in self.__class__?

So, to summarize, in Python 3:

class B(A):
    def __init__(self):
        print("B __init__")
        super().__init__()

    def foo(self):
        print("B foo")
        super().foo()

is equal to:

class B(A):
    def __init__(self):
        print("B __init__")
        super(B, self).__init__()

    def foo(self):
        print("B foo")
        super(B, self).foo()

but you should use the former, as it saves you repeating yourself.

In Python 2, you are stuck with the second form only.

For your bind_foo() method, you'll have to pass in an explicit class from which to search the MRO from, as the Python compiler cannot determine here what class is used when you bind the new replacement foo:

def bind_foo(self, klass=None):
    old_foo = self.foo
    if klass is None:
        klass = type(self)

    def new_foo():
        old_foo()
        super(klass, self).foo()

    self.foo = new_foo

You could use __class__ (no self) to have Python provide you with the closure cell, but that'd be a reference to A, not C here. When you are binding the new foo, you want the search for overridden methods in the MRO to start searching at C instead.

Note that if you now create a class D, subclassing from C, things will go wrong again, because now you are calling bind_foo() and in turn call super() with D, not C, as the starting point. Your best bet then is to call bind_foo() with an explicit class reference. Here __class__ (no self.) will do nicely:

class C(A):
    def __init__(self):
        print("C __init__")
        super().__init__()
        self.bind_foo(__class__)

Now you have the same behaviour as using super() without arguments, a reference to the current class, the one in which you are defining the method __init__, is passed to super(), making the new_foo() behave as if it was defined directly in the class definition of C.

Note that there is no point in calling bind_foo() on super() here; you didn't override it here, so you can just call self.bind_foo() instead.

Comments

[No Harm In Trying]

So how would I solve the problem I have at hand then with child C?

[Blckknght]

I suspect the no-argument super doesn't work in the definition of new_foo because that's not a method (just a regular function being assigned to an instance variable). It's probably a bad idea to mix inheritance-style method overloading (where you can call super) with instance variable overloading.

[Martijn Pieters]

@Blckknght: you really don't want to have the automatic class used there; you want to determine the class at call time (so when bind_foo() is called, not when the bind_foo() function object is created).

[No Harm In Trying]

Thanks for the help! Suppose if I want D to inherent from C, C from B and B from A; but I want the D's foo to call D's implementation, then C's, then B's, then A's. Whats the best way to do so?

[Martijn Pieters]

@NoHarmInTrying: by giving each class a foo implementation that does the right thing; call super().foo(). Also see Raymond Hettinger's blog post on super() use. That's what super() is for, cooperative class design.

[No Harm In Trying]

Is it not possible without explicitly putting super() in every foo()?

[Martijn Pieters]

@NoHarmInTrying: you could loop over type(self).mro() and see for each class if there is a foo attribute, then call that function directly passing in self. You would not use super() there, that is not what super() is for.

[No Harm In Trying]

I didnt know know type(self).mro() existed, that help things greatly. Thanks!