Is super() broken in Python-2.x?

Question

It's often stated that super should be avoided in Python 2. I've found in my use of super in Python 2 that it never acts the way I expect unless I provide all arguments such as the example:

super(ThisClass, self).some_func(*args, **kwargs)

It seems to me this defeats the purpose of using super(), it's neither more concise, or much better than TheBaseClass.some_func(self, *args, **kwargs). For most purposes method resolution order is a distant fairy tale.

• Other than the fact that 2.7 is the last major release to Python 2, why does super remain broken in Python 2?
• How and why has Python 3's super changed? Are there any caveats?
• When and why should I use super going forward?

Solution 1

super() is not broken -- it just should not be considered the standard way of calling a method of the base class. This did not change with Python 3.x. The only thing that changed is that you don't need to pass the arguments self, cls in the standard case that self is the first parameter of the current function and cls is the class currently being defined.

Regarding your question when to actually use super(), my answer would be: hardly ever. I personally try to avoid the kind of multiple inheritance that would make super() useful.

Edit: An example from real life that I once ran into: I had some classes defining a run() method, some of which had base classes. I used super() to call the inherited constructors -- I did not think it mattered because I was using single inheritance only:

class A(object):
    def __init__(self, i):
        self.i = i
    def run(self, value):
        return self.i * value

class B(A):
    def __init__(self, i, j):
        super(B, self).__init__(i)
        self.j = j
    def run(self, value):
        return super(B, self).run(value) + self.j

Just imagine there were several of these classes, all with individual constructor prototypes, and all with the same interface to run().

Now I wanted to add some additional functionality to all of these classes, say logging. The additional functionality required an additional method to be defined on all these classes, say info(). I did not want to invade the original classes, but rather define a second set of classes inheriting from the original ones, adding the info() method and inheriting from a mix-in providing the actual logging. Now, I could not use super() in the constructor any more, so I used direct calls:

class Logger(object):
    def __init__(self, name):
        self.name = name
    def run_logged(self, value):
        print "Running", self.name, "with info", self.info()
        return self.run(value)

class BLogged(B, Logger):
    def __init__(self, i, j):
        B.__init__(self, i, j)
        Logger.__init__("B")
    def info(self):
        return 42

Here things stop working. The super() call in the base class constructor suddenly calls Logger.__init__(), and BLogged can't do anything about it. There is actually no way to make this work, except for removing the super() call in B itself.

[Another Edit: I don't seem to have made my point, judging from all the comments here and below the other answers. Here is how to make this code work using super():

class A(object):
    def __init__(self, i, **kwargs):
        super(A, self).__init__(**kwargs)
        self.i = i
    def run(self, value):
        return self.i * value

class B(A):
    def __init__(self, j, **kwargs):
        super(B, self).__init__(**kwargs)
        self.j = j
    def run(self, value):
        return super(B, self).run(value) + self.j

class Logger(object):
    def __init__(self, name, **kwargs):
        super(Logger,self).__init__(**kwargs)
        self.name = name
    def run_logged(self, value):
        print "Running", self.name, "with info", self.info()
        return self.run(value)

class BLogged(B, Logger):
    def __init__(self, **kwargs):
        super(BLogged, self).__init__(name="B", **kwargs)
    def info(self):
        return 42

b = BLogged(i=3, j=4)

Compare this with the use of explicit superclass calls. You decide which version you prefer.]

This and similar stories are why I think that super() should not be considered the standard way of calling methods of the base class. It does not mean super() is broken.

Solution 2

super() is not broken, in Python 2 or Python 3.

Let's consider the arguments from the blog post:

• It doesn't do what it sounds like it does.

OK, you may agree or disagree on that, it's pretty subjective. What should it have been called then? super() is a replacement for calling the superclass directly, so the name seems fine to me. It does NOT call the superclass directly, because if that was all it did, it would be pointless, as you could do that anyway. OK, admittedly, that may not be obvious, but the cases where you need super() are generally not obvious. If you need it, you are doing some pretty hairy multiple inheritance. It's not going to be obvious. (Or you are doing a simple mixin, in which case it will be pretty obvious and behave as you expect even if you didn't read the docs).

If you can call the superclass directly, that's probably what you'll end up doing. That's the easy and intuitive way of doing it. super() only comes into play when that doesn't work.

• It doesn't mesh well with calling the superclass directly.

Yes, because it's designed to solve a problem with doing that. You can call the superclass directly if, and only if, you know exactly what class that is. Which you don't for mixins, for example, or when your class hierarchy is so messed up that you actually are merging two branches (which is the typical example in all examples of using super()).

So as long as every class in your class hierarchy has a well defined place, calling the superclass directly works. If you don't, then it does not work, and in that case you must use super() instead. That's the point of super() that it figures out what the "next superclass" is according to the MRO, without you explicitly having to specify it, because you can't always do that because you don't always know what it is, for example when using mixins.

• The completely different programming language Dylan, a sort of lisp-thingy, solves this in another way that can't be used in Python because it's very different.

Eh. OK?

• super() doesn't call your superclass.

Yeah, you said that.

• Don't mix super() and direct calling.

Yeah, you said that too.

So, there is two arguments against it: 1. The name is bad. 2. You have to use it consistently.

That does not translate to it being "broken" or that it should be "avoided".

Solution 3

You seem to imply in your post that

def some_func(self, *args, **kwargs):
    self.__class__.some_func(self, *args, **kwargs)

is not an infinite recursion. It is, and super would be more correct.

Also, yes, you are required to pass all arguments to super(). This is a bit like complaining that max() doesn't work like expected unless you pass it all the numbers you want to check.

In 3.x, however, fewer arguments are needed: you can do super().foo(*args, **kwargs) instead of super(ThisClass, self).foo(*args, **kwargs).

---

Anyway, I'm unsure as to any situations when super should be avoided. Its behavior is only "weird" when MI is involved, and when MI is involved, super() is basically your only hope for a correct solution. In Single-Inheritance it's just slightly wordier than SuperClass.foo(self, *args, **kwargs), and does nothing different.

I think I agree with Sven that this sort of MI is worth avoiding, but I don't agree that super is worth avoiding. If your class is supposed to be inherited, super offers users of your class hope of getting MI to work, if they're weird in that way, so it makes your class more usable.

Solution 4

Did you read the article that you link it? It doesn't conclude that super should be avoided but that you should be wary of its caveats when using it. These caveats are summarized by the article, though I would disagree with their suggestions.

The main point of the article is that multiple inheritance can get messy, and super doesn't help as much as the author would want. However doing multiple inheritance without super is often even more complicated.

If you're not doing multiple inheritance, super gives you the advantage that anyone inheriting from your class can add simple mixins and their __init__ would be properly called. Just remember to always call the __init__ of the superclass, even when you're inheriting from object, and to pass all the remaining arguments (*a and **kw) to it. When you're calling other methods from the parent class also use super, but this time use their proper signature that you already know (i.e. ensure that they have the same signature in all classes).

If you're doing multiple inheritance you'd have to dig deeper than that, and probably re-read the same article more carefully to be aware of the caveats. And it's also only during multiple inheritance when you might a situation where an explicit call to the parent might be better than super, but without a specific scenario nobody can tell you whether super should be used or not.

The only change in super in Python 3.x is that you don't need to explicitly pass the current class and self to it. This makes super more attractive, because using it would mean no hardcoding of either the parent class or the current class.

Solution 5

@Sven Marnach:

The problem with your example is that you mix explicit superclass calls B.__init__ and Logger.__init__ in Blogged with super() in B. That won't work. Either you use all explicit superclass calls or use super() on all classes. When you use super() you need to use it on all classes involved, including A I think. Also in your example I think you could use explicit superclass calls in all classes, i.e use A.__init__ in class B.

When there is no diamond inheritance I think super() doesn't have much advantage. The problem is, however, that you don't know in advance if you will get into any diamond inheritance in the future so in that case it would be wise to use super() anyway (but then use it consistently). Otherwise you would end up having to change all classes at a later time or run into problems.

View more solutions

Comments

[slandau]

You don't give any example of why you think super is broken, other than that it isn't shorter than calling the base class explicitly which is neither here nor there... You obviously don't understand what super() is for (for up-calling safely with multiple inheritance).

[Matt Joiner]

@fuzzyman: That would be subjective, and fuzzy isn't part of my name :)

[slandau]

Well, no. "Broken" is not a subjective concept and super is needed in some circumstances. So you don't explain why you think super is broken (other than by reference to an article that doesn't say that) and you don't show any appearance of understanding what super is for (hint: it isn't the equivalent of the alternative you show).

[RubenLaguna]

When you say that TheBaseClass.some_func(self, *args, **kwargs) is better than super you are assuming that TheBaseClass is always the ancestor of ThisClass which may not be the case if you have multiple inheritance (or someone adds it afterwards). It could be that super(ThisClass, self) resolves to something else than TheBaseClass. super(). So the question is do you want to call the "next guy" in the chain (most likely) or your want to call TheBaseClass exactly.

[Sven Marnach]

I strongly disagree with that super() is the "only hope" when multiple inheritance is involved. There are clas hierachies that require super(), but many class hierarchies with multiple inheritance don't.

[Devin Jeanpierre]

@Sven super is the only way to create anything resembling a general solution (across different kinds of inheritance graphs), as far as I understand it. Of course if you aren't interested in a general solution, then super doesn't give you anything. On the other hand, you aren't losing anything by using super either. So, I would use super.

[Sven Marnach]

@Devin: Did you actually read this page linked by the OP? At least you don't address any of the problems mentioned there. You are losing something if you use super(). I used to use it a lot, and it broke a lot of my code. Meanwhile, everytime I encounter super() in my code I edit it out.

[Devin Jeanpierre]

@Sven I've read it a few times over the years. I can't find what you're thinking of from a skim. What does it say you lose by using super, that you retain by using SuperClass.method(...)? I addressed the problems mentioned in the question, not the article, because he actually had questions.

[Sven Marnach]

@Devin: sorry, I did not want to sound harsh. Specifically what you are losing is that you don't know what method you are actually calling. That bit me quite a few times over the years. Usually, I know which __init__() I want to call, and I don't want super() to select a different one depending on context. And I do use multiple inheritance.

[Matt Joiner]

Is the last line of B meant to read super(B, self), and should those defs be classes?

[Lennart Regebro]

@Sven: But that's the whole point of it. You use super() in cases where you do not know exactly which method you need to call. :-) If you did, you could just call it.

[Lennart Regebro]

I have no idea what you mean with that.

[Sven Marnach]

@Lennart: My only point is that super() should not be considered the standard way of calling base class methods. Many people do that, and Devin also suggest doing this.

[Devin Jeanpierre]

@Sven Perhaps you should read the article I linked ;). There are good reasons to want super-like behavior. If you disagree with the MRO produced, you change the order of inheritance. It is true that there are some inheritance chains that are impossible to order sensibly, but they are very strange (check out the examples). I suggest using it as the regular way because you really do lose nothing by doing it in SI code, and (sane) MI code can always be made to work with it (not always the case for explicit base calls).

[Sven Marnach]

@Devin: I read the article several times. I gave a full example of what you lose even in single inheritance in my answer.

[Rosh Oxymoron]

It's unclear how subclassing is better than self._logger = Logger("B") - mixing arbitrary classes in a tree has never worked. You can think of a similarly broken example but with direct calling. The classes need to be designed to work together. And super can help if they aren't. But, as the article states, you're better off with passing the arguments as keywords when calling the superclass. Then your problem wouldn't exist if the Logger class correctly used super.

[Sven Marnach]

@Rosh: All this works perfectly fine if I just don't use super(). I don't consider it broken at all. I would like to call b.run_logged(value) on a BLogged instance b, not b._logger.run_logged(value).

[slandau]

Matt, trying to insult the people who reply to you does nothing for your technical arguments.

[Sven Marnach]

Well, I know why my example does not work. It was meant as an example how routinely using super() has disadvantages. You can make the example work with super(), but it will get really ugly, while it is straight-forward with explicit superclass calls. Note that there is diamond inheritance involved since both A and Logger derive from object. Again, my whole point is that super() should not be considered the standard way of calling base class methods.

[Devin Jeanpierre]

@Sven in single inheritance the MRO is identical to the reverse of the inheritance chain. You know exactly which method you are calling.

[Sven Marnach]

@Devin: Then look at the example I gave. When I first designed the hierarchy, it was single inheritance only, so I did not care whether to use super() or explicit calls. Later, when I introduced mix-ins further down in the hierarchy, the old super() calls bit me.

[Lennart Regebro]

FYI, my profile reads string theory research papers and gives criticism on it. It's a pretty awesome profile. But I still don't understand what you mean. My profile probably does, but it's too busy to answer my emails.

[Devin Jeanpierre]

@Sven Your example is of MI, not SI. The problem arises from ignorance of how super works: you can't mix super and SuperClass.method, because of what super does. That aside, you seem to think that SI code can be reused without alterations to become MI code: this is false in the general case (see: diamond inheritance problem), although super makes it true for more cases, and easier/possible for the remainder. It will always work with mixins, provided you use the constructs correctly, so if your complaint is that it removes the ability to later use mixins from SI code, that is incorrect.

[Matt Joiner]

@Lennart Regebro: Sorry for the confusion, it's a more unusual use for the word 'read': definition 6: en.wiktionary.org/wiki/read#Verb. I mean tongue-in-cheek, that if your answer were more conversational I would think it written by a Java fan.

[Sven Marnach]

@Devin: Maybe it's my limited command of the English language, but you seem to get me completely wrong. I perfectly know how super() works. I know why my example broke and how to fix it, both with super() and with explicit superclass calls. The version using super() is ugly and hard to grasp, while the version using explicit calls is straight-forward. I happen to prefer the latter.

[Sven Marnach]

Also note that in my example, you could simply mix in other classes into the hierarchy (thus "reusing SI code whithout alterations to become MI code") if it wouldn't be for the super() call -- without it everything is straight-forward.

[Devin Jeanpierre]

@Sven If you know how to fix it using only super calls, fine, but at the time you claimed that the only way to fix it was to remove super. As for your solution, you make it unnecessarily ugly with **kwargs crap. I don't think you understand super() if you think that's necessary. And I don't think you understand mixins if you have diamond inheritance with them. That said, I've had enough of this thread -- the "question"-asker never really wanted an answer, just a repetition of his preconceptions, and you have already given it.

[Sven Marnach]

@Devin: I would be honestly interested in how to do it better and how to use mix-ins with new-style classes without getting any diamonds. Your last comment was actually pretty insulting and contained pretty little technical information.

[Lennart Regebro]

Java fans are not in my experience more talkative than Pythonistas... Ah well.

[Wayne Werner]

For those coming upon this in the future, Raymond Hettinger gave a fantastic talk: Super is considered Super.

[mirek]

I think super() should be always called, with simple inheritance too. That way you do real OOP with inheritance, and methods will behave exactly like properties do (and I think this is correct and clean behaviour). You don't say about the property: if not defined here, take it from A class. You say: take it from Parent. If you define the method then in proper OOP it should do "same as Parent"+something more. With super() you have the code which will work without changes if you add the missing (yet unused) method somewhere in the ancestors chain, if you change the ancestors chain and so on.