When is "i += x" different from "i = i + x" in Python?

Question

I was told that += can have different effects than the standard notation of i = i +. Is there a case in which i += 1 would be different from i = i + 1?

Solution 1

This depends entirely on the object i.

+= calls the __iadd__ method (if it exists -- falling back on __add__ if it doesn't exist) whereas + calls the __add__ method¹ or the __radd__ method in a few cases².

From an API perspective, __iadd__ is supposed to be used for modifying mutable objects in place (returning the object which was mutated) whereas __add__ should return a new instance of something. For immutable objects, both methods return a new instance, but __iadd__ will put the new instance in the current namespace with the same name that the old instance had. This is why

i = 1
i += 1

seems to increment i. In reality, you get a new integer and assign it "on top of" i -- losing one reference to the old integer. In this case, i += 1 is exactly the same as i = i + 1. But, with most mutable objects, it's a different story:

As a concrete example:

a = [1, 2, 3]
b = a
b += [1, 2, 3]
print a  #[1, 2, 3, 1, 2, 3]
print b  #[1, 2, 3, 1, 2, 3]

compared to:

a = [1, 2, 3]
b = a
b = b + [1, 2, 3]
print a #[1, 2, 3]
print b #[1, 2, 3, 1, 2, 3]

notice how in the first example, since b and a reference the same object, when I use += on b, it actually changes b (and a sees that change too -- After all, it's referencing the same list). In the second case however, when I do b = b + [1, 2, 3], this takes the list that b is referencing and concatenates it with a new list [1, 2, 3]. It then stores the concatenated list in the current namespace as b -- With no regard for what b was the line before.

---

^{1In the expression x + y, if x.__add__ isn't implemented or if x.__add__(y) returns NotImplemented and x and y have different types, then x + y tries to call y.__radd__(x). So, in the case where you have}

^{foo_instance += bar_instance}

^{if Foo doesn't implement __add__ or __iadd__ then the result here is the same as}

^{foo_instance = bar_instance.__radd__(bar_instance, foo_instance)}

^{2In the expression foo_instance + bar_instance, bar_instance.__radd__ will be tried before foo_instance.__add__ if the type of bar_instance is a subclass of the type of foo_instance (e.g. issubclass(Bar, Foo)). The rationale for this is that Bar is in some sense a "higher-level" object than Foo so Bar should get the option of overriding Foo's behavior.}

Solution 2

Under the covers, i += 1 does something like this:

try:
    i = i.__iadd__(1)
except AttributeError:
    i = i.__add__(1)

While i = i + 1 does something like this:

i = i.__add__(1)

This is a slight oversimplification, but you get the idea: Python gives types a way to handle += specially, by creating an __iadd__ method as well as an __add__.

The intention is that mutable types, like list, will mutate themselves in __iadd__ (and then return self, unless you're doing something very tricky), while immutable types, like int, will just not implement it.

For example:

>>> l1 = []
>>> l2 = l1
>>> l1 += [3]
>>> l2
[3]

Because l2 is the same object as l1, and you mutated l1, you also mutated l2.

But:

>>> l1 = []
>>> l2 = l1
>>> l1 = l1 + [3]
>>> l2
[]

Here, you didn't mutate l1; instead, you created a new list, l1 + [3], and rebound the name l1 to point at it, leaving l2 pointing at the original list.

(In the += version, you were also rebinding l1, it's just that in that case you were rebinding it to the same list it was already bound to, so you can usually ignore that part.)

Solution 3

Here is an example that directly compares i += x with i = i + x:

def foo(x):
  x = x + [42]

def bar(x):
  x += [42]

c = [27]
foo(c); # c is not changed
bar(c); # c is changed to [27, 42]

Comments

[Ashwini Chaudhary]

+= acts like extend() in case of lists.

[kojiro]

@AshwiniChaudhary That's a pretty subtle distinction, considering that i=[1,2,3];i=i+[4,5,6];i==[1,2,3,4,5,6] is True. Many developers may not notice that id(i) changes for one operation, but not the other.

[mgilson]

@kojiro -- While it's a subtle distinction, I think it is an important one.

[kojiro]

@mgilson it is important, and so I felt it needed an explanation. :)

[jakub.g]

Related question regarding differences between the two in Java: stackoverflow.com/a/7456548/245966

[Martijn Pieters]

possible duplicate of What does plus equals (+=) do in Python?

[abarnert]

Well, += calls __iadd__ if it exists, and falls back to adding and rebinding otherwise. That's why i = 1; i += 1 works even though there's no int.__iadd__. But other than that minor nit, great explanations.

[mgilson]

does __iadd__ actually call __add__ in the event of an AttributeError?

[abarnert]

Well, i.__iadd__ doesn't call __add__; it's i += 1 that calls __add__.

[mgilson]

errr... Yeah, that's what I meant. Interesting. I didn't realize that was done automatically.

[lvc]

The first attempt is actually i = i.__iadd__(1) - iadd can modify the object in place, but doesn't have to, and so is expected to return the result in either case.

[abarnert]

Note that this means that operator.iadd calls __add__ on AttributeError, but it can't rebind the result… so i=1; operator.iadd(i, 1) returns 2 and leaves i set to 1. Which is a bit confusing.

[mgilson]

@abarnert -- I always assumed that int.__iadd__ just called __add__. I'm glad to have learned something new today :).

[abarnert]

@lvc: You're right, but that makes it even more confusing. You can use even return something other than self. Let me see how to make that clear in the answer.

[abarnert]

@JBernardo: Yes, += always rebinds the variable, but operator.iadd does not. That's why the docs explicitly say "a = iadd(a, b) is equivalent to a += b, not iadd(a, b) is equivalent to a += b.

[mgilson]

@abarnert -- I suppose maybe to be complete, x + y calls y.__radd__(x) if x.__add__ doesn't exist (or returns NotImplemented and x and y are of different types)

[abarnert]

If you really want to be completist, you'd have to mention that the "if it exists" bit goes through the usual getattr mechanisms, except for some quirks with classic classes, and for types implemented in the C API it instead looks for either nb_inplace_add or sq_inplace_concat, and those C API functions have stricter requirements than the Python dunder methods, and… But I don't think that's relevant to the answer. The main distinction is that += tries to do an in-place add before falling back to acting like +, which I think you've already explained.

[mgilson]

Yeah, I suppose you're right ... Although I could just fall back on the stance that the C API isn't part of python. It's part of Cpython :-P

[wim]

To further complicate the comments discussion: x + y tries type(y).__radd__ first if issubclass(type(y), type(x)).

[mgilson]

@wim -- Thanks for this comment, I've added an additional paragraph at the bottom...

[Kindred]

What do you mean ["] on top of i... [."]?

[mgilson]

I mean that the new object is going to be assigned to the name i -- Whatever was there previously will be one step closer to being garbage collected.

[Kelly Bundy]

@wim That doesn't seem quite right, either, since a class is considered a subclass of itself (I guess just like a set is a subset of itself and a string is a substring of itself) and doing x + y with equal types tries __add__ first.

[wim]

@HeapOverflow It's odd that a class is considered subclass of itself, I don't understand the reasoning for that. I should qualify as strict subclass, of course. Your point also reveals a documentation bug in the note here.

[wim]

Added github.com/python/cpython/pull/22257 for the docs bug.