List of lists changes reflected across sublists unexpectedly
Solution 1
When you write [x]*3
you get, essentially, the list [x, x, x]
. That is, a list with 3 references to the same x
. When you then modify this single x
it is visible via all three references to it:
x = [1] * 4
xs = [x] * 3
print(f"id(x): {id(x)}")
# id(x): 140560897920048
print(
f"id(xs[0]): {id(xs[0])}\n"
f"id(xs[1]): {id(xs[1])}\n"
f"id(xs[2]): {id(xs[2])}"
)
# id(xs[0]): 140560897920048
# id(xs[1]): 140560897920048
# id(xs[2]): 140560897920048
x[0] = 42
print(f"x: {x}")
# x: [42, 1, 1, 1]
print(f"xs: {xs}")
# xs: [[42, 1, 1, 1], [42, 1, 1, 1], [42, 1, 1, 1]]
To fix it, you need to make sure that you create a new list at each position. One way to do it is
[[1]*4 for _ in range(3)]
which will reevaluate [1]*4
each time instead of evaluating it once and making 3 references to 1 list.
You might wonder why *
can't make independent objects the way the list comprehension does. That's because the multiplication operator *
operates on objects, without seeing expressions. When you use *
to multiply [[1] * 4]
by 3, *
only sees the 1-element list [[1] * 4]
evaluates to, not the [[1] * 4
expression text. *
has no idea how to make copies of that element, no idea how to reevaluate [[1] * 4]
, and no idea you even want copies, and in general, there might not even be a way to copy the element.
The only option *
has is to make new references to the existing sublist instead of trying to make new sublists. Anything else would be inconsistent or require major redesigning of fundamental language design decisions.
In contrast, a list comprehension reevaluates the element expression on every iteration. [[1] * 4 for n in range(3)]
reevaluates [1] * 4
every time for the same reason [x**2 for x in range(3)]
reevaluates x**2
every time. Every evaluation of [1] * 4
generates a new list, so the list comprehension does what you wanted.
Incidentally, [1] * 4
also doesn't copy the elements of [1]
, but that doesn't matter, since integers are immutable. You can't do something like 1.value = 2
and turn a 1 into a 2.
Solution 2
size = 3
matrix_surprise = [[0] * size] * size
matrix = [[0]*size for _ in range(size)]
Live visualization using Python Tutor:
Solution 3
Actually, this is exactly what you would expect. Let's decompose what is happening here:
You write
lst = [[1] * 4] * 3
This is equivalent to:
lst1 = [1]*4
lst = [lst1]*3
This means lst
is a list with 3 elements all pointing to lst1
. This means the two following lines are equivalent:
lst[0][0] = 5
lst1[0] = 5
As lst[0]
is nothing but lst1
.
To obtain the desired behavior, you can use a list comprehension:
lst = [ [1]*4 for n in range(3) ]
In this case, the expression is re-evaluated for each n
, leading to a different list.
Solution 4
[[1] * 4] * 3
or even:
[[1, 1, 1, 1]] * 3
Creates a list that references the internal [1,1,1,1]
3 times - not three copies of the inner list, so any time you modify the list (in any position), you'll see the change three times.
It's the same as this example:
>>> inner = [1,1,1,1]
>>> outer = [inner]*3
>>> outer
[[1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1]]
>>> inner[0] = 5
>>> outer
[[5, 1, 1, 1], [5, 1, 1, 1], [5, 1, 1, 1]]
where it's probably a little less surprising.
Solution 5
my_list = [[1]*4] * 3
creates one list object [1,1,1,1]
in memory and copies its reference 3 times over. This is equivalent to obj = [1,1,1,1]; my_list = [obj]*3
. Any modification to obj
will be reflected at three places, wherever obj
is referenced in the list.
The right statement would be:
my_list = [[1]*4 for _ in range(3)]
or
my_list = [[1 for __ in range(4)] for _ in range(3)]
Important thing to note here is that the *
operator is mostly used to create a list of literals. Although 1
is immutable, obj = [1]*4
will still create a list of 1
repeated 4 times over to form [1,1,1,1]
. But if any reference to an immutable object is made, the object is overwritten with a new one.
This means if we do obj[1] = 42
, then obj
will become [1,42,1,1]
not as some may assume. This can also be verified:[42,42,42,42]
>>> my_list = [1]*4
>>> my_list
[1, 1, 1, 1]
>>> id(my_list[0])
4522139440
>>> id(my_list[1]) # Same as my_list[0]
4522139440
>>> my_list[1] = 42 # Since my_list[1] is immutable, this operation overwrites my_list[1] with a new object changing its id.
>>> my_list
[1, 42, 1, 1]
>>> id(my_list[0])
4522139440
>>> id(my_list[1]) # id changed
4522140752
>>> id(my_list[2]) # id still same as my_list[0], still referring to value `1`.
4522139440
Charles Anderson
40 years or so as a professional software engineer, working in computer-aided design, petrochemical automation, semiconductor manufacturing, and data management. Most recent experience in JavaScript / Node.js, Python, HTML/CSS, and C++. Also Git, Mercurial, Jenkins, JIRA/Crucible, Less, Stylus, XML, UML. Now retired!
Updated on July 15, 2022Comments
-
Charles Anderson almost 2 years
I needed to create a list of lists in Python, so I typed the following:
my_list = [[1] * 4] * 3
The list looked like this:
[[1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1]]
Then I changed one of the innermost values:
my_list[0][0] = 5
Now my list looks like this:
[[5, 1, 1, 1], [5, 1, 1, 1], [5, 1, 1, 1]]
which is not what I wanted or expected. Can someone please explain what's going on, and how to get around it?
-
Karl Knechtel over 2 yearsNote that the same logic applies to a list of dicts, because of the same fundamental problem of aliasing a mutable object. See stackoverflow.com/questions/46835197/… for a more specific question.
-
Karl Knechtel about 2 yearsAre there more specific questions for when the list of lists is created in other ways (but has the same problem)? For example, by using
.append
in a loop?
-
-
mipadi over 15 yearsYou can use the "is" operator to discover this. ls[0] is ls[1] returns True.
-
Allanqunzi about 9 yearsI am surprised that no body points out that, the answer here is misleading.
[x]*3
store 3 references like[x, x, x]
is only right whenx
is mutable. This does't work for e.g.a=[4]*3
, where aftera[0]=5
,a=[5,4,4].
-
spfront about 9 yearsTechnically, it's still correct.
[4]*3
is essentially equivalent tox = 4; [x, x, x]
. It's true, though, that this will never cause any problem since4
is immutable. Also, your other example isn't really a different case.a = [x]*3; a[0] = 5
won't cause problems even ifx
is mutable, since you're not modifyingx
, only modifyinga
. I wouldn't describe my answer as misleading or incorrect - you just can't shoot yourself in the foot if you're dealing with immutable objects. -
timgeb about 8 years@Allanqunzi you are wrong. Do
x = 1000; lst = [x]*2; lst[0] is lst[1]
->True
. Python does not distinguish between mutable and immutable objects here whatsoever. -
Sergiy Kolodyazhnyy about 7 yearsJust a small addition to the nice answer here: it's evident that you're dealing with same object if you do
id(lst[0][0])
andid(lst[1][0])
or evenid(lst[0])
andid(lst[1])
-
Ahmed Mohamed almost 7 yearsSo, why if we write matrix= [[x] * 2] doesn't make 2 elemnts for the same object like the example you describe, it seems to be the same concept, what am i missing?
-
nadrimajstor almost 7 years@AhmedMohamed Indeed it does make a list with two elements of the exact same object that
x
refers to. If you make a globally unique object withx = object()
and then makematrix = [[x] * 2]
these does come as true:matrix[0][0] is matrix[0][1]
-
Ahmed Mohamed almost 7 years@nadrimajstor so why the change in matrix[0] doesn't affect matrix[1] like the example above with 2d matrix.
-
nadrimajstor almost 7 years@AhmedMohamed Surprise come when you make a "copy" of mutable sequence (in our example it is a
list
) so if arow = [x] * 2
than amatrix = [row] * 2
where both rows are exactly the same object, and now changes to one rowmatrix[0][0] = y
suddenly reflect in the other one(matrix[0][0] is matrix[1][0]) == True
-
nadrimajstor almost 7 years@AhmedMohamed Take a look at Ned Batchelder - Facts and Myths about Python names and values as it might offer a better explanation. :)
-
Martijn Pieters almost 6 yearsIt's not about literals.
obj[2] = 42
replaces the reference at index2
, as opposed to mutating the object referenced by that index, which is whatmyList[2][0] = ...
does (myList[2]
is a list, and the assigment alters the reference at index 0 in tha list). Of course, integers are not mutable, but plenty of object types are. And note that the[....]
list display notation is also a form of literal syntax! Don't confuse compound (such as lists) and scalar objects (such as integers), with mutable vs. immutable objects. -
U12-Forward over 5 yearsNote, fourth step can be dropped if you make second step:
a.insert(0,[5,1,1,1])
-
ERJAN over 2 yearsit's kinda late, but is there a book where i can read about these subtleties of python?
-
Lei Yang about 2 yearscan anyone find documents about the
*
operator in docs.python.org? i tried but cnanot find any. -
Jasmijn about 2 years@LeiYang It's listed under Common Sequence Operations
-
JobHunter69 almost 2 yearsDoesn't explain why modifying a 1d list causes a copy while a 2d list doesn't cause any copy