Understanding nested list comprehension

Question

I want to understand nested list comprehension. Below, I listed a list comprehension expression and their for loop equivalent.
I wonder if my understanding is correct on those.

For example,

[(min([row[i] for row in rows]),max([row[i] for row in rows])) 
for i in range(len(rows[0]))]

is equivalent to

result=[]
for i in range(len(rows[0])):
  innerResult=[]
  for row in rows:
    innerResult.append(row[i])
  innerResult2=[]
  for row in rows:
    innerResult2.append(row[i])
  tuple=(min(innerResult), max(innerResult2))
  result.append(tuple)

If I may generalize, I guess

[exp2([exp1 for x in xSet]) for y in ySet]

form can be translated to the following. (I hope I'm correct on this)

result=[]
for y in ySet:
  innerResult =[]
  for x in xSet:
    innerResult.append(exp1)
  exp2Result = exp2(innerResult)
  result.append(exp2Result)

For simpler case,

[exp1 for x in xSet for y in ySet] 

is equal to

result=[] 
for x in xSet:
  for y in ySet: 
    result.append(exp1)

whereas,

[[exp1 for x in xSet] for y in ySet]

is equal to

result=[]
for y in ySet:
  innerResult=[]
  for x in xSet:
    innerResult.append(exp1)
  result.append(innerResult)

I asked a similar question on Equivalent for loop expression for complex list comprehension
The answers given there reconstruct the form after understanding what it does internally.
I'd like to know how it works systematically so I can apply the concept to other slightly varying examples.

Solution 1

The short answer is: yes, you are correct in your understanding.

There's only a catch: the way you normally use nested list comprehension in python code is to operate on multidimensional arrays.

A typical example is when you operate on matrices:

>>> matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> [[el - 1 for el in row] for row in matrix]
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]

As you can see the "nesting" works by operating on each dimension of the matrix.

In the examples you provided, it seems that ySet [unfortunate name btw, as sets are one of the types provided with python] is just a generic counter, which makes a bit harder to follow what is going on under the hood.

As for your first example:

>>> rows = ([1, 2, 3], [10, 20, 30])
>>> [(min([row[i] for row in rows]),max([row[i] for row in rows])) for i in range(len(rows[0]))]
[(1, 10), (2, 20), (3, 30)]

You might wish to look into the zip built-in function:

>>> zip(rows[0], rows[1])
[(1, 10), (2, 20), (3, 30)]

or for maximum brevity and elegance:

>>> zip(*rows)
[(1, 10), (2, 20), (3, 30)]

HTH!

Solution 2

Indeed, you are correct. This is described in detail in the Expressions section in the Python Language Reference.

Note especially the order of nesting of several fors in a single list comprehension, which is always left-to-right:

>>> matrix = [[1, 2], [3, 4]]
>>> [item for item in row for row in matrix] # oops!
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    [item for item in row for row in matrix]
NameError: name 'row' is not defined
>>> [item for row in matrix for item in row] # nesting is in left-to-right order
[1, 2, 3, 4]

Comments

[eugene]

I've looked at the link. Indeed, there's an explanation for [item for row in matrix for item in row] but couldn't find explanation for [[exp for item in row] for row in matrix] format. Am I missing something from the grammar listed there?

[taleinat]

You're not missing anything, but that's because your example doesn't use any more special grammar, it is just two simple list comprehensions! The first [exp for item in row] will create a list given a row. The "outer" list comprehension will create a list, in which each item is a list created by the "inner" list-comp, one for each row in the matrix.

[eugene]

Oh. As you pointed out in your answer, the order is important. And, I was wondering if there's a section on library reference which talks about ordering of [[list comprehension] list comprehension] format.

[taleinat]

There in no such section in the docs that I know of. The reason is that in a list comprehension, the expression evaluated for each item can be any valid Python expression. In this context, an "inner" list comprehension is just like any other Python expression.

[WestCoastProjects]

wow whee.. am I the only one surprised by this ordering: now an explanation why the macro level (matrix in this case..) occurs before the nested/micro level (row level..) might be in order..

[taleinat]

@javadba If I remember correctly, the reasoning is that it is like the order of nested for loops in code.

[WestCoastProjects]

That actually helps .. Guido's thinking is typically somewhere between 90 to 180 degrees to mine: no exception here.