Why is starred iterable unpacking in a return statement invalid syntax without parentheses before Python 3.8?
I suspect this is an accident, based on the comments from this commit for Python 3.2.
That commit enabled the assignment expression to take a testlist_star_expr
production (what allows the unparenthesized unpacking), but left the return statement taking a testlist
production. I suspect the commit just missed this (and possibly other locations, but I'm focusing on the return_stmt
production for now).
I went ahead and modified the Python Grammar/Grammar file to allow this. All of the tests continue to pass, including those in the test_grammar.py
file (but this doesn't seem terribly exhaustive).
If you're curious, this is the change I made. Feel free to clone or download my fork.
UPDATE: I've submitted a bpo issue and a pull request for the return (and yield) unpacking.
Related videos on Youtube
Comments
-
jmd_dk almost 2 years
The Python language (especially 3.x) allows very general unpacking of iterables, a simple example of which is
a, *rest = 1, 2, 3
Over the years, this unpacking has been gradually generalized (see e.g. PEP 3132 and PEP 448), allowing it to be used in more and more circumstances. As so, I was surprised to discover that the following is invalid syntax in Python 3.6 (and remains so in Python 3.7):
def f(): rest = [2, 3] return 1, *rest # Invalid
I can make it work by encapsulating the returned tuple in parentheses like so:
def f(): rest = [2, 3] return (1, *rest) # Valid
The fact that I use this in a
return
statement seems to be important, ast = 1, *rest
is indeed legal and results in the same with and without parentheses.
Have this case simply been forgotten by the Python developers, or are there any reason why this case is invalid syntax?
Why I care
This breaks an important contract I thought I had with the Python language. Consider the following (also valid) solution:
def f(): rest = [2, 3] t = 1, *rest return t
Normally when I have code like this, I consider
t
to be a temporary name, which I ought to be able to get rid of simply be replacingt
in the bottom line with its definition. In this case though, this leads to the invalid codedef f(): rest = [2, 3] return 1, *rest
It's of course no big deal to have to place parentheses around the return value, but usually additional parentheses are only needed to distinguish between several possible outcomes (grouping). Here this is not the case, as leaving out the parentheses does not produce some other unwanted behavior, but rather no behavior at all.
Update
Since Python 3.8 (see item 7 on this list), the generalized syntax discussed above is now valid.
-
cs95 over 6 yearsThis is really more a consequence of the grammar syntax than anything else.
-
lapisdecor over 6 yearsYou can't also just return *rest, it's invalid syntax.
-
jmd_dk over 6 years@lapisdecor Yea, but that's consistent with the fact that
t = *rest
is invalid. Also,return *rest
andt = *rest
does not represent any actual unpacking, so I don't find it a problem that this is not allowed. If it were allowed,*rest
on its own would then only be a confusing syntax fortuple(rest)
. -
user2357112 over 6 yearsThis happens with more than just
return
. Unpackings are also forbidden in ayield
argument, a subscript, the RHS of an augmented assignment (but not a regular assignment), and on the right of thein
in afor
statement, despite unparenthesized tuples being allowed in all those positions, because the syntax for those things usesexpression_list
instead ofstarred_expression
. -
senderle over 6 yearsNote the difference between
t = *rest
andt = *rest,
. The latter is valid. -
Dan Boschen over 3 yearsFor the same reason of concern for consistency, why when we pass *x in isolation into a function it maps to a tuple in the function, but when we unpack a, *x = iterable it unpacks as a list to x, and it can't be done in isolation (I understand the return from the function maps to a tuple since a,b is a tuple parenthesis or not, but also why can't the return be done in isolation
return *x
??
-
-
Martijn Pieters over 6 yearsDid you make a pull request with this change yet?
-
David Cuthbert over 6 yearsNot yet. Wanted to see if this correctly solves what @jmd_dk was after, and maybe look at a few other cases (like the
yield_stmt
production) before sending it on. -
Chris_Rands over 6 yearsCreating a 'fix' seems a bit pre-mature, maybe submit a bug report first if you think this was an accident bugs.python.org
-
Michael Scott Asato Cuthbert over 5 yearsThe fix is set to appear in Python 3.8