Unpacking arguments: only named arguments may follow *expression

Question

The following works beautifully in Python:

def f(x,y,z): return [x,y,z]

a=[1,2]

f(3,*a)

The elements of a get unpacked as if you had called it like f(3,1,2) and it returns [3,1,2]. Wonderful!

But I can't unpack the elements of a into the first two arguments:

f(*a,3)

Instead of calling that like f(1,2,3), I get "SyntaxError: only named arguments may follow *expression".

I'm just wondering why it has to be that way and if there's any clever trick I might not be aware of for unpacking arrays into arbitrary parts of argument lists without resorting to temporary variables.

Solution 1

As Raymond Hettinger's answer points out, this may change has changed in Python 3 and here is a related proposal, which has been accepted. Especially related to the current question, here's one of the possible changes to that proposal that was discussed:

Only allow a starred expression as the last item in the exprlist. This would simplify the unpacking code a bit and allow for the starred expression to be assigned an iterator. This behavior was rejected because it would be too surprising.

So there are implementation reasons for the restriction with unpacking function arguments but it is indeed a little surprising!

In the meantime, here's the workaround I was looking for, kind of obvious in retrospect:

f(*(a+[3]))

Solution 2

It doesn't have to be that way. It was just rule that Guido found to be sensible.

In Python 3, the rules for unpacking have been liberalized somewhat:

>>> a, *b, c = range(10)
>>> a
0
>>> b
[1, 2, 3, 4, 5, 6, 7, 8]
>>> c
9

Depending on whether Guido feels it would improve the language, that liberalization could also be extended to function arguments.

See the discussion on extended iterable unpacking for some thoughts on why Python 3 changed the rules.

Solution 3

Thanks to the PEP 448 - Additional Unpacking Generalizations,

f(*a, 3)

is now accepted syntax starting from Python 3.5. Likewise you can use the double-star ** for keyword argument unpacking anywhere and either one can be used multiple times.

Solution 4

f is expecting 3 arguments (x, y, z, in that order).

Suppose L = [1,2]. When you call f(3, *L), what python does behind the scenes, is to call f(3, 1, 2), without really knowing the length of L.

So what happens if L was instead [1,2,3]?

Then, when you call f(3, *L), you'll end up calling f(3,1,2,3), which will be an error because f is expecting exactly 3 arguments and you gave it 4.

Now, suppose L=[1,2]1. Look at what happens when you callf`:

>>> f(3,*L) # works fine
>>> f(*L) # will give you an error when f(1,2) is called; insufficient arguments

Now, you implicitly know when you call f(*L, 3) that 3 will be assigned to z, but python doesn't know that. It only knows that the last j many elements of the input to f will be defined by the contents of L. But since it doesn't know the value of len(L), it can't make assumptions about whether f(*L,3) would have the correct number of arguments.

This however, is not the case with f(3,*L). In this case, python knows that all the arguments EXCEPT the first one will be defined by the contents of L.

But if you have named arguments f(x=1, y=2, z=3), then the arguments being assigned to by name will be bound first. Only then are the positional arguments bound. So you do f(*L, z=3). In that case, z is bound to 3 first, and then, the other values get bound.

Now interestingly, if you did f(*L, y=3), that would give you an error for trying to assign to y twice (once with the keyword, once again with the positional)

Hope this helps

Solution 5

You can use f(*a, z=3) if you use f(*a, 3), it do not know how to unpack the parameter for you provided 2 parameters and 2 is the second.

View more solutions

Comments

[Antti Haapala -- Слава Україні]

f(*a, 3) now works in Python 3.5

[dreeves]

I think I may disagree with this, in particular the "python doesn't know that" part. As Raymond Hettinger says, it seems to be an arbitrary restriction. I could imagine an efficiency argument against the liberalization though.

[dreeves]

Interesting! Thanks! I didn't mean to phrase it as "have to" but was curious about the rationale. Sounds like it may have been rather arbitrary. As for a workaround, looks like apply(f, a+[3]) would do it, though I see that apply() is deprecated.

[inspectorG4dget]

Possible. I've shared this question on G+, tagged to Guido. Let's see if he says something

[dreeves]

Thanks again for the elucidating answer. See also my answer for more about this, and the workaround I was looking for.

[Hotschke]

And if a is tuple: f(*(a+(3,)))

[Yann Dubois]

I don't understand why this was downvoted, it's a very good answer for python 2. More readable than the *(a+(b,)) trick

[Friedrich]

The brackets are not needed. Thus for lists f(*a+[3]) and for tuples f(*a+(3,))