Forced naming of parameters in Python

Question

In Python you may have a function definition:

def info(object, spacing=10, collapse=1)

which could be called in any of the following ways:

info(odbchelper)                    
info(odbchelper, 12)                
info(odbchelper, collapse=0)        
info(spacing=15, object=odbchelper)

thanks to Python's allowing of any-order arguments, so long as they're named.

The problem we're having is as some of our larger functions grow, people might be adding parameters between spacing and collapse, meaning that the wrong values may be going to parameters that aren't named. In addition sometimes it's not always clear as to what needs to go in. We're after a way to force people to name certain parameters - not just a coding standard, but ideally a flag or pydev plugin?

so that in the above 4 examples, only the last would pass the check as all the parameters are named.

Odds are we'll only turn it on for certain functions, but any suggestions as to how to implement this - or if it's even possible would be appreciated.

Solution 1

In Python 3 - Yes, you can specify * in the argument list.

From docs:

Parameters after “*” or “*identifier” are keyword-only parameters and may only be passed used keyword arguments.

>>> def foo(pos, *, forcenamed):
...   print(pos, forcenamed)
... 
>>> foo(pos=10, forcenamed=20)
10 20
>>> foo(10, forcenamed=20)
10 20
>>> foo(10, 20)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: foo() takes exactly 1 positional argument (2 given)

This can also be combined with **kwargs:

def foo(pos, *, forcenamed, **kwargs):

To complete example:

def foo(pos, *, forcenamed ):
    print(pos, forcenamed)

foo(pos=10, forcenamed=20)
foo(10, forcenamed=20)
# basically you always have to give the value!
foo(10)

output:

Traceback (most recent call last):
  File "/Users/brando/anaconda3/envs/metalearning/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3444, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-12-ab74191b3e9e>", line 7, in <module>
    foo(10)
TypeError: foo() missing 1 required keyword-only argument: 'forcenamed'

So you are forced to always give the value. If you don't call it you don't have to do anything else named argument forced.

Solution 2

You can force people to use keyword arguments in Python3 by defining a function in the following way.

def foo(*, arg0="default0", arg1="default1", arg2="default2"):
    pass

By making the first argument a positional argument with no name you force everyone who calls the function to use the keyword arguments which is what I think you were asking about. In Python2 the only way to do this is to define a function like this

def foo(**kwargs):
    pass

That'll force the caller to use kwargs but this isn't that great of a solution as you'd then have to put a check to only accept the argument that you need.

Solution 3

True, most programming languages make parameter order part of the function call contract, but this doesn't need to be so. Why would it? My understanding of the question is, then, if Python is any different to other programming languages in this respect. In addition to other good answers for Python 2, please consider the following:

__named_only_start = object()

def info(param1,param2,param3,_p=__named_only_start,spacing=10,collapse=1):
    if _p is not __named_only_start:
        raise TypeError("info() takes at most 3 positional arguments")
    return str(param1+param2+param3) +"-"+ str(spacing) +"-"+ str(collapse)

The only way a caller would be able to provide arguments spacing and collapse positionally (without an exception) would be:

info(arg1, arg2, arg3, module.__named_only_start, 11, 2)

The convention of not using private elements belonging to other modules already is very basic in Python. As with Python itself, this convention for parameters would only be semi-enforced.

Otherwise, calls would need to be of the form:

info(arg1, arg2, arg3, spacing=11, collapse=2)

A call

info(arg1, arg2, arg3, 11, 2)

would assign value 11 to parameter _p and an exception risen by the function's first instruction.

Characteristics:

• Parameters before _p=__named_only_start are admitted positionally (or by name).
• Parameters after _p=__named_only_start must be provided by name only (unless knowledge about the special sentinel object __named_only_start is obtained and used).

Pros:

• Parameters are explicit in number and meaning (the later if good names are also chosen, of course).
• If the sentinel is specified as first parameter, then all arguments need to be specified by name.
• When calling the function, it's possible to switch to positional mode by using the sentinel object __named_only_start in the corresponding position.
• A better performance than other alternatives can be anticipated.

Cons:

• Checking occurs during run-time, not compile-time.
• Use of an extra parameter (though not argument) and an additional check. Small performance degradation respect to regular functions.
• Functionality is a hack without direct support by the language (see note below).
• When calling the function, it's possible to switch to positional mode by using the sentinel object __named_only_start in the right position. Yes, this can also be seen as a pro.

Please do keep in mind that this answer is only valid for Python 2. Python 3 implements the similar, but very elegant, language-supported mechanism described in other answers.

I've found that when I open my mind and think about it, no question or other's decision seems stupid, dumb, or just silly. Quite on the contrary: I typically learn a lot.

Solution 4

You can do that in a way that works in both Python 2 and Python 3, by making a "bogus" first keyword argument with a default value that will not occur "naturally". That keyword argument can be preceded by one or more arguments without value:

_dummy = object()

def info(object, _kw=_dummy, spacing=10, collapse=1):
    if _kw is not _dummy:
        raise TypeError("info() takes 1 positional argument but at least 2 were given")

This will allow:

info(odbchelper)        
info(odbchelper, collapse=0)        
info(spacing=15, object=odbchelper)

but not:

info(odbchelper, 12)                

If you change the function to:

def info(_kw=_dummy, spacing=10, collapse=1):

then all arguments must have keywords and info(odbchelper) will no longer work.

This will allow you to position additional keyword arguments any place after _kw, without forcing you to put them after the last entry. This often makes sense, e.g. grouping thing logically or arranging keywords alphabetically can help with maintenance and development.

So there is no need to revert to using def(**kwargs) and losing the signature information in your smart editor. Your social contract is to provide certain information, by forcing (some of them) to require keywords, the order these are presented in, has become irrelevant.

Solution 5

The python3 keyword-only arguments (*) can be simulated in python2.x with **kwargs

Consider the following python3 code:

def f(pos_arg, *, no_default, has_default='default'):
    print(pos_arg, no_default, has_default)

and its behaviour:

>>> f(1, 2, 3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() takes 1 positional argument but 3 were given
>>> f(1, no_default='hi')
1 hi default
>>> f(1, no_default='hi', has_default='hello')
1 hi hello
>>> f(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() missing 1 required keyword-only argument: 'no_default'
>>> f(1, no_default=1, wat='wat')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() got an unexpected keyword argument 'wat'

This can be simulated using the following, note I've taken the liberty of switching TypeError to KeyError in the "required named argument" case, it wouldn't be too much work to make that the same exception type as well

def f(pos_arg, **kwargs):
    no_default = kwargs.pop('no_default')
    has_default = kwargs.pop('has_default', 'default')
    if kwargs:
        raise TypeError('unexpected keyword argument(s) {}'.format(', '.join(sorted(kwargs))))

    print(pos_arg, no_default, has_default)

And behaviour:

>>> f(1, 2, 3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() takes exactly 1 argument (3 given)
>>> f(1, no_default='hi')
(1, 'hi', 'default')
>>> f(1, no_default='hi', has_default='hello')
(1, 'hi', 'hello')
>>> f(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f
KeyError: 'no_default'
>>> f(1, no_default=1, wat='wat')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 6, in f
TypeError: unexpected keyword argument(s) wat

The recipe works equally as well in python3.x, but should be avoided if you are python3.x only

View more solutions

Comments

[ggorlen]

Naming a parameter object overwrites a builtin class. I'd suggest obj or something like that.

[Brandon]

I liked your old answer better. Just put a comment as to why you are only accepting **kwargs in the function. After all, anyone can change anything in source code -- you need documentation to describe intent and purpose behind your decisions.

[Eric]

"Checking occurs during run-time, not compile-time." - I think that's true of all function argument checking. Until you actually execute the line of the function invocation, you don't always know which function is being executed. Also, +1 - this is clever.

[Mario Rossi]

@Eric: It's just that I'd have preferred static checking. But you are right: that wouldn't have been Python at all. Though not a decisive point, Python 3's "*" construct is also dynamically checked. Thanks for your comment.

[Eric]

Also, if you name the module variable _named_only_start, it becomes impossible to reference it from an external module, which takes out a pro and a con. (single leading underscores at module scope are private, IIRC)

[Mario Rossi]

Regarding naming of the sentinel, we could also have both a __named_only_start and a named_only_start (no initial underscore), the second to indicate that named mode is "recommended", but not to the level of being "actively promoted" (as one is public and the other is not). Regarding the "privateness" of _names starting with underscores, it's not very strongly enforced by the language: it can be easily circumvented by the use of specific (non-*) imports or qualified names. This is why several Python documents prefer to use the term "non-public" instead of "private".

[Graeme Perrow]

This doesn't answer the question. Whether or not it's a good idea is irrelevant - someone might do it anyway.

[s0undt3ch]

As Graeme mentioned, someone will do it anyway. Also, if you're writing a library to be used by others, forcing(python 3 only) the passing of keyword only arguments allows extra flexibility when you have to refactor your API.

[Dagrooms]

And have no idea how to call your method without reading your code, increasing cognitive load on your consumer :(

[Dagrooms]

Not only do you have to add keyword checks, but think about a consumer that knows they have to call a method with the signature foo(**kwargs). What do I pass into that? foo(killme=True, when="rightnowplease")

[Noufal Ibrahim]

It depends really. Consider dict.

[Neil]

Ah, so kwargs.pop('foo') is a Python 2 idiom? I need to update my coding style. I was still using this approach in Python 3 🤔

[David S.]

Because of the mentioned reason this is really bad practice and should be avoided.

[Phil]

There's no actual answer in this answer!

[Phil]

what happens if spacing is desired to be 0? (answer, you get 10). This answer is as wrong as all the other **kwargs answers for all the same reasons.

[shahjapan]

@phil yes I understood for that I think we should check whether its None or not - have updated the answer.

[Nuclear03020704]

As @Phil says, as currently stands i.e. an update with the old answer effectively scratched, there is no actual answer in this post. Yeah, this post is over 10 years old though.. let the relics of time be at work.

[rkachach]

In addition to the response. You can consider also using Python typing hints feature to enforce your function signature. This way you can detect bad calls by using mypy checks for example.

[Jean Monet]

PEP 3102 – Keyword-Only Arguments: peps.python.org/pep-3102