Python convert args to kwargs

Question

I am writing a decorator that needs to call other functions prior to call of the function that it is decorating. The decorated function may have positional arguments, but the functions the decorator will call can only accept keyword arguments. Does anyone have a handy way of converting positional arguments into keyword arguments?

I know that I can get a list of the variable names of the decorated function:

>>> def a(one, two=2):
...    pass

>>> a.func_code.co_varnames
('one', 'two')

But I can't figure out how to tell what was passed in positionally and what was as keyword.

My decorator looks like this:

class mydec(object):
    def __init__(self, f, *args, **kwargs):
        self.f = f

    def __call__(self, *args, **kwargs):
        hozer(**kwargs)
        self.f(*args, **kwargs)

Is there a way other than just comparing kwargs and co_varnames, adding to kwargs anything not in there, and hoping for the best?

Solution 1

Note - co_varnames will include local variables as well as keywords. This probably won't matter, as zip truncates the shorter sequence, but may result in confusing error messages if you pass the wrong number of args.

You can avoid this with func_code.co_varnames[:func_code.co_argcount], but better is to use the inspect module. ie:

import inspect
argnames, varargs, kwargs, defaults = inspect.getargspec(func)

You may also want to handle the case where the function defines **kwargs or *args (even if just to raise an exception when used with the decorator). If these are set, the second and third result from getargspec will return their variable name, otherwise they will be None.

Solution 2

Any arg that was passed positionally will be passed to *args. And any arg passed as a keyword will be passed to **kwargs. If you have positional args values and names then you can do:

kwargs.update(dict(zip(myfunc.func_code.co_varnames, args)))

to convert them all into keyword args.

Solution 3

If you're using Python >= 2.7 inspect.getcallargs() does this for you out of the box. You'd just pass it the decorated function as the first argument, and then the rest of the arguments exactly as you plan to call it. Example:

>>> def f(p1, p2, k1=None, k2=None, **kwargs):
...     pass
>>> from inspect import getcallargs

I'm planning to do f('p1', 'p2', 'p3', k2='k2', extra='kx1') (note that k1 is being passed positionally as p3), so...

>>> call_args = getcallargs(f, 'p1', 'p2', 'p3', k2='k2', extra='kx1')
>>> call_args
{'p2': 'p2', 'k2': 'k2', 'k1': 'p3', 'p1': 'p1', 'kwargs': {'extra': 'kx1'}}

If you know the decorated function won't use **kwargs, then that key won't appear in the dict, and you're done (and I'm assuming there's no *args, since that would break the requirement that everything have a name). If you do have **kwargs, as I have in this example, and want to include them with the rest of the named arguments, it takes one more line:

>>> call_args.update(call_args.pop('kwargs'))
>>> call_args
{'p2': 'p2', 'k2': 'k2', 'k1': 'p3', 'p1': 'p1', 'extra': 'kx1'}

Update: for Python >= 3.3, see inspect.Signature.bind() and the related inspect.signature function for functionality similar to (but more robust than) inspect.getcallargs().

Solution 4

Well, this may be overkill. I wrote it for the dectools package (on PyPi), so you can get updates there. It returns the dictionary taking into account positional, keyword, and default arguments. There is a test suite in the package (test_dict_as_called.py):

def _dict_as_called(function, args, kwargs):
    """ return a dict of all the args and kwargs as the keywords they would
    be received in a real function call.  It does not call function.
    """

    names, args_name, kwargs_name, defaults = inspect.getargspec(function)
    
    # assign basic args
    params = {}
    if args_name:
        basic_arg_count = len(names)
        params.update(zip(names[:], args))  # zip stops at shorter sequence
        params[args_name] = args[basic_arg_count:]
    else:
        params.update(zip(names, args))    
    
    # assign kwargs given
    if kwargs_name:
        params[kwargs_name] = {}
        for kw, value in kwargs.iteritems():
            if kw in names:
                params[kw] = value
            else:
                params[kwargs_name][kw] = value
    else:
        params.update(kwargs)
    
    # assign defaults
    if defaults:
        for pos, value in enumerate(defaults):
            if names[-len(defaults) + pos] not in params:
                params[names[-len(defaults) + pos]] = value
            
    # check we did it correctly.  Each param and only params are set
    assert set(params.iterkeys()) == (set(names)|set([args_name])|set([kwargs_name])
                                      )-set([None])
    
    return params

Solution 5

Nadia's answer is correct, but I feel like a working demo of that answer is useful.

def decorator(func):
    def wrapped_func(*args, **kwargs):
        kwargs.update(zip(func.__code__.co_varnames, args))
        print(kwargs)
        return func(**kwargs)
    return wrapped_func

@decorator
def thing(a,b):
    return a+b

Given this decorated function, the following calls return the appropriate answer:

thing(1, 2)  # prints {'a': 1, 'b': 2}  returns 3
thing(1, b=2)  # prints {'b': 2, 'a': 1}  returns 3
thing(a=1, b=2)  # prints {'a': 1, 'b': 2}  returns 3

Note however that things start getting weird if you start nesting decorators because the decorated function now no longer takes a and b, it takes args and kwargs:

@decorator
@decorator
def thing(a,b):
    return a+b

Here thing(1,2) will print {'args': 1, 'kwargs': 2} and error with TypeError: thing() got an unexpected keyword argument 'args'

View more solutions