Python: How can I enable use of kwargs when calling from command line? (perhaps with argparse)

Question

suppose I have the module myscript.py; This module is production code, and is called often as %dir%>python myscript.py foo bar.

I want to extend it to take keyword arguments. I know that I can take these arguments using the script below, but unfortunately one would have to call it using

%dir%>python myscript.py main(foo, bar).

I know that I can use the argparse module, but I'm not sure how to do it.

import sys

def main(foo,bar,**kwargs):
    print 'Called myscript with:'
        print 'foo = %s' % foo
        print 'bar = %s' % bar
        if kwargs:
            for k in kwargs.keys():
                print 'keyword argument : %s' % k + ' = ' + '%s' % kwargs[k]   

if __name__=="__main__":
    exec(''.join(sys.argv[1:]))

Solution 1

If you want to pass in keyword arguments as you would in the main function, key=value, you can do it like so:

import sys

def main(foo, bar, *args):
    print "Called my script with"

    print "foo = %s" % foo
    print "bar = %s" % bar

    for arg in args:
        k = arg.split("=")[0]
        v = arg.split("=")[1]

        print "Keyword argument: %s = %s" % (k, v)


if __name__ == "__main__":
    if len(sys.argv) < 3:
        raise SyntaxError("Insufficient arguments.")
    if len(sys.argv) != 3:
        # If there are keyword arguments
        main(sys.argv[1], sys.argv[2], *sys.argv[3:])
    else:
        # If there are no keyword arguments
        main(sys.argv[1], sys.argv[2])

Some examples:

$> python my_file.py a b x=4
Called my script with
foo = a
bar = b
Keyword argument: x = 4
$> python my_file.py foo bar key=value
Called my script with
foo = foo
bar = bar
Keyword argument: key = value

However, this assumes that the key and value do not have any whitespace between them, key = value will not work.

If you are looking for --argument kinds of keyword arguments, you should use argparse.

Solution 2

@Moon beat me to it with a similar solution, but I'd suggest doing the parsing beforehand and passing in actual kwargs:

import sys

def main(foo, bar, **kwargs):
    print('Called myscript with:')
    print('foo = {}'.format(foo))
    print('bar = {}'.format(bar))
    for k, v in kwargs.items():
        print('keyword argument: {} = {}'.format(k, v))

if __name__=='__main__':
    main(sys.argv[1], # foo
         sys.argv[2], # bar
         **dict(arg.split('=') for arg in sys.argv[3:])) # kwargs

# Example use:
# $ python myscript.py foo bar hello=world 'with spaces'='a value'
# Called myscript with:
# foo = foo
# bar = bar
# keyword argument: hello = world
# keyword argument: with spaces = a value

Solution 3

First, you won't be passing an arbitrary Python expression as an argument. It's brittle and unsafe.

To set up the argument parser, you define the arguments you want, then parse them to produce a Namespace object that contains the information specified by the command line call.

import argparse
p = argparse.ArgumentParser()
p.add_argument('foo')
p.add_argument('bar')
p.add_argument('--add-feature-a', dest='a', action='store_true', default=False)

In your __main__ block, you'll parse the arguments, then pass a dictionary produced from the Namespace to main.

if __name__ == '__main__':
    args = p.parse_args()
    main(**vars(args))

Then you'll call your script with a line like

# foo = "3", bar = "6", a = True
python myscript.py 3 6 --add-feature-a

or

# foo = "moo", bar="7.7", a = False
python myscript.py moo 7.7

There's a lot more you can do with argparse, but this is a simple example for getting the value it produces into main.

Solution 4

in two lines of code I can get args and kwargs that I can manipulate like standard args and kwargs:

import sys


if __name__=='__main__':
  argv=argv[1:] 
  kwargs={kw[0]:kw[1] for kw in [ar.split('=') for ar in argv if ar.find('=')>0]}
  args=[arg for arg in argv if arg.find('=')<0]

  #and you can the use args and kwargs as so:
  if 'reset' in args:
    do_some_functions_with_reset()
  a_device_var=kwargs.get('device',False):
  #or whatever you want to do with args and kwargs

and the result is :

$python test.py reset device=foo format=1 bar
->args=['reset','bar']
->kwargs={'device':'foo','format':'1'}

View more solutions

Comments

[Ian]

You already seem to have provided the solution but want us to write the program for you. How about you first go to the documentation, try something and post again when you get stuck trying argparse.

[Moon Cheesez]

I don't think there is a need for argparse, I will come up with an answer soon.

[chepner]

There's no need for argparse, but it makes calling the script with optional arguments much easier.

[Dr. John A Zoidberg]

With my current solution, the main method needs to be called explicitly, I want to know how to do it without this limitation.

[Moon Cheesez]

What do you mean by keyword argument? How is it passed into the program?

[Vladimir Panteleev]

I'm not sure what interpretation of "keyword arguments" you are using in this answer, but it is not what the term generally refers to in Python.

[J.K]

In kwargs={'device':'foo','format':'1'}, "=" should be replaced by ":"