Passing a List to Python From Command Line

Question

I would like to make my python script run from the command line when supplies with some arguments. However, one of the arguments should be a list of options specific to one segment of the script. Would string parsing be the only way to do this by actually constructing the list after the "command line list" string is split from commas? If so, how would you go about that?

Example: -details=['name', 'title', 'address']

Solution 1

Program:

import sys, ast, getopt, types

def main(argv):            
    arg_dict={}
    switches={'li':list,'di':dict,'tu':tuple}
    singles=''.join([x[0]+':' for x in switches])
    long_form=[x+'=' for x in switches]
    d={x[0]+':':'--'+x for x in switches}
    try:            
        opts, args = getopt.getopt(argv, singles, long_form)
    except getopt.GetoptError:          
        print "bad arg"                       
        sys.exit(2)       

    for opt, arg in opts:        
        if opt[1]+':' in d: o=d[opt[1]+':'][2:]
        elif opt in d.values(): o=opt[2:]
        else: o =''
        print opt, arg,o
        if o and arg:
            arg_dict[o]=ast.literal_eval(arg)

        if not o or not isinstance(arg_dict[o], switches[o]):    
            print opt, arg, " Error: bad arg"
            sys.exit(2)                 

    for e in arg_dict:
        print e, arg_dict[e], type(arg_dict[e])        

if __name__ == '__main__':
    main(sys.argv[1:])        

Command line:

python py.py --l='[1,2,3,[1,2,3]]' -d "{1:'one',2:'two',3:'three'}" --tu='(1,2,3)'

Output:

args:  ['--l=[1,2,3,[1,2,3]]', '-d', "{1:'one',2:'two',3:'three'}", '--tu=(1,2,3)']
tu (1, 2, 3) <type 'tuple'>
di {1: 'one', 2: 'two', 3: 'three'} <type 'dict'>
li [1, 2, 3, [1, 2, 3]] <type 'list'>

This code snippet will take short or long command switches like -l or --li= and parse the text after the switch into a Python data structure like a list, tuple or a dict. The parsed data structure ends up in a dictionary with the long-form switch key.

Using ast.literal_eval is relatively safe. It can only parse python data definitions.

Solution 2

argparse is nice for this, it's in the standard library as of 2.7 and 3.2 but otherwise a pip install away.

Your main concern of specifying a variable-length list can be addressed by making the list interpreted as a single argument in the shell by using quotes (could depend on your shell I suppose):

% python prog.py 'name title address' spam

where prog.py contains

import sys
my_list = sys.argv[1].split() 
# my_list is ['name', 'title', 'address']
if 'name' in my_list:
   do_something()

or similar. Use an argument with split to delimit your list:

% python prog.py "you're a foo, lift the bar"

my_list = [x.strip() for x in  sys.argv[1].split(',')]
# my_list is ["you're a foo", "lift the bar"]

But please use argparse instead; especially if you want to use use -c style flags.

One way to interpret your question is:

"I'm already using argparse, since that's the sensible way to interpret command line arguments in Python. How do I specify that some options are within a specific category?"

In your question you've shown an example of something the shells I use of would choke on;

% python prog.py -v -details=['name', 'title', 'address'] --quickly -t 4

wouldn't make it to python to be parsed because they'd use spaces to separate arguments and might use [ and ] as shell syntax.

I suggest the following instead

% python prog.py -v --details name title address --quickly -t 4

where a prog.py file of

import argparse

parser = argparse.ArgumentParser() 
parser.add_argument('-v', action='store_true')
parser.add_argument('--details', nargs='*')
parser.add_argument('--quickly', action='store_true')
parser.add_argument('-t')

args = parser.parse_args()
#args is Namespace(details=['asdf', 'a', 'a'], quickly=False, t='4', v=True)
details = args.details
#details is ['asdf', 'a', 'a']

Now, as per your question, you didn't have to do the string parsing yourself.

Solution 3

Yes, argparse is your best bet, and if you want to provide a list of values to one of your named arguments, it looks like this (the nargs parameter is the key to this):

>>> import argparse
>>> arg_parser = argparse.ArgumentParser()
>>> arg_parser.add_argument('--details',
                            nargs='*',
                            type=str,
                            default=[],
                            help='a list of the details')

# your args on the command line like this example
>>> the_args = arg_parser.parse_args("--details 'name' 'title' 'address'".split())
>>> print the_args.details
["'name'", "'title'", "'address'"])

Comments

[phooji]

docs.python.org/library/argparse.html#module-argparse

[Thomas]

Is your question how to go about "constructing the list after the "command line list" string is split from commas?" That's how I've interpreted your question, please clarify

[Serdalis]

It appears he wants to pass in a python list from command line.

[Charlie Parker]

did you ever find a solution built in to the argparse command library or something like that, that doesn't require potentially hacky custom code?

[Ignacio Vazquez-Abrams]

Yes, the asker has already figured out the .split() bit, based on the question. It is the next step that is causing problems.

[Thomas]

Hm, reading the question again I agree, what the asker is missing is unclear.

[Charlie Parker]

why do you have the x.strip()?

[Thomas]

@CharlieParker looks like I wanted to get rid of the leading space in " lift the bar" that would otherwise have been there