Python optparse and spaces in an argument

Question

When using optparse i want to get the whole string after an option, but I only get part of it up to the first space.

e.g.:

python myprog.py --executable python someOtherProg.py

What I get in 'executable' is just 'python'.

Is it possible to parse such lines using optparse or do you have to use argparse to do it?

€: I have already tried enclosing it in "s. But after digging further into the code I found out that the subprocess invocation can't handle the argument.

The string with the commandline gets crammed into a list 'args'.

args = [self.getExecutable()] + self.getArgs().split()

It's like

"[python D:\\\workspace\\\myprog\\\src\\\myprog.py]"

That gives me the System can't find file exception. When I use

args[0]

it works. But I loose the arguments to the executable.

The subprocess module builds a cmdline from a list if it does not get a string in the first place, so I can't explain that behavior at the moment.

Solution 1

You can enclose them in quotes to make them work with the existing code.

python myprog.py --executable "python someOtherProg.py"

Is it possible to parse such lines using optparse or do you have to use argparse to do it?

I don't know if/how you can do it with optparse as I haven't really worked with optparse.

I can however help you out with argparse. Here is a quick example:

#!/usr/bin/python
import argparse, sys

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description = 'Demonstration of Argparse.')
    parser.add_argument('-e', '--executable', nargs = '+', help = 'List of executables')
    args = parser.parse_args(sys.argv[1:])
    print args.executable

And usage:

manoj@maruti:~$ python myprog.py --executable python someOtherProg.py
['python', 'someOtherProg.py']

I'd also recommend switching from optparse to argparse. Optparse is deprecated since 2.7.

Solution 2

I've found another good alternative shlex - A lexical analyzer class for simple shell-like syntaxes.

Source link: How to parse a command line with regular expressions?

>>> import shlex
>>> shlex.split('"param 1" param2 "param 3"')
['param 1', 'param2', 'param 3']
>>> shlex.split('"param 1" param2 "param 3"')
Traceback (most recent call last):
    [...]
ValueError: No closing quotation
>>> shlex.split('"param 1" param2 "param 3\\""')
['param 1', 'param2', 'param 3"']

Solution 3

The behavior you see comes from the fact that it's your shell, not python, that parses the command line and separates it into the words of sys.argv. Python is launched by the shell via exec() with argv already populated.

Most shells will split argv items at spaces unless you tell them not to by quoting or escaping.

Quotes work as described above.

In many shells you could do this:

python myprog.py --executable python\ someOtherProg.py

The backslash escapes the following space without requiring quotes.

Solution 4

If you know how many words after the argument flag you are going to get, you can modify the way you create the --executable option in in optparse to properly handle the situation:

Instead of taking a single word after the option flag you can set the optparse parser to look for two (or more) words:

from optparse import OptionParser
parser = OptionParser()

parser.add_option("-f", "--file", action="store", dest="filename",
                       help="File to be processed.", metavar="FILE")
parser.add_option("-e", "--executable", action="store", dest="my_exe",
                       help="Command to be executed", metavar="EXE",
                       nargs=2)

In this snippet, the -f or --file option only expects a single word and stores it as a string (the default) in the filename variable.

In contrast the -e, --executable option expects two words because of the nargs=2 option. This will result in the two words found behind the -e or --executable flag to be stored as strings in a Python list my_exe.

Check out: http://docs.python.org/library/optparse.html for more info on optparse, and remember it has been deprecated as of 2.7 in favour of argparse.

Solution 5

Just to finalize this answer list if you cannot upgrade to argparse.

Optparse is not able to handle these situations (multiple strings). You can only use nargs to specify particular amount of valiables, but there is nothing like "one or more". You need to hack it or use different library (e.g. argparse or other).

View more solutions

Comments

[shahjapan]

can you tell me how it can be handled using argparse ? w/o using sys.argv I guess argparse too doesn't have feature like to parse_string instead parse_args, e.g. to parse string having spaces like "one or more"

[lzap]

argparse is limited too, I don't have much experience with it :-(

[shahjapan]

see another solution provided by me, should work with argparse as well as optparse as its independent by using shlex lib.

[Urhixidur]

Prepare to be stunned: under Python 3.5, an escaped space is unescaped by sys.argv ! In C, we have silly python some : argv[1] = "python", argv[2] = "some". silly "python some" : argv[1] = "python some". And silly python\ some : argv[1] = "python some". But with Python 3.5 we have silly.py python some : argv[1] = "python", argv[2] = "some". silly.py "python some" : argv[1] = "python some". And silly.py python\ some : argv[1] = "python\\", "some". Amazing!