Python dependencies between groups using argparse

Question

I started to learn Python, and now I'm learning the great benefits of argparse. Using argparse, I have created two groups of arguments: group_list and group_simulate. Each of the groups has its own arguments -- the user can specify only one argument in each group (achieved using parser.add_mutually_exclusive_group()).

And now my target is present a syntax error if the user specified arguments from both groupgs and not from only one of them -- I want to achieve this by using the capabilities of argparse and not by writing a method that asks if this and this was specified print syntax error.

import argparse
parser = argparse.ArgumentParser(
        description='this is the description',
        epilog="This is the epilog",
        argument_default=argparse.SUPPRESS  
        )

parser.add_argument('-v', '--verbose', help='verbose', action='store_true', default=False)

group_list = parser.add_mutually_exclusive_group()
group_list.add_argument('-m', help='list only modules', action='store_const', dest='list', const='modules', default='all')
group_list.add_argument('-p', help='list only ports', action='store_const', dest='list', const='ports', default='all')
group_list.add_argument('--list', help='list only module or ports', choices=['modules','ports'], metavar='<modules/ports>', default='all')

group_simulate = parser.add_mutually_exclusive_group()
group_simulate.add_argument('-M', help='simulate module down', nargs=1, metavar='module_name', dest='simulate')
group_simulate.add_argument('-P', help='simulate FC port down', nargs=1, metavar='fc_port_name', dest='simulate')
group_simulate.add_argument('-I', help='simulate iSCSI port down', nargs=1, metavar='iSCSI_port_name', dest='simulate')
group_simulate.add_argument('--simulate', help='simulate module or port down', nargs=1, dest='simulate')

args = parser.parse_args()

print args

So talking more specifically:

allowed:

test.py
output: Namespace(list='all', verbose=False)
test.py -m
output: Namespace(list='modules', verbose=False)
test.py -P asfasf
output: Namespace(P=['asfasf'], list='all', verbose=False)

not allowed:

test.py -m -P asfsaf
expected output: <the help message>
test.py -P asfasf -m
expected output: <the help message>

I have tried to achieve the wanted target with the option of add_subparsers from argparse but without any success.

So my question is how to achieve this situation?

Solution 1

You can use a common mutually-exclusive-group as "root" of the two subgroups:

import argparse
parser = argparse.ArgumentParser(
        description='this is the description',
        epilog="This is the epilog",
        argument_default=argparse.SUPPRESS  
        )

parser.add_argument('-v', '--verbose', help='verbose', action='store_true', default=False)

root_group = parser.add_mutually_exclusive_group()

group_list = root_group.add_mutually_exclusive_group()
group_list.add_argument('-m', help='list only modules', action='store_const', dest='list', const='modules', default='all')
group_list.add_argument('-p', help='list only ports', action='store_const', dest='list', const='ports', default='all')
group_list.add_argument('--list', help='list only module or ports', choices=['modules','ports'], metavar='<modules/ports>', default='all')

group_simulate = root_group.add_mutually_exclusive_group()
group_simulate.add_argument('-M', help='simulate module down', nargs=1, metavar='module_name', dest='simulate')
group_simulate.add_argument('-P', help='simulate FC port down', nargs=1, metavar='fc_port_name', dest='simulate')
group_simulate.add_argument('-I', help='simulate iSCSI port down', nargs=1, metavar='iSCSI_port_name', dest='simulate')
group_simulate.add_argument('--simulate', help='simulate module or port down', nargs=1, dest='simulate')

args = parser.parse_args()

print args

Result:

$ python test.py -m -P asfafs
usage: test.py [-h] [-v] [[-m | -p | --list <modules/ports>]
                [-M module_name | -P fc_port_name | -I iSCSI_port_name | --simulate SIMULATE]
test.py: error: argument -P: not allowed with argument -m 

$ python test.py -m -p
usage: test.py [-h] [-v] [[-m | -p | --list <modules/ports>]
                [-M module_name | -P fc_port_name | -I iSCSI_port_name | --simulate SIMULATE]
test.py: error: argument -p: not allowed with argument -m

Solution 2

Use Docopt! You shouldn't have to write a usage doc and then spend hours trying to figure out how to get argparse to create it for you. If you know POSIX you know how to interpret a usage doc because it is a standard. Docopt know how to interpret usage docs that same as you do. We don't need an abstraction layer.

I think the OP has failed to describe their own intentions based on what I read in their help text. I'm going to try and speculate what they are trying to do.

---

test.py

"""
usage: test.py [-h | --version]
       test.py [-v] (-m | -p)
       test.py [-v] --list (modules | ports)
       test.py [-v] (-M <module_name> | -P <fc_port_name> | -I <iSCSI_port_name>)

this is the description

optional arguments:
  -h, --help                show this help message and exit
  -v, --verbose             verbose
  -m                        list only modules (same as --list modules)
  -p                        list only ports   (same as --list ports)
  --list                    list only module or ports
  -M module_name            simulate module down
  -P fc_port_name           simulate FC port down
  -I iSCSI_port_name        simulate iSCSI port down

This is the epilog
"""

from pprint import pprint
from docopt import docopt

def cli():
    arguments = docopt(__doc__, version='Super Tool 0.2')
    pprint(arguments)

if __name__ == '__main__':
    cli()

While it would be possible to communicate all of the usage in a single line with complex nested conditionals, this is more legible. This is why docopt makes so much sense. For a CLI program you want to make sure you communicate to the user clearly. Why learn some obscure module syntax in the hope that you can convince it to create the communication to the user for you? Take the time to look at other POSIX tools with option rules similar to your needs and copy-pasta.

Comments

[Elia]

This is exactly what I was looking for! Thank you very much Bakuriu

[hpaulj]

Nesting groups like this works, but does not do anything significant. When an action is added to group_simulate it gets added to root_group as well. It is also added to parser and its optional arguments group. The net effect is that all 7 actions are mutually exclusive. Also the usage code does not handle nested groups. Note the [[ and the lack of | between the nested groups.

[hpaulj]

It turns out that while group_simulate has a container attribute that points to the root_group, root_group does not have a list of its nested groups. There is a _mutually_exclusive_groups attribute, but this is shared (same reference) among the parser and all groups (exclusive or not). bugs.python.org/issue10984 has a patch with a formatter that can display overlapping exclusive groups - but it show's each group independently, not nested.

[hpaulj]

What usage doc would produce this desired behavior? I'm not too impressed with the docopt sales pitches so far.

[Bruno Bronosky]

@hpaulj Because the OP accepted that answer, which as you pointed out has unbalanced brackets in the usage message and has the net effect of puttin all 7 into one group... I'm not sure what they REALLY want. It's hard to say, but I'll take a stab at it.

[hpaulj]

Is this modeled on argp with multiple independent parsers (each usage line handled by a different parser)? Python already has a parser based on getopt, which is much simpler.

[Bruno Bronosky]

@hpaulj I'm not sure how the internals work. I only dive into that when something doesn't work. What is argp? What could possibly be easier than writing the exact user message you want shown and having it interpreted directly?

[hpaulj]

I was trying to follow up on your claim that knowing POSIX argument handling was enough to write a good docopt usage. A search on 'posix arguments' lead me to a GNU C Library, which has 2 parsers, getopt and argp, gnu.org/software/libc/manual/html_node/Argp.html

[hpaulj]

I don't see anything in the conventions page like gnu.org/software/libc/manual/html_node/… about multiple usage lines or your xor ( | ) syntax. That looks like a docopt extension, not pure POSIX.