Argparse: Multiple arguments with the same destination left-to-right order guarantee

Python Help
,
1

Reading the argparse docs, I didn’t find a formal guarantee that arguments are parsed from left-to-right, or “in order”, which is especially useful for a “last argument wins” approach when using the same destination.

For example:

>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo', action='store_const', const='foo', dest='val')
>>> parser.add_argument('--bar', action='store_const', const='bar', dest='val')
>>> parser.parse_args(['--bar', '--foo'])
Namespace(val='foo')
>>> parser.parse_args(['--bar', '--foo', '--bar'])
Namespace(val='bar')

This works intuitively, but isn’t documented. Is it simply missing a note in the documentation, or isn’t it safe to assume?

2 Likes
2

I would just do a test with a short program. I even have a program dir called “test” just for tests like this. Like this: python app.py --arg1:100 --arg1:200

A gotcha in Python 3.14: (long) arguments with a single dash are processed differently now. I think that started in 3.13. So I’m converting all my long arguments to two dashes --.

3

I have tested it, and it works as expected, that’s not the issue.

For context, I suggested in a PR review for an open source project (runners: openocd: add BIN file flash support by sylvioalves · Pull Request #101888 · zephyrproject-rtos/zephyr · GitHub) to use this mechanism, but it was commented that this isn’t officially documented, and therefor we shouldn’t rely on it.

1 Like
4

I see, you want a formal guarantee. I suggest you find the Github page and create a Feature Request for the docs.

1 Like
5

Although when present, args is typed as Sequence[str] in the typeshed (i.e. not Iterable), calling it with an iterator of strings, results in parsing them ‘L’ → ‘R’, so .parse_args just iterates over args.

>>> parser.parse_args((s for s in ['--bar', '--foo', '--bar']))
Namespace(val='bar')

If this is intentional (as it would appear from the source code arg_strings_iter = iter(arg_strings) plus an args.extend call) then it should be straightforward to request a change of type signature on Typeshed.

Personally I think reversing them first (R → L) is quite surprising. It should be documented clearly, if this feature is ever added. But I noticed:

POSIX recommends these conventions for command line arguments
…

1 Like
6

Thank you @JamesParrott. Short of being documented, it’s great to see this is at least intentional in the code.

I was more concerned about a top-level iteration based on something else, like the add_argument("-x",...)s for instance. Hopefully that does not make sense and will never happen.

7

What exactly is the difference?

8

I fixed my statement to say “Long arguments with one dash are processed differently.” An argument like “-version” won’t get passed to the program itself, it gets passed to Python I think. I had to fix all my programs to use two dashes with long arguments, so “-version” became “–version”.

9

Could you show an example?

10

Just for the record, this was already discussed in the more specific BooleanOptionalActioncontext: `argparse.BooleanOptionalAction` does not yield mutually exclusive options · Issue #119375 · python/cpython · GitHub

Like it or not, it is not unusual for build systems to have many “layers”/”overlays” and then have more specific layers overriding less specific layers with a latter --feature overriding an earlier --no-feature or vice-versa.

Don’t get me wrong: this is very far from “standard practice”. The devil is in the details! But it’s not unusual. And when conflicting options are not rejected, I don’t remember any example where an earlier one wins.

11

So I’m not entirely sure if this is an actual solution to my question, or if this implies additional actions, such as updating the documentation?

12

It’s strong informal reassurance, not a formal guarantee.

If the order is ever reversed, all the Python CLI apps out there accepting multiple arguments will break. I see no appetite whatsoever to deal with a back lash that big, simply to save the few people that need multiple args processing the other way round to normal iteration, from adding in reversed to their code.

1 Like
13

The parse_args doc says " args - List of strings to parse. …" The proposal is to add something like ‘, in normal order’ to the end of the sentence. The reason is that parsing each arg has a side effect and that the net effect of parsing all args may depend on the order of parsing.

I suggest opening a feature request on the tracker. Reference this discussion. The consensus here, I believe, is that the behavior should never change, so that the order should be guaranteed. The main issue is whether it already is or not, which is to say, whether the addition would be redundant or not. Making the request minimal, such as 3 words, makes the bar for adding it relatively low. Let the few coredevs involved with argparse decide.

2 Likes