Rationale for supporting `foo(kw=1, *args)` - keyword arguments before positional unpackings

Ideas
1

In the following, the question of which parameters foo accepts is ignored, the point is only about the syntax of the call.

As far as I know, foo(kw=1, "pos") was never allowed in Python. The call syntax always rejected positional arguments following keyword arguments, and it’s exactly what the error says in that case in 3.11 : “SyntaxError: positional argument follows keyword argument”.

The order in which positional unpacking (*args) and keyword unpacking (**kwargs) can be passed in a call basically respects that : even though you can have several of those in a single call since PEP 448, the limitations on what argument ordering is allowed rely on the logic of “positional first, keyword after”.

Except that there is one counter-example : for some reason, Python implementations before and after PEP 448 allowed positional unpackings after keyword arguments, as long as they were placed before keyword unpackings.
In other words, foo(kw=1, "pos") and foo(**d, *t) are forbidden, but foo(kw=1, *t) is accepted.
It seems to be done on purpose, but no rationale is given about it in 448. The text and explanations legalize it quite clearly, and the pseudocode sort-of features it, but it’s not dwelled on much more than that (the PEP itself is not very long).
The “Disadvantages” section even says, quote, “The allowable orders for arguments in a function call are more complicated than before.” and goes on to give a quite complex explanation of the new rule, when “positionals come first and keywords later” would be much simpler.

As far as I can see, that syntax was made legal by PEP 448 and was untouched ever since.
EDIT : not true, it actually predated 448.

My question is, since it appears it was done on purpose, what is or was the rationale to allow this particular syntax ?

2 Likes
2

I worked on implementing PEP 448. If I remember correctly, Guido commented on the issue tracker about this. Like you, and me, he would have preferred to block keyword arguments before positional unpacking. However, foo(kw=1, *args, **kwargs) was allowed previous to PEP 448. We didn’t create this exception; we had to continue to allow it.

1 Like
3

Added this: Consider adding a rule to enforce positional unpacking always precede keyword arguments · Issue #4060 · charliermarsh/ruff · GitHub

4

Thanks for the clarification ! I updated parts of the OP reflecting this.
It seems then that this is simply an old idiosyncratic behavior which doesn’t really adapt to the current Python landscape.
I would consider it a mega-antipattern, but I’d be happy to hear positive opinions about it, if any.

5

If enough people feel strongly about this, it may be worth turning this into a syntax warning. Too bad Ruff doesn’t give statistics on prevalence.

1 Like
6

I would definitely agree about a syntax warning. The only obstacle I can see about deprecating it would be compatibility issues, and the usefulness of forbidding something without setting the place for another feature.
The only one I could imagine against a warning would be if it’s expensive to compute in non-trigger cases.

7

See also Mailman 3 Order of positional and keyword arguments - Python-Dev - python.org

The main issue to me is that it breaks the order of evaluating arguments. In all other cases arguments are evaluated from left to right, and in the case of a keyword argument followed an unpacked positional argument the order is broken.

>>> def f(*a, **kw): pass
... 
>>> f(kw=print(1), *[print(2)])
2
1
4 Likes
8

As for the mailman comments, which I think we can port here:
I disagree with some of what @pf_moore says, for reasons additional to @storchaka’s point about argument evaluation order. Take the following wrappers:

def p_wrapper(*args, **kw):
        return wrapped_fn(some_arg=1, *args, **kw)

def g_wrapper(*args, **kw):
        return wrapped_fn(*args, some_arg=1, **kw)

The second wrapper does exactly the same thing as the first one, while not relying on the questioned syntax. That’s my first point : this syntax is not necessary for any feature.

What’s more, the second wrapper is more explicit in that you can read from left to right, first the positional arguments then the keyword ones. This is an invariant of which this syntax is the only exception. If wrapped_fn takes *args, **kwargs and stores them somehow, you can read from the call, first what will end up in args, then what will end up in kwargs. It enables clarity and readability.

So, as a combination of those two points, this feature is the only degree of ordering liberty you have, when writing a call, which has zero impact behavior-wise. That makes it ambiguous : in the Django example quoted in the mailman thread, the contributor choosing to leave the code as it is is mistaken in thinking it changes anything feature-wise. That’s quite bad, in my opinion.

So, if we can all agree that this syntax is at best useless, why not take things slowly ? I propose we make it a SyntaxWarning, without committing to a future deprecation or even a DeprecationWarning, and see what comes out of it. If nobody brings forth valid complaints or use cases, it can be re-discussed in 3.13.
What do you all think ?

2 Likes
9

Opened as an issue.

Another example of ambiguity, because it doesn’t hurt (and it actually happened to me not so long ago):

class B(A): # class A's signature is unknown, on purpose
    def __init__(self, a, b, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.a = a
        self.b = b

class C(B):
    def __init__(self, *args, **kwargs):
        super().__init__(5, b=6, *args, **kwargs)

This is unstable code. If no positional is passed when calling C, it will work as intended, passing 5 as a and 6 as b. However, if any positional is passed when calling C, it will find itself in the *args, and the first one will be bound to B’s parameter b. But since b is also passed by keyword, it will be double-bound and the call will fail.
But if you scan the call arguments from left to right, you see the 5 for a, the 6 for b, and think it’s all right because every other Python call rule points you towards thinking that’s all you need to check.

TLDR : this syntax makes it dangerous to pass a pos-or-kw parameter a value by keyword.