PEP 736: Keyword Argument Shorthand: Final Draft

PEPs
1

This is a discussion thread for PEP 736 which is currently being prepared for its final draft.

Previous threads have generated a lot of conversation already so it would be really appreciated if you could read the PEP and previous discussions [1][2] before contributing and confirm whether your suggestion has already been addressed satisfactorily.

Based on the previous thread, the following decisions were made to the PEP:

Autoformatters

Maintainers of black wrote that if this PEP were to be adopted, black would be adapted to autoformat function calls of the form f(x=x) to f(x=). There was significant disagreement as to whether this would be desirable. Prior to this, the draft of the PEP noted:

As with any other language feature, the programmer should exercise their own judgement about whether it is prudent to use it in any given context. We do not recommend enforcing a rule to use the feature in all cases where it may be applicable.

As described above, we propose that a reasonable rule of thumb would be to use this in cases where a parameter and its argument have the same semantics in order to reduce unintentional desynchronisation without causing inappropriate coupling.

We resolved to keep this text unchanged. As PEP authors, while we have our recommendations, we don’t think it’s our place to attempt to enforce philosophical rules about the language’s use. While this remains controversial, a lot of ink has been spilt on the issue and we’d encourage commenters not to add more without reviewing the previous thread to confirm that their point has not already been raised.

Highlighting exceptional arguments

Based in part on a blog post about the analogous feature added to Ruby, an additional advantage of this syntax was pointed out. Visual noise generated from keyword arguments being forwarded can give the impression that all the arguments are of this form. By applying this shorthand syntax, the exceptional arguments which are not of this form are more clearly emphasised.

How to Teach This section

The largest addition to the proposal was to expand the How to Teach This section. It now provides greater detail on how beginners and experienced practitioners can learn about the syntax and how to apply it. We additionally propose to name the syntax “keyword argument shorthand” to improve discoverability.

The AST

The implications that this syntax would have for the AST were clarified. The AST of the shorthand and traditional form would be identical, with the addition of a boolean flag attribute on AST nodes to indicate whether they are implicit in the code. This would enable, for example, linters to report on mixed styles.

IDEs

The conversation around IDEs was fleshed out and a section was added to the PEP with some recommendations to IDE maintainers.

18 Likes
2

Agreed, and I think the proposed text is perfectly acceptable for the PEP. We can spill more ink on the feature PRs for when autoformatters decide to actually do the thing we don’t like!

More broadly, I’m -0 on the idea, just because I don’t think it’s worth it. The readability arguments don’t convince me, and code is always read far more than it’s written, so write-ability doesn’t convince me either. I’m always happy to write more code if it’ll make it easier to read.

But the PEP itself is good :+1:

12 Likes
3

Just to be clear, this isn’t in the current draft. Should we be expecting to see a PR in the next few days?

4

I believe Joshua is referring to the Reference Implementation section.

We will extend this implementation to add an AST node attribute indicating for keywords whether the value was elided.

Admittedly, explicitly spelling out that the AST would be identical other than the added attribute might help clarify it.

2 Likes
5

I believe Joshua is referring to the Reference Implementation section.

Yep that’s right.

Admittedly, explicitly spelling out that the AST would be identical other than the added attribute might help clarify it.

Fair point, this could be a little clearer, thanks.

3 Likes
6

When considering language changes like this, I find it useful to play with the functionality in an editor. To facilitate such exploration and feedback from the community, I’ve implemented provisional support for draft PEP 736 in the latest version of pyright. If you’re not familiar with pyright, it’s both a static type checker (like mypy) and a language server (like jedi).

If you want to play with the proposed PEP 736 syntax, you can install the latest version of pyright (1.1.373) in your own editor or use the web-based pyright playground. You’ll need to tell pyright that it should assume Python version 3.14.

Here are a few of my own observations about the PEP 736 changes…

With today’s method of specifying keyword arguments (e.g. param1=param1), the left identifier unambiguously refers to the target parameter in the callee and the right identifier refers to a local symbol. These can have different static types, and hovering over them in a smart editor reveals these differences.

image image

Once these two identifiers are “fused” into one, it’s not as clear what should be displayed when hovering. In my implementation, I chose (somewhat arbitrarily) to display information about the local symbol.

image

Similarly, when renaming the identifier, it’s not immediately obvious whether the intent of the user is to rename the target parameter in the callee or the local symbol.

image image

A similar issue will exist for semantic highlighting, where tokens are colored according to their semantic meaning. It’s not clear what semantic meaning should be ascribed to the “fused” identifier.

So while the new syntax is less verbose (which can aid in readability), it adds a little ambiguity. I personally don’t find this to be a big problem.

Overall, I’m +0 on the proposal in the PEP, but I’m interested to hear what others think once they’ve had a chance to play with the feature.

If the PEP authors are looking for additional “prior art” examples, Javascript and TypeScript support a similar shortcut for object literals: { field1: 0, field2: null, field3 }. This tends to be used extensively in modern Javascript code.

13 Likes
7

Thanks Eric!

I think the point about search and replace refactoring now becoming a tad more difficult is worth recording as known within the PEP. More syntax context and expansion of the shorthand into the full keyword=value form is now required in such a refactoring for this situation.

As for what IDEs do with either highlighting or surfacing additional information to the user on this new intentionally sparse shorthand syntax, I might have chosen something merged such as (parameter) param1: float = (variable): int for the hover tip on the line so that discrepancies in expectations vs local type remain visible to the user. All in one place without having to painstakingly hover over two different UI elements and remember the others context in order to reveal the discrepency. BUT… this is beyond what we specify in a language PEP. What to do there is squarely up to tooling authors. Ex: It’s an existing UX choice to not show info about both the LHS and RHS of a kw=val parameter rather than separately in an IDE today. We don’t dictate what is “right” from our side.

I did note with some irony that one of the blog posts linked to had examples (was this in the Ruby one?) being rendered by some code prettifier that did not understand the new syntax and led to an amusing highlight choice alternating colors after every =,. Challenges like that will persist throughout the ecosystem no matter what upon syntax changes until such at a distance source tooling is upgraded and redeployed with knowledge of it. The same held true for the yield, async, await, and := additions from Pythons past.

I remain overall in favor of this PEP, despite my own initial internal “Eek, change! What will new coders think?” misgivings, because per the PEP it both (a) it has already proven useful in many other languages and (b) the prevalence of this code design pattern is large enough that many years from now when the world can use this modern syntax everywhere it’ll be looked upon fondly. This is the kind of user friendly carrot that improves the quality of developer life enough to make users want to upgrade.

It doesn’t aim to solve fundamental design choices when plumbing commonly named through a large call stack, but it helps make life involving that practical common design a little less painful in the face of reality.

11 Likes
8

Why not both? That’s my understanding of the vision of the PEP

9

Are match-case statements included in this change? I can’t find anything about them in the draft.

match myvar:
    # does this turn into MyClass(attr=attr)?
    case MyClass(attr=):
        ...

and also function and class definitions:

a = 123

def my_func(a=):
    ...

class MyClass(a=, metaclass=MyMeta):
    ...
5 Likes
10

Hi all,

this is the first time I contribute to a discussion about a PEP, I hope my suggestion will be appropriate.

I’ve read many concerns about how this PEP may create confusion as it’s hard to guess whether who wrote the code is deliberately using the shorthand or they forgot to specify the value to assign.

I agree with this and have a proposal that IMHO could make it easy to make the shortcut unanbiguous and hard to use it by mistake, while still providing the desired efficiency.

It is using the dot after the assignment . like so:

my_function(
    my_first_variable=.,
    my_second_variable=.,
    my_third_variable=.
)

AFAIK it matches with all the benefits already listed in the PEP: encourages use of named arguments, reduces verbosity, encourages consistent variable names, highlights arguments not following the pattern and it’s applicable to dictionary construction.

In addition, it uses more explicit code and it’s harder to mix it up with a plain missing variable error.

Maybe it’s to late to change the PEP, but I haven’t seen this mentioned in the various discussions and threads related to this topic and thought it would still be good to share this idea.

5 Likes
11

This is an interesting variation, but the dot is still pretty easy to miss. However, using @ could work:

my_function(
    my_first_variable=@,
    my_second_variable=@,
    my_third_variable=@
)

It’s using it as a mnemonic for “the value stored AT the local variable with the same name as the parameter”, rather than indicating any relationship with function decorators or matrix multiplication.

Tangent: if @ was adopted for this use case, then we could potentially also introduce @"" as a way to refer to the string form of assignment target names (as in MyTuple = namedtuple(@"", "a b c"))

6 Likes
13

I like this too! :+1:

1 Like
14

FWIW, I remain unconvinced (I won’t reiterate my previous arguments), so consider me still -1.

7 Likes
15

Thank you for keeping up the effort for this. If passed this would be the best thing to happen, for code bases I have worked on, since the introduction of f-strings.

1 Like
16

I’m also -1 for the same reasons that have been brought up by Steve Dower, Erlend E. Aasland, and Paul Moore in the other discussion thread.

I would perhaps be 0- if this remains an optional choice, people can use it if they want to. I’m uncomfortable with the idea that formatters are already planning to adopt this as the “preferred”/“default” behavior by automatically remove the expanded syntax.

Even though this syntax is similar to the f-string’s f"{x=}" syntax, it has different meaning. When writing f-strings, the x= and the expanded form refers to the same variable, and this variable was created (and named) by the same person.

The name of the function argument on the left of the = sign could be named by different person by the person writing the variable on the right.

The rationale about “encourage consistent variable names”, to me it looks more like “encourage users to name their variable to match the argument name decided by the function author”, and I’m feeling apprehensive of this. Maybe in many cases they do end up doing this, but not always, and I would like for users to still have that freedom. I don’t think “matching up variable names” should become the encouraged behavior or should be considered the “best practise”, especially in this case it’s not really ensuring both variables are the same throughout, but more like adopting the name chosen by function author.

15 Likes
17

But nobody is taking this freedom away though? After adopting this PEP anyone will still be able to write func(x=y), and formatters won’t do anything to that syntax. The PEP will make life easier for people who do like matching up variable names and won’t take anything away from people who don’t, just as shorthand f'{x=}' doesn’t prevent (or discourage) anyone from writing f'x={y}'.

2 Likes
18

This PEP proposes a paradigm shift in API design (explicitly mentioned in the Rationale). Add to this that it will be expected that linters and formatters will prefer this syntax. This will de facto adjust how people are writing code, including how APIs are designed. My answer to your question would be: yes, parts of this «freedom» is de facto being taken away.

(I have no further to say on this matter, so I won’t be adding more comments regarding this proposal.)

11 Likes
19

As mentioned in my post in the other thread, we can use a dunder keyword instead of a symbol to make the code look less cryptic/more Pythonic, at the cost of brevity:

my_function(
    my_first_variable=__same__,
    my_second_variable=__same__,
    my_third_variable=__same__
)

This has now evolved into a possible keyword __target_text__ per our discussion in that thread:

MyTuple = namedtuple(__target_text__, "a b c")
20

Hi,

Thanks for your great PEP: it’s well documented and looks complete to me. I like the examples, usage statistics and the “How To Teach This” section. Also, I agree that func(x=) is the best proposed syntax for this feature. IMO the PEP is ready for SC pronouncement!

I’m not sure if it’s still useful to collect opinions about PEP 736, but here is mine :slight_smile: (I missed previous discussions and earlier PEP versions.)

The Objections section miss the fact that changing the Python syntax requires the whole Python ecosystem to adopt this new syntax. For example, as Gregory wrote, some code formatters don’t support the recent Ruby syntax yet (2 years old). Linters, IDE, tools to refactor the code, code formatters, etc. have to be updated. It has a cost on the whole Python ecosystem.

To write code, I was never annoyed to repeat my_first_variable=my_first_variable thanks to code completion which is now a basic feature of all code editors.

About the “visual noise”, the PEP example is good:

add_middleware(
    excluded_urls=excluded_urls,
    server_request=server_request,
    client_request=client_request,
    client_response=client_response,
    span_details=_get_span_details(),  # <== here
    tracer=tracer,
    meter=meter,
)

Without PEP 736, to highlight the _get_span_details() call and make the code more regular, you can add a local span_details variable:

span_details = _get_span_details()  # <== moved here
add_middleware(
    excluded_urls=excluded_urls,  # only regular x=x arguments
    server_request=server_request,
    client_request=client_request,
    client_response=client_response,
    span_details=span_details,
    tracer=tracer,
    meter=meter,
)

It’s always tempting to add syntax sugar in Python, but I’m not convinced by the visual noise, reducing verbosity or other PEP arguments, compared to the cost on the Python ecosystem.

I’m also confused by func(arg=): for me, the function parameter/argument name and the variable name are two very different concepts and I’m not sure about merging them into just arg=. For example, if I point the cursor on the parameter name, I expect an IDE to show the documentation/hints of this parameter. If I point the cursor on the variable name, I expect the IDE to guide me on the variable usage (ex: find where it comes from, its type, etc.).

I’m not really against the feature, it’s more a matter of “taste” at this point :slight_smile: I vote -0 on the PEP.

Good luck to get your PEP accepted :wink: Anyway, it’s a great document to summarize 10 years of dsicussions on this topic!

8 Likes