IDLE does not support it, and realistically that’s the only one we could possibly use to justify a language design decision.
I don’t think we should let any IDE features influence the language. It should be readable in B&W plain text printed on a piece of paper, and then it will only be more readable elsewhere.
The only exception I make here is that it should be easy to search for it in a search engine. “What does equals sign in a function call mean in Python” seems like a reasonable thing to expect, so it already meets that for me.
No, I’m not, but I’m trying to get an idea of the extent to which keyword arguments are currently being used, in response to an earlier claim that keyword arguments don’t need any encouragement since they’re already perfectly well encouraged. Welcome to long threads and long tangents.
So how does this work for code reviews in the Github UI, for example? The argument that IDEs obsolete attempts at making programming languages more ergonomic has never held water.
Even if your favorite text editor supports language servers (not everybody uses VSCode), language servers do not always work properly. pylance does not seem to support Cython, for example, and it probably does not support extensions generated using other tools such as nanobind or PyO3.
I think with type hinting becoming so big, nobody is ever going to try and introspect native modules anymore. Their developers will need to start producing type stub files to go with them. So it’s not about “not working properly” and more about a fundamental premise (“everything is fully typed”) that leads to this result.
A syntactic change in pure Python code would obviously be supported.
Part of this has grown historically, as the separation into keyword-only arguments is relatively recent (at least in the sense of: “present in all commonly supported Python versions”). But it has several advantages from a usage and maintenance perspective – obviously not 100% of the time but still.
As a concrete example, scikit-learn has moved almost all their APIs to keyword-only usage – their functions tend to have lots of arguments, so it makes sense that they adopted this quickly. Other libraries (incl. e.g. SciPy) are cautiously considering the same thing (within reason; some APIs are positional by nature, but making all the optional bits kwarg-only is still desirable). Part of the reason these things take time is because we want to minimize breakage obviously, and it’s a fair bit of churn, though the end result is IMO worth it.
Agreed; and I am of the opinion that this is the perfect default. Yes, there are a few situations where you really want something to be positional-only, and that’s now an option (which for a long time it wasn’t), but otherwise, allowing people to pass arguments by name is almost never a problem. (Almost. It’s a bit annoying when trying to map arguments to parameters, and it does mean that your parameter names are a part of your API, but otherwise not a problem to have the flexibility.)
So if callers MAY use keyword arguments - and you have an example of changing that to kwonly, but examples of moving to posonly are quite rare - then why don’t they? Is it because it’s completely unnecessary? (Likely true for functions with small numbers of parameters, more dubious as that number increases.) Or is it because writing open("some-file", mode="wb") is more hassle than it’s worth? I suspect it’s frequently the latter, but there’s no easy way to know.
TL;DR: I’m still -1 to PEP-736. As @pf_moore already noted, I think this sets a precedent for bad API design. For years, Python has encouraged multiple functions (clean APIs) instead of loading a few APIs loaded with keywords (flags and whatnot). With PEP-736, we will be encouraging the latter, which is IMO a bad thing.
Here’s my largely unformatted brain-dump after reading through the PEP a couple of times:
The first example looks contrived to me; a function with three arguments, all with 17-18 character long convoluted names? IMO, we should discourage such APIs instead of encourage them.
Is there data backing this claim, “The case of a keyword argument name matching the variable name of its value is prevalent among all major Python libraries”? The “study” referenced later in the PEP suggests on the contrary that this only accounts for a minority of keyword parameter uses.
It seems to me the rationale and motivation is a collection of subjective views only. To me, that seems like a weak case for such an impactful change:
“This verbosity and redundancy discourages use of named arguments and reduces readability by increasing visual noise.” This is highly subjective; you can also view this as explicit is better than implicit.
“On the flipside, positional arguments are often used simply to minimise verbosity and visual noise.” There is no data backing this claim. Related: See PEP-570. It is a good read.
“Encourages use of named variables”: this goes against PEP-570 recommendations.
“A common problem is that semantically identical variables have different names depending on their contexts”. Is there data backing this claim?
“Applicability to dictionary construction […] Whether to further support similar syntax in dictionary literals is an open question out of the scope of this PEP”: if it is out of scope for the PEP, you should probably leave it out of the PEP.
In Applicability, you refer to SciKit Learn, but you do not mention that the proof-of-concept PR was actually rejected as not being an improvement.
Also in Applicability, you’ve included Lib/test in your data. I don’t think the test suite code makes for representable data to back your claim; consider excluding Lib/test.
Based on your (IMO questionable data), you conclude that " the f(x=x) keyword argument pattern is widespread, accounting for 10-20% of all keyword argument uses". That is a minority, not the majority, of keyword parameter use.
“this syntax communicates “here is a parameter, go find its argument” which is more appropriate given the semantics of named arguments.” OTOH, what we have today is even simpler: “here is a parameter and here is its argument”; it is readable and explicit. The proposed “go find your argument” is neither more readable nor more explicit. On the contrary, it is objectively implicit wrt. the argument.
In Rejected Ideas, The feature is not explicit: Fencing this off by calling it a “misunderstanding” is arrogant and unfair. Currently, we have explicit code since both the parameter and argument is visible. With this proposal, the argument is implicit.
It’s a special form of f(a, b, **kwc, **kwd, e, f) which while contrived illustrates that e, f, could only be keyword-arguments here. I think the other use of double ** here really makes it qualitatively different than the * alternative presented in the proposal.
There are no new character combinations to grok here, it just builds on already known syntax.
But we already have lots of APIs in the real world that do use an abundance of parameters. I would not even call them bad; real world usage shows that it’s often simply a necessity. All this is the situation without PEP 736 today. The proposal does not exist in a vacuum, but instead addresses a real, existing pain point.
This is irrelevant to my point. PEP 736 will encourage a new way of Python programming; it will encourage an API style that is not currently encouraged. That there already exists APIs with “an abundance of parameters” does not change this fact.
Has this proven to be true of the assignment operator? This same criticism was levelled at PEP 572 before it was accepted. Have we seen abominations all through people’s codebases, or have := expressions been used only where they’re actually useful? You speak as if this is an utter certainty, as if the laws of the universe guarantee that this feature will be misused.
I’d be lying if I said I’d never wanted this feature (and yes, especially so when not using an IDE).
On the other hand, this breaks referential transparency. This might complicate the mental model of programming for beginners or make refactoring harder. I don’t see much mention of this in “how to teach this”, maybe folks with teaching experience in the languages referenced by the PEP could chime in on whether this worry is real.
Anyway, this has come up a bunch over the years and inspires a lot of commentary, so thank you Joshua for working on the PEP!
I don’t think we need to call upon the “laws of the universe” or “utter certainty”; neither of us knows how much this will affect API design, but that the features of a language to a lesser or larger degree affect how people write code (and thus the APIs they design) is a fact.
Yes, but to what extent and in what direction? That’s why I mentioned PEP 572, which landed in Python 3.8 - we’ve had a few years now to see what sort of impact it has. Has it been as doom-and-gloom as people predicted, or are programmers smart enough to use features where they are useful? Your tone made it seem as if there was absolutely no way it could be anything other than bad.
I’m a bit rusty on my new-programmer-teaching as my past few years have been with far more experienced programmers, but JavaScript’s object shorthand was never a major hangup for anyone. I would often see novices write things out longhand {thing: thing} when the shorthand would have worked, but if they were reading someone else’s code with the shorthand, it was never a source of confusion.
(Either that, or JS had just so many OTHER sources of confusion that it didn’t matter…)
I’m not sure if this has been mentioned before, but my main criticism of this PEP is that it’s too specific - that if a shorthand for homonymous parameters and variables is desired, it should come from splatting shorthand-form dicts, which the languages mentioned as inspiration in the PEP also possess, instead of introducing another function invocation variation. For example, Ruby’s shorthand call syntax draws directly from its implicit variable hash syntax; compare {a:, b:} with c(a:, b:). (Ruby uses the same delimiter for both which does make things easier.) Counterintuitively, the proposed syntax would encourage using the dict function instead of the more idiomatic dict literal for these cases. I find the implictness/confusion reservations unconvincing. Rather, I think this feature should be generalised to support a wider array of uses than it would if it were limited to keyword arguments. That splatting shorthand dicts should come at a greater cost than simply omitting the value should alleviate misuse/overuse concerns.
When working on code involving data or distributed systems and calls across language/process/machine boundaries, it is common to wind up working with language independent schema’d data formats wrapped up into a native API in high level languages as some form or object or named field like struct for easy programmatic access to the hierarchical data structure.
Examples: Protocol Buffers and gRPC, Thrift, CaptnProto, and a pile of others. Importantly: not an ugly to work with dict as raw json defaults to.
Constructors for these often take keyword arguments matching their field names, the values of which code may well already have in equivalently named locals.
The API design pattern leading to this being common isn’t just about Python and isn’t going to go away.
I would not analyze the Python Standard Library if looking to figure out what any sort of modern common coding practices are. The standard library as a whole is a pretty “horrible” body of very old code that we do not like to make changes to on purpose. (it wakes the scary Rabbit)
I’d probably try to select a diverse subset of modern projects that haven’t been around for more than a decade, if even five years, including application code and perhaps include some recently shared Jupyter or Colab notebooks.
I think this is a misunderstanding; this case being ‘prevalent’ does not imply that it is the majority of uses, only that it is common. However, one could argue that the use of ‘all’ in the PEP is not justified. Additionally, the term ‘study’ is not used in the PEP and it’s unclear what you’re quoting, if you could please clarify.
Is there data we could reasonably collect that you think would shed light on this to improve the PEP?
In my experience with technical writing, it is often beneficial to mention things that are not in scope to avoid any confusion. This is the first PEP I’ve contributed to so I’m uncertain of the standard protocols, but this appears to be very common among past PEPs. I’d be happy to remove this comment if that is standard.
As mentioned in the description of this thread and the PEP, I absolutely do not advocate for a universal application of this syntax everywhere it is applicable. I agree such application is inadvisable. The purpose of this exercise was to collect statistics on applicability as presented in the PEP. I will restate this in the Applicability section for absolute clarity.
I did consider this and would gladly to tweak the PEP according if there was broad agreement on that point. However, my conclusion at the time was that test code is still real Python code that needs to be written and maintained, so if this syntax is applicable there that should not be discounted.