I think Carol suggested that we should augment the docs with guidance about all parameter options available (read the linked post).
@pablogsal One request for the PEP: add a âHow to teach thisâ section (per the template in PEP 12 or the list of suggested sections in PEP 1). Doesnât have to be a lot of text. Perhaps a suggested change to the Tutorial in the docs? Surely thereâs a place there where keyword args are discussed, that would be a good place to discuss positional args as well, leading with a realistic example.
2 Likes
-1 from me as well. This is not an âadvanced featureâ, this is just a distraction.
2 Likes
I definitely think, âWe shouldnât have positional-only arguments at allâ is a valid position to take here, though as a library maintainer the arguments in the PEP are pretty compelling.
What I donât think is a very strong position is continuing to allow positional-only arguments via *args and in the C API, but maintaining the asymmetry between the C API and Python. If you take as a given that Python will continue to support positional-only arguments through other means, then I think my point stands that keeping this out of the language only leads to more confusion, because without support in the Python language itself, these things are hard to explain and hard to document.
I also understand the position that adding this syntax can be seen as an endorsement that would encourage more of this kind of API design. Itâs almost certainly true that even if the PEP explicitly says that this is just to remedy the asymmetry and not an endorsement, people will still take it as an endorsement. However, at the end of the day, I donât think thatâs a very good reason to reject this change, because it means end users will be suffering from awkward implementations and documentation (and inconsistency between C and Python implementations).
3 Likes
range() has never been converted to use Argument Clinic, presumably because the existing implementation is fine and it wouldnât really gain us anything. Argument Clinic could theoretically handle the funny semantics of range() by using âoptional groupsâ, which are groups of optional parameters enclosed in square brackets. Thatâd look something like this[1]:
range([start,] start_or_stop, [step], /)
When there are optional groups on both the left and right side of required parameters, and itâs ambiguous as to which one to choose first, Argument Clinic gives priority to optional groups on the left side. I designed that specifically to handle range().
In my opinion, range() is a special case that is unfortunately breaking the rules. Itâs convenient but surprising. Certainly I wouldnât want to try to contort Python syntax to accommodate itâbeyond saying "implement it yourself with *args and len()". I would not propose, and doubt I would ever like, an extension to Pythonâs function declaration syntax that made it directly expressible. I guess that makes me a syntactic reactionary.
.
[1] Argument Clinic would actually require the declaration to be spread across multiple lines, to make parsing easier or something.
4 Likes
FWIW Iâm +0.2 on the PEP. Itâs always been a little surprising that Python has so many functions whose semantics are not implementable in Python without rolling it yourself using *args and len(). And it would make the dream of âreimplement the Python standard library in pure Pythonâ a lot easier to realize. But I could live without it, and I can understand why some folks say itâs unneeded and not worth the semantic overhead.
1 Like
Wouldnât this work fine?
def range(start, stop=None, step=1):
if stop is None:
start, stop = 0, start
<continue as normal>
That reduces it to a single âweirdâ case, which is that your positional argument goes into âstartâ but is interpreted as âstopâ (and providing âstart=â without âstop=â is also a bit weird, but at least itâs visibly weird at the call site). All other uses, positional named or otherwise, work the same as today, no?
I would say âyes, for small values of âworkâ.â Yes it supplies the physical API we expect. But semantically it doesnât match. If you only examine the signature, if looks like if you only provide one argument youâre specifying âstartâ. You have to read the implementation / documentation in order to understand the weird âstopâ / âstart, stopâ / âstart, stop, stepâ behavior.
FWIW PyPy spells it def range(x, y, step):, seemingly deliberately to avoid confusing people about which is start and which is stop.
2 Likes
Donât you mean deliberately to confuse people so much that they have no choice but to read the docs? 
Which brings us back around the circle again to what I think is the real question - which docs should users have to read in order to use an API? Currently the signature is not sufficient because of *args hacks. Clever new syntax will add a requirement to read the function definition syntax docs which though this has the advantage of once per person vs. once per API. Though if people ought to read the API docs anyway, then theyâll find out about the tricks pulled in that API to make things work.
We already have / showing up in API docs, but it itself is undocumented and untaught, which causes confusion. Replacing it with invisible magic and requiring people to read the API docs is a good approach in my view. Formalising it and requiring people to learn it is the alternative.
I might be slowly convincing myself as I think out loud here But not quite yet. I still think the ânot full-time developerâ demographic is critical to Python right now, and theyâre the ones most happy with magic and least happy with un-Googleable syntax.
I heard the steering council is already thinking about the long term plan for Python though, which should include the target demographics, so Iâd like to wait for that before choosing one of our audiences as more important than another.
1 Like
One small detail: the â/â has been, indeed, undocumented for a while, but this has recently changed and now he have some sort of documentation for it:
https://bugs.python.org/issue21314
I agree with this, but I also think that there is a lot of pain inflicted to the users due to the difficulties of keeping APIs backwards compatible. We have recently talked about how maybe CPython cannot improve performance-wise without breaking some bonds. Even if this is a bit far from this particular case, it has certainly some common roots. In the standard library, there are multiple âincorrectâ names in parameters that cannot be changed because of this reason as well as some functions that should take arbitrary argument lists and cannot, and then there is the matter of documenting this interfaces and their properties.
I think is possible (and challenging) to find a good balance between making the maintenance work for library developers easier while keeping the rest of the target demographics happy as well.
By the time people need that much language theory to plan ahead for the possibility of a class being subclassed â explicitly deciding they should make something positional only â theyâre well out of ordinary into advanced territory. Developers who find reason to care will likely already be wishing the names matching were Liskov enforced, in hindsight, on existing APIs years after tons of code is using them with varying names where it is too late to change.
Even with positional only as a feature, I would never promote its common use. As a Python best practices guideline or readability suggestion, suggest that people add , / to their single argument method signature as a design pattern because it subjectively looks ugly without obvious meaning to casual readers of the code.
Python default behaviors matter. They control the Hyrumâs Law consequences for most code.
Our language defaults to allowing anything to be subclassable. Just as it defaults to allowing Python defâd callables to use positional or keywords for argument passing. Those defaults are fundamental invariants we can never change. As positional only is not a default, it doubt theyâll be commonly used for method signatures, especially when the syntax causes people to make faces.
I donât have a definite problem with an ugly syntax per se, but that is primarily because I see this as a feature that really shouldnât be used.
In my view, I thought we really only wanted positional only arguments for use by our standard library so we can reconcile behaviors of pure Python code with that of C API code. Making matching the exact API semantics more likely when pure Python is used on PyPy while a C thing is used by CPython and MicroPython. Folks like Steve are arguing for the same synchronization: Instead by suggesting we force the builtins and standard library to accept keyword arguments everywhere that it does not.
Given we always say âdonât look at the standard library if you want examples of good codeâ (because weâve got a lot of 1.4-isms all over the place), I didnât have a problem with the syntax. If we anticipate reasons for it to be widely used outside of the stdlib (as other comments in this thread are pointing out), perhaps we could do better with the syntax.
At this point I need to go re-read the PEP.
2 Likes
I have alternate syntax suggestion to toss into the mix. This may trigger an immediate negative reaction from some, but give it a look and at least ponder it as this would also be concise which is one thing the ,/ syntax has going for it.
Repurpose the old Python 2 tuple unpacking argument syntax. It was removed in Python 3 because it was extremely uncommon, not a lot of people even knew of it.
Instead of the existing PEP-570âs proposed:
def castle(name, should_burn, /, sink_into_swamp):
That would look like:
def castle((name, should_burn), sink_into_swamp):
Long time pythonistas and Guidos: Donât read that with your remaining Python 2 eye. All Iâm suggesting is that the () around the first parameters causes them to be positional only. Instead of a ,/. No actual tuples involved. I doubt most of the worldâs Python developers would mind this syntax. Itâd be the first time theyâve seen it.
Good: Concise. No use of / as a mystery parameter. Less violent slashing in the code. () are prettier.
Bad: Python 2 literature covers this syntax and some tiny minority of very old Python 2-only code actually uses it with a semi-different meaning. This could also trip up people porting the rare Python 2 that used tuple unpacking args code to Python 3, but their API already needs changing as is so they already have that problem. If that was the only hold up, it could be ameliorated by going through an extra release cycle behind a from __future__ import.
Similarities: In Python 2 this was a way to specify required positional only arguments. Unfortunately they needed to be manually wrapped in an iterable when calling the function. Weâd be restoring a subset of that concept without any of the odd magic iterable passing and unpacking.
Alternatives, well, weâve got other tokens that could do this instead of (). {} or [] perhaps. Though I think both of those would lead to more confusion as they imply mutable types rather than a mere ordered grouping so Iâm not going there.
No tuples were harmed to bring you this message.
2 Likes
Thanks folks for the many comments, thoughts, and suggestions. Thank you @pablogsal for being responsive to feedback on the PEP as well as being collaborative with others.
To those like @larry and @gpshead who have shared history and perspective, to @steve.dower, @rhettinger, and @pitrou for sharing concerns about complexity and clearly communicating how to teach users, and to @pganssle and others for adding perspectives from different use cases, I thank you for your dedication to the language, its users, and its sustainability.
Complexity comes from serving many different groups of users and from constraints from past decisions. A computer language is complex. It will be complex whether new code is added or not.
What is imperative is communicating clearly to users when and why to use any language feature while balancing the stability of the language and productivity of different groups, including the core devs maintaining the language. This is not easy which is no surprise to anyone. Yet itâs so important for sustainability of Python.
tldr: Communication with each other and our diverse user groups (web, science, education, ops, etc.) is key to our success.
6 Likes
There are two possible ways of dealing with complexity:
- Avoid adding it in the first place
- Build the infrastructure and institutions that allow to mitigate its negative effects (but not necessarily eliminate them, as thatâs rarely entirely possible)
Which approach is most adequate should be judged on case-by-case basis.
3 Likes
@gpshead Would you be more in favor of the PEP if it had some language in it to indicate that this is an option we are adding because we want to enable this widely-used pattern, but that the fact that it has language-level support is not a particular endorsement that it should be more widely used?
@steve.dower Would that allay your concerns at all?
As I said in my earlier post, I think that if this PEP passes, there will probably always be people who take it as an endorsement that this is a particularly Pythonic pattern, but that doesnât mean we have to lean into that.
I think it would be reasonable enough for the PEP to say that this is a widely used pattern, in the standard library and in many fundamental libraries, and language-level support will make things better for both the designers of those APIs and their users but that doesnât mean that you should default to using positional-only parameters, and end-users should make a thorough consideration of the design trade-offs.
AFAIK, that âwidely used patternâ is mostly only used in C code, and itâs largely for accidental reasons, rather than a deliberate design choice of disallowing named arguments for a subset of the C-implemented builtins and library functions.
Put differently: right now, C code writers have to get out of their way to allow named arguments (by using a slightly more complex API), so the default / âgood enoughâ choice is to not allow them. If, conversely, they had to get out of their way not to allow named arguments, do you think it would still be a âwidely usedâ pattern?
As soon as you are adding syntax for a functionality, you are signaling that itâs a Pythonic pattern. Thatâs how most people will interpret the adding of new syntax.
1 Like
The PEP gives a few examples where this is definitely a deliberate choice, for example, the motivation section notes that bool, float, list, int and tuple were all changed from named arguments to positional-only arguments.
@guido provided some arguments as to why you would want to do this to preserve Liskov substitutability in certain class hierarchies, and there is the example of APIs like dict.update, where a named positional parameter prevents updating a specific keyword (in that case self).
I agree that there are probably more uses of it than there otherwise would be if it were not the default for C function definitions, but Iâm not particularly convinced that those are even the majority of cases.
Yeah, I admit that this is true, but there is definitely a difference between saying, âWeâre adding this because itâs a capability we desperately need and you should use it all the timeâ and saying, âWeâre adding this because a lot of people are using hacks to do this and we think it addresses a specific need.â
Consider that we havenât seen a huge rash of keyword-only arguments severely restricting APIs since the * syntax was introduced. I suspect that positional-only arguments will follow a similar trajectory - itâs nice to have if you find you need it, but most people will not need it.
FWIW, the primary reason I have not used keyword only arguments much is that so much of the code I deal with needs to be Python 2 compatible for a few more years while the real world catches up to 3. There are places I absolutely would recommend keyword only.
Iâm less likely to find a code health reason to recommend positional only. It patches a wart to reconcile multiple existing ways of creating callables. Iâd also accept patching it the other way by requiring C API callables to accept keywords, simplifying the C API (hard). Even though I would never want to see keyword args used on things like len(), max(), or floor() for style reasons. Unnecessary use of keyword arguments on such well known call sites are anti-patterns that linters can happily call out.
I still leaning positive (+0?) on this PEP because I want to avoid restricted implementation specific API behaviors. My + would increase if there were no / involved as I personally find that ugly. Adopt another syntax and fix-up the argument clinic help text rendering to stop showing itâs own / to users in favor of whatever syntax is actually adopted for use in native Python and I think the language will be a less confusing place for users.
A sentence in the docs explicitly discouraging people from bothering to use the feature is unnecessary. Describe the feature as being added in order to allow pure Python code to more easily match the exact behavior of CPython API builtin or extension module callables. Someone will absolutely find other reasonable uses for it.
We should avoid shaming developers for using features we add.
1 Like
Is everyone aware that the â/â is meant to be the opposite of ââ? This is meant as a mnemonic. âEverything after ââ is keyword only, everything before â/â is positional only, everything in between can be used either way.â
The problem with the syntax, which I agree is ugly, is that there is no less ugly syntax, at least nothing that doesnât feel just as arbitrary. So I donât think I will go for any of the alternate proposals. (I did see Gregâs () idea.)
1 Like
I agree, but being used in 5 constructors does not equate to âwidely usedâ IMHO.
Iâm sure bothering about the Liskov substitutability of named arguments is an interesting theoretical concern, and perhaps worthy of a blog post (or a witty conference talk), but certainly not a legitimate reason to add syntax to a programming language.
Ok, but whereâs the proof that a lot of people are using hacks to do it? So far, we know about a couple of uses in CPython itself, and perhaps a couple of uses elsewhere.
Iâm sure it addresses a specific need. The question is how specific or how general is it. My intuition is that itâs extremely specific. Myself, it probably occurred to me once or twice during my Python programming lifetime. And itâs not very difficult to satisfy, even without dedicated syntax.
I find itâs more immediately useful than positional-only parameters, as it helps avoid ambiguous and confusing function calls such as do_something(false, true, true). Positional-only parameters do not have that virtue.