While it’s tempting to look at popular open source projects I believe the majority of Python code is closed source over which we have little access. But beyond this I think the biggest point against the double dunder is the fact it creates asymmetry with keyword only parameters. The keyword only parameters use a marker in signature, IMHO as such positional only parameters should do so too, and the / character is a decent solution (ideally perhaps we should have one star for positional, two for keyword only but this ship has sailed now given keyword only use one star).
(Continuing the argument seems pointless given the SC has already declared their position, but here goes…)
I don’t buy the symmetry argument.
The bare star came about because the alternative is to use a named *args and then raise if anything is passed in it. Leaving the name out is the obvious way to implement that pattern in syntax (though unfortunately, it’s still not compatible with older versions). Even though “multiply” and “multiple arguments” seem to be almost the same thing, we call them “star args” and internally they are “varargs”. So the best the symmetry argument can apply is that “divide” has equally little to do with “positional” 
Now, the thing about incompatible syntax is that it is an investment in the future, since it cannot be widely used in the libraries it is intended for until everyone drops 3.7 support (and I assume functions that can "take any named parameter including ‘self’ have to be in generic libraries, because it’s far too easy in domain-specific libraries or applications to choose a non-conflicting name). Async/await was incompatible syntax that is finally standard across all living versions of Python 3. It took a while, but I think it’ll be worth it because there are no other good ways to express it in code, there’s a lot of consistency with other languages, and it’s something that end-users have to use often.
I’m not convinced positional-only arguments are the same at all. The only rationale I’ve seen that doesn’t have a simpler and more compatible solution is the “call later” example. I write those all the time, to be honest, but I suspect I’m an outlier. As those who teach have said, this doesn’t come up often, and I believe them.
Adding syntax (instead of convention or a decorator) in the name of symmetry for a library-developer problem that library developers won’t be able to use for years seems like the wrong decision.
(And that’s the end of my contribution here until code review time. It’s up to our duly elected steering council now.)
2 Likes
Some reasons I don’t like the dunder prefix proposal:
Using the dunder prefix to indicate positional-only parameters would introduce a precedent in Python where the name determines semantics. We don’t do that elsewhere – the form of names is a convention (e.g. self, and the rules for capitalizing class names).
Indicating that a parameter must be used positionally wants syntax, just like indicating that it must be used as a keyword, or that it may be omitted.
Another problem with using __foo is that now you end up having to write __foo in the body of the function (unless you start with foo = __foo). This is not good for readability.
Finally, the dunder prefix begs the question what it means if a dunder-prefix parameter is preceded by a non-dunder-prefix parameter. I.e. what to make of this?
def foo(arg1, __arg2):
...
Is it a syntax error? Does it mean arg1 is also positional?
This would also affect self – and I don’t want to have to write __self (except in some very special corner case).
4 Likes
See issue36518 which significantly decreases the need in positional-only parameters in Python (although PEP 570 still has a value for solving other problems). The change is surprisingly tiny.
1 Like
I would like to consider that separately, and very carefully – in fact I think it’s PEP-worthy, not something to just do with a BPO issue and a GitHub PR. It is a very subtle change in the meaning of **kwds, affecting every function whose definition uses **kwds. It certainly doesn’t solve the problems that PEP 570 sets out to solve.
6 Likes
I find the dunder prefix idea ugly but I think this is not accurate:
What about self.__private = 1? It exists but personally I don’t think it is a good example of a successful feature. Having to use dunder names internal to the function seems very ugly to me and I would be sad if we go with that solution (even though it has a certain simplistic elegance to it).
I am currently liking a decorator as the best solution. There is no reason it has to be slow since the decorator doesn’t have to return a wrapped function. Instead, it could set a property on the code object, e.g.
func.__code__.__positional_args__ = n
That property on the code object can do whatever magic is needed. You can’t implement it in pure Python but you don’t need any new syntax.
1 Like
One issue with using a decorator is how to document positional only parameters? For parameters using clinic, they are already using the ‘/’ in their docstring. In this respect PEP 570 does better (assuming you are okay with the potentially confusing ‘/’ in the docstring).
For the decorator approach, I’m not sure of a good solution. Perhaps because we have gotten by this far without this feature, there is no great need to explicitly document these parameters. I feel it is best practice to always pass positional parameters by position, even if they are allowed to be passed as a keyword. So, having explicit syntax in the docstring that tells you some parameters must be passed by position only doesn’t add much. In general, you should already be passing all positional parameters by position.
If you try to call the function the wrong way, you will immediately get an error message that tells you exactly what’s wrong.
To save everybody’s energy, I am going to accept this PEP. Carol and Pablo are currently doing some editing (https://github.com/python/peps/pull/975) and once that lands I’ll accept it.
Thanks everybody for giving this proposal serious thought! By pondering the alternatives I have come to a much clearer understanding of the design space here and why the proposal in PEP 570 is the one to go with.
5 Likes
Accepted! Congrats Pablo and co-authors.
8 Likes
The current implementation changes the signature of PyCode_New (and the undocumented types.CodeType constructor), many low-level extensions need to adjust.
(Just for information, so everyone involved is aware.)
See also https://bugs.python.org/issue37032 as a way of eliminating the need for the majority of packages to adjust in the future.
These comments sound cryptic. My colleague Michael Sullivan (mypyc co-author) just discovered that the signature of PyCode_New() changed. I have one question: is that going to change back to what it was in 3.7? Or is the change a done deal?
The change is pretty much a done deal: many projects have already make adaptations and the implementation without it is much messier and inneficient. It is also not by far the first time it happens (check out https://bugs.python.org/msg343377) and it mirrors how keyword only parameters were introduced.
Sure. Then I’m very mystified by what was meant by the comments about the new replace() method, and what you meant by “a way of eliminating the need for the majority of packages to adjust in the future.”
Many people create code objects in python to modify another code object slightly (adding flags, changing bytecode). Having code.replace makes this practice not suffering any more from additions to the constructor because you just need to change what you need (like creating a copy of the code object changing only the name, or the bytecode…) even if the constructor have more parameters. Without this the situation is suboptimal because you need to call different constructors depending on the Python version your package runs on. All Python projects that needed to change were that I helped with or I checked falls into this category.
On the C-side there are no improvements.
Apologies for the confusion.
1 Like
Sorry, looking at the thread for the first time, and my apologies if this was discussed above.
Had this idea come up earlier, it would have been more “natural” (easier to teach) to use * and ** to separate position-only and kw-only, similar to the use of *args and **kwargs. (With all the existing code, I realise it is now too late… maybe for a future Python4.)
I don’t think you can state that as a fact. The current solution lets us define syntax that goes positional-only, positional-or-keyword, keyword-only where there is symmetry in how you transition across the parameter list. Your solution would lose that.
That’s highly doubtful. If have added and removed syntax, but never swapped syntax.