Some places this could be useful in the standard library:
collections.abc.Generator - most people don’t use the last 2 type parameters, omitting them should be equivalent to collections.abc.Generator[T, None, None] or Any (I trust the opinions of the typeshed maintainers about this more than my own opinion here).
builtins.slice - Make slice generic over the start, stop and step parameters. start parameter should default to int , stop default to the type of start and step default to int | None .
datetime.datetime - for informing people of timezone awareness/naiveness at type time. default and bound should be timezone | None. datetime[timezone] would mean timezone-aware datetimes are expected to be passed, datetime[None] means naive datetimes and datetime[timezone | None] means you don’t care about awareness. This is relatively commonplace in other languages’ datetime libraries.
builtins.frozenset - default and potentially bound could be changed to Hashable to give better types and potentially catch more bugs before they reach production respectively.
Are type checkers expected to always show the full type (e.g., Generator[str, None, None]) in error messages and other ouputs, even if the type arguments are default values? If this feature is widely used, this could lead to quite verbose error messages. (I imagine that library authors will be more inclined to make something generic if they can set a default value.) Maybe type checkers could at least omit default None type arguments in their output.
I think this is fine to leave up to each type checker, I personally wouldn’t because I think the parameters could be useful especially if the error is related to the omitted parameters.
This is an important point. I hope that libraries will be able to use this feature to produce shorter messages rather than longer: Generator[int] in the first example.
Yeah, maybe let them decide or perhaps they’ll provide an option. I find long messages inscrutable.
The issue is kind of similar to type aliases, right? Some code makes frequent use of type aliases, some of which can be quite long. When typecheckers report errors or notes using the expanded aliases the output can get very verbose and even hard to understand.
This is currently left up to typecheckers, and I don’t know if any of them simplify output using aliases.
I feel it’d be a shame to hold this PEP back on account of PEP 695 integration. If the syntax extension is the only thing the SC is wary of here, perhaps the PEP can be considered without it?
I fully agree. The main value of the PEP is the default argument to TypeVar, etc. and not the new syntax (which itself is conditional on PEP 695).
From a pure typing perspective a delay doesn’t change much. You can already use default with the typing_extensions variant of TypeVar, ParamSpec, and TypeVarTuple. It’s thus just up to the type checkers to support it. AFAIK pyright already does. For mypy I’ve already opened a first PR and plan to continue working on it once that is merged. Not sure about the others though.
I’ll be pretty sad if PEP 695’s acceptance means that we will lose the ability to easily introduce new features into the typing system without introducing new syntax changes.
The implementation going with pep 695 as well is already very close to completion. It pretty much just needs tests see GitHub - Gobot1234/cpython at pep-696
I don’t quite see the correlation. Is it just because you expect enough things to impact generics that they will inherently require syntax changes?
As for PEP 696 coming after 695, PEP 696 specifically says what happens if 695 comes in, so while it might start without any syntax changes, it has a “time bomb” of syntax changes based on PEP 695’s status. And I will say the SC discussed this and that’s why we punted on 696; we want to see the impact of 695 before we introduce something that seems inherently tied to it (as outlined in PEP 696 itself).
Could you clarify what kind of impact you are looking for here? Is it just about whether we get through the betas successfully after 695 is merged, or is it about waiting until 3.12 is final and seeing how 695 gets used in real-world codebases?
Yes. This part of the typing system feels to me like it’s in flux at the moment. Python 3.5 introduced one typevar-like concept: TypeVar. Python 3.10 introduced a second typevarlike concept: ParamSpec. Python 3.11 introduced a third typevar-like concept: TypeVarTuple. Now with Python 3.12 (via PEP 695), we are introducing the concept of “autovariance” to typevars, which is a major change to how typevarlikes are declared and used. PEP 696 introduces type defaults for typevarlikes; there’s a proposal for a TypeVarDict floating about; and discussions are ongoing about extending PEP 646 via a “Map” operator.
By cementing the status quo (“We have three typevarlikes: TypeVar, ParamSpec, TypeVarTuple”) in new syntax, we will make it harder to modify the way these typevarlikes work without changing syntax again or introducing new syntax. We will also make it harder add new typevarlikes without introducing new syntax – they will not be considered “first-class” typevarlikes if the existing typevarlikes get dedicated syntax built into the language but the new typevarlikes don’t.
To me, this feels like something of a misunderstanding. Typing users have been asking for defaults for typevarlikes since at least 2016 (Allow specifying a default for omitted type parameters · Issue #307 · python/typing · GitHub), whereas the syntax proposals in PEP 695 are comparatively very new. I don’t remember the first draft of PEP 696 that was shared on the typing-sig mailing list having any proposals for new syntax, because none are required for this feature if it is implemented independently of PEP 695. It was suggested that James should add a sentence about how this PEP might integrate with the syntax proposals of PEP 695, should they both be accepted in the same Python version (or should PEP 696 be implemented after PEP 695 had been implemented), and I think that was the correct decision. But I don’t think it’s correct to say that PEP 696 is inherently tied to PEP 695. It’s more correct, in my view, to say that PEP 695 significantly complicates the implementation of PEP 696 (and would do for any future modifications to how typevarlikes work, as well).
If PEP 696 were implemented in Python 3.12, and PEP 695 were deferred to 3.13, all we would have to do to implement PEP 696 at runtime would be to add a new keyword argument (default) to typing.TypeVar, typing.ParamSpec and typing.TypeVarTuple.
I dislike the idea of introducing new syntax until we’re certain we know all of the concepts we want to enshrine in the new syntax. Otherwise we’ll end up with constant “syntax churn” in every Python release, which seems like a bad outcome to me.
I don’t think the Map operator would require any syntactic changes under PEP 695, since no change is needed to how type parameters are declared.
The TypeVarDict proposal is not very fleshed out yet, but it seems to me that it could be implemented as a TypeVar with its bound set to TypedDict, plus some new operators like typing.KeyType[] and typing.ValueType[] that would work on TypedDicts. This approach would work under PEP 695 without any further syntactic changes.
You are right in general that enshrining type parameters in the syntax makes it more difficult to change them, and it is unfortunate that PEP 696 is getting delayed for this reason. However, I don’t anticipate too many proposals other than PEP 696 that will really require syntax changes.
I actually think it is good to put a bit of a brake on introducing more kinds of “TypeVarLikes” other than ParamSpec and TypeVarTuple. These two are already among the most complex type system features; mypy doesn’t even fully implement TypeVarTuple yet and there are numerous bugs with the ParamSpec support. We should think very carefully before we add more kinds of type parameters that introduce even more complexity into the type system.
I don’t want to get too bogged down in whether any one specific proposal would require new syntax or not, because my point is sort of that I don’t really feel able to predict, at the moment, what new proposal might get brought up next year, given how much flux there seems to be in this part of the typing system at the moment. And, as you say, these proposals are all only half-fleshed-out at the moment; it’s not really clear how any of them would work in their entirety at the moment.
This is a very reasonable position! However, I don’t feel like we’ve attempted to come to any kind of consensus around this point, as a community. As far as I know, PEP 695 does not explicitly acknowledge anywhere that the proposals it outlines will make innovating in some parts of the typing system more difficult, let alone make the argument that this might be a good thing.
I actually agree with Jelle about pulling the brakes a bit. I find that if I haven’t thought about them for a while, it takes me a long time to reconstruct an understanding of TypeVarTuple and ParamSpec in my head. I don’t have the same problem with basic type vars or contravariance or even the weird AnyStr thing.
I don’t think it is the place of PEP 695 to take this kind of thing in consideration though. That’s the task of the Steering Council, which is explicitly given the mandate of looking out for the big picture, for the whole community, for the long haul. And I think they’ve made a very sensible decision in this case.
Maybe there’s no consensus in the typing community about this yet, but I suspect that in the larger Python community the desire to slow down is greater than the need for new typing features. Obviously this is a bit disappointing for typing folks, but I think it’s important to have some balance with the rest of the community, many of whom are still in the “typing is not for me” stage.
The latter. Language design is a marathon, not a race, so I’m personally fine waiting a bit to see what needs crop up after PEP 695 gets used in the community (including people clamouring for PEP 696 and its syntax proposal). Then again, I might not be on the SC anymore at the point.
But the problem isn’t. For me, the age of the proposal didn’t play into approving PEP 695, it was the benefits of how the syntax improves a situation and for the size of the audience who will benefit with the cost of asking Python developers to learn the new syntax.
PEP 696 could maybe be an opportunity to specify how generic function arguments with default values could be made safe.
The current situation is that pyright accepts the following without complaint:
from typing import TypeVar
T = TypeVar("T", bound=int | None)
def f(x: T = None) -> T:
return x
reveal_type(f()) # None
reveal_type(f(1)) # int
reveal_type(f(None)) # None
whereas mypy complains with
main.py:5: error: Incompatible default for argument "x" (default has type "None", argument has type "T") [assignment]
main.py:8: note: Revealed type is "<nothing>"
main.py:9: note: Revealed type is "builtins.int"
main.py:10: note: Revealed type is "None"
There is a long thread about this in the mypy repo and as I understand it, the main concern for supporting this is performance (and considerations like what happens in type stubs).
But what if defaults were only allowed for arguments with generic types that have a default type?
from typing_extensions import TypeVar
T = TypeVar("T", bound=int | None, default=None)
def f(x: T = None) -> T: # OK
return x
reveal_type(f()) # None
def g(x: T = 0) -> T: # type error
return x
This seems to solve the concerns nicely. The rule would be something like “if an argument has a default value and its type is a type var, then the type var must have a default and that default must be compatible with the default value.”
Something like the following would be unaffected:
T = TypeVar("T", bound=int)
class A(Generic[T]):
def __init__(self, x: T | None = None): ...
I was just reading over PEP 696 and had an editorial suggestion. Right in the abstract, the PEP uses the term TypeVarLike and puts that in monofont. I scratched my head for a bit because I didn’t remember what that symbol refers to [1], so I did several searches looking for it, and asked around a bit. I think in fact, PEP 696 introduces this term, since it shows up nowhere else that I’m aware of.
I think the PEP also tries to roughly define this in the following parenthetical text:
This PEP introduces the concept of type defaults for TypeVarLike s (TypeVar , ParamSpec and TypeVarTuple ),
I suggest that the PEP be rephrased to make it clearer that TypeVarLike isn’t an actual name of something that exists, but instead a description of a class of typing symbols. I.e. “TypeVar-like” might be a better way of describing this concept. It might even help to be explicit about what that term means, not just by way of example, but by describing the category of things that the term refers to.
