This is mainly continuation of comments in TypeGuard 724 PEP discussion. A lot of the recent comments are less about PEP and more about general typing stability expectations so feel it would be better to continue discussion here and leave original topic about TypeGuard proposal.
Some of my core thoughts are covered here and this extends few them and one proposal for a way of handling typing stability perception.
I agree strongly with this specific quote. I think there are three core aspects of type system today.
- Standard library typing features: This includes TypedDict, Paramspec, Annotated, overload, and many more features defined in python standard library and has runtime impact. This also includes syntax changes like improved generic typing/union support. This is partly coupled to python language although typing-extensions is very helpful way to decouple some typing features.
I think standard library/runtime typing changes should have high stability expectation and am in agreement with other comments that typing should follow similar stability promises as core language.
Type System/Checker Behavior: This covers type specifications for how inference works and many type checkers mypy, pyright, pyre, pytype, pyanalyze, and more we have today. The current TypeGuard PEP I consider to fall entirely in 2 as written. There are no standard library/runtime changes involved here and each type checker may have it’s own policy and expectations for backwards compatibility. While mypy is very important and useful library it is not python language and similar to how other foundational libraries (numpy, django, flask, matplotlib) do not evolve through PEPs, type checkers should also be able to evolve without PEPs and determine their own stability expectations.
Types in libraries/stubs: Typeshed is core part of this along with growing number of py.typed libraries. This also leads to typing instability especially for libraries where types are still relatively new/evolving frequently. Here I think individual libraries own both their runtime and types and are not part of typing council/language’s stability expectations.
I think distinction between 1 vs 2 is partly blurry for typing evolution because new typing features commonly jump to PEPs quickly. When a new feature like TypeGuard, Paramspec, TypeVarTuple, Intersection types, and more are being discussed it is often with plan that after discussion immediately work on PEP and then implement the PEP. Often the type checker implementation for PEP comes months/year+ after PEP is accepted. For a complex feature like TypeVarTuple how can we be confident in specification and inclusion in standard library if users can’t use feature until long after PEP has been accepted? And by bringing PEP as a proposal to language immediately I think it leads to higher stability/implied consensus on typing then present.
Instead my proposal would be for moderate/large typing features (intersections, paramspec, typevartuple all good examples) they should be proposed and agreed on within type checkers/typing ecosystem without standard libary support/PEP. The feature would still have a proposal document (TEP/Typing Enhancement Proposal) that Typing Council could review. A TEP could be structured very similar to PEP with main difference only being TEPs should never change standard library and are restricted to type checker/type system changes.
For these features it is then possible for them to be implemented in typing_extensions.experimental (or another library) and to have experimental phase. How long depends on complexity of that feature and time for it to be implemented and used. After users/type checkers have experience with the feature then the TEP can be converted to PEP with adjustments based on feedback. At that stage the feature can move to standard library and be viewed as more stable.
For complex feature like Intersections I think it will be very hard to have confident specification/understanding of many interactions it is with type system without experience using feature. Paramspec similarly have core behaviors (how does paramspec and method interact) that are undecided upon but likely would have been discovered if usage of the feature was possible before PEP acceptable. My code idea is to enable typing features to develop in clear separation from standard library during experimental phase and when ready for more stability/user expectations then it moves to standard library. Users who are comfortable using typing features on edge can explore them similar to how features are often added to typing-extensions before PEP is accepted. Other users can wait and have reasonable policy that only typing features that have reached PEP stage and graduated should be used.