This seems like something that could be published first as an extension to a type checker (presumably mypy) and gain some real world use before it gets committed into a standard?
You can’t get any real “real world” use out of this – sure, you might be able to check typing in CI with a plugin but then your IDE experience is all red underlines. You can’t ask users to participate in an experimental project “hey, here’s a new API I made, please use it with all these restrictions, I need to gather data for Python core devs”. This is just impractical and will not give you any actionable data.
Perhaps what we really need is a better extension model for type checkers, so that libraries that need this level of complexity can provide something separate from their code but still in the repository/package and accessible?
Typechecker extensions are 100% not the way to go. We will not arrive to a future where all typecheckers implement one plugin API, this just won’t happen (mypy is in Python, ty is in rust, pyright is TS), but let’s assume it did:
- Now your library requires a plugin
- As a library author your job is tough – things go wrong and you have to debug your code base AND your plugin. Not to mention the author will now have to maintain two disjointed code parts – the library and the plugin to work with it. So much friction.
- As a library user your life is tough – things go wrong and it’s unlikely you will be able to debug and test the plugin
- Plugins/extensions will not reduce the complexity, they will in fact greatly increase it for everyone involved (library authors, typechecker maintainers, users).
- Going this path basically pronounces that we are giving some of the control over how Python language evolves to typechecker maintainers. Python will become severely tied to typecheckers capabilities and extension APIs. This will add friction and confusion.
Last but not least… typecheckers already have an “extension API” they all have to support – that the typing module! Let’s enhance the existing API instead of inventing a new super low level one.
Not going to lie, this is not a compelling example to me. I don’t want to see this in code, scroll past it, or implement a static evaluator in a type checker to calculate the result (that last one is probably the most valid feedback, and yes, my job used to involve implementing static evaluators in type analysis engines for Python
).
In all fairness it’s highly unlikely you’d see that code! You will likely see something like this:
def select[ModelT, K: typing.BaseTypedDict](
typ: type[ModelT],
/,
**kwargs: Unpack[K],
) -> ComputeSelectReturnType[ModelT, K]:
raise NotImplementedError
And if you’re curious how typing is plugged in here you’d ctrl+click onto the ComputeSelectReturnType type and learn. For ORMs specifically, the complexity of dealing with databases, generating SQL, generating migrations and reflecting tables to Python classes absolutely dwarfs the complexity of the little typing code we propose to add. I’m saying this as someone who spent their career in messing with dynamic code like this.
I’d like to steer this discussion to a place where we don’t judge this proposal from the position of feelings of complexity. Complexity is there, for sure, but in reality this PEP isn’t making an unimaginable leap of faith, instead it does: allow if/for in type expressions; add a number of primitives to the typing module. I’m not saying this is trivial, it’s far from it, but this is nowhere near the complexity of Python metaclass machinery or adding something like yield/async/await to the language. All I’m saying is that gut feelings and “I don’t want to see this in code” will not get us far. Not trying to pick on your comment Steve (I like you a lot!) but I already see this kind of sentiment in some discussions and I’d like to see less of it as it’s not actionable.