Type annotations in the stdlib

Core Development
7

(I’m a maintainer of typeshed and mypy)

There are three use cases for type annotations in the standard library:

  • used by type checkers to type check downstream user code (aka make type checkers use stdlib instead of typeshed)
  • used for the purpose of alerting core devs to potential bugs (aka let core devs run mypy on stdlib)
  • used by end users as documentation

These three things actually have somewhat different requirements, so +1 to the suggestion to tread carefully.

The thing I feel strongest about is that it would be a mistake to have type checkers use annotations from the standard library instead of typeshed. Here are some reasons:

  • typeshed has a much faster pace of development and lower stability than the standard library. All typeshed changes are made available to users of all versions of Python.
  • Moreover, we’re able to use typing features added to typing.py in new versions of Python on all versions of Python
  • The development cycle of typeshed and type checkers is linked in several ways. It is common for typeshed to workaround specific issues in type checkers. It is common for type checkers to patch typeshed (mypy definitely does this, I think pyright does as well).
  • The tests and tooling we have for type annotations in typeshed would be hard to get merged into CPython (e.g. at a minimum you’d want to make sure that type annotations make sense to a type checker)
  • The expertise is split. Core devs typically know less about typing than typeshed maintainers and suddenly core devs would be responsible for typing of their modules (when many core devs aren’t sold on typing to begin with)
  • Accurate typing is hard! Adding typeshed quality annotations to the standard library would add e.g. a bunch of new protocols and aliases and annoying Literal types. typeshed is also forced to be opinionated in certain ways (think union returns, collection types, use of Any, deciphering intentions) and I think these decisions would be harder and harder to reverse if part of standard library
  • Accurate type annotations may not be zero runtime cost. At a minimum you’re going to have more import cycles. You could if TYPE_CHECKING some of those away, but then runtime users of annotations might be surprised that typing.get_type_hints or whatever raises on stdlib
  • No type checker authors are asking for this. This would necessitate changes in all type checkers
  • You’d still need to use stubs for all the modules written in C, so not sure how much “cohesiveness” this would actually get you
  • etc etc etc

Seems great if core devs find typing useful enough that they’d like to use it in their workflow. Some thoughts:

  • There are already a couple modules in the standard library that have type annotations for this purpose
  • This workflow seems to work best for modules that are developed somewhat separately from CPython. In particular, in repos where type checking is run as part of tests
  • Alex has a fair point about users mistaking such type hints for “type hints that type checkers use”, but for now this issue isn’t too bad, and at the current scale of use, other issues that crop up seem solvable with tooling

Thoughts about type hints as documentation:

15 Likes