Merge typed_ast back into CPython

@encukou
Maybe you didn’t realize typed_ast is a fork of ast, with exactly two things added? It adds fields to certain nodes that hold the type comment, and it adds a bitmap indicating which lines contain # type: ignore comments (which look like type comments but really are a different thing – they can occur in places where a type comment would not be legal). Note that both type comments and # type: ignore are described in a PEP. This is not your ordinary third-party package.

Most of the maintenance to typed_ast is keeping it up to date with changes to Python’s syntax for each new Python release. Currently this must either be done by starting afresh with a new fork of ast, and then re-applying the (very stable) changes to support type comments, or by taking the entire diff between ast for two adjacent Python versions and manually applying it to typed_ast. Bot of these processes are difficult due to code reformatting and unrelated other changes to the ast module in the CPython repo.

In addition, flake8 currently uses the ast module, and it would benefit from having the type comments: an import that is only used in a type comment should not be counted as unused. There are probably other tools that would benefit from having the type comments available too (though IIRC pylint has its own parser, Astroid).

The feature could be extended to e.g. keeping track of # noqa comments (which flake8 currently parses separately). Those are more like # type: ignore than like type comments, in that they apply to arbitrary lines. They do optionally allow additional text, but that’s simple to solve.

1 Like