Permitting more precise inferred types from ambiguous overloads

Typing
11

While this proposal permits more precise inferred return types than the current spec, mypy, pyright, and pyrefly all deviate from the overload evaluation section of the spec, and at least for pyrefly, this would force us to produce less precise types across the board.

I ran pyrefly on mypy primer with its default semantics (choose a return type that all materializations of all other candidate return types are assignable to, or Any if such a return type doesn’t exist), then with those semantics plus requiring that the chosen return type be assignable to all other candidate return types (let’s call this “safe” semantics). There were changes in 122 projects, and the error delta was:

Error kind                     Added  Removed      Net
---------------------------  -------  -------  -------
unknown-argument-type           6507      239    +6268
unknown-variable-type           3365       83    +3282
implicit-bool                     92      352     -260
assert-type                      300       19     +281
no-any-return-implicit           260        0     +260
unsupported-operation             74      184     -110
bad-argument-type                 39      200     -161
missing-attribute                 22      216     -194
[remaining error kinds had less than 100 added or removed errors apiece]
---------------------------  -------  -------  -------
Total                          10842     1679    +9163

The dominant impact is added errors like unknown-argument-type from producing more Anys, although there are also some removed errors, like unsupported-operation, which could be fixed false positives. (Note: the error kinds for unknown types are off-by-default, so I’m not claiming that pyrefly would actually produce 11k more errors, just using the errors as a measure of how types would shift.)

I also took a closer look at two specific projects, numpy and scipy-stubs, which both have a lot of overloaded functions and extensive typing test suites, and found plenty of examples where an assert_type expected a type that would violate the proposed new requirement.

(Everything from this point onward is an analysis that I was originally compiling for Spec change: ambiguous arguments in overload call evaluation . I kept putting off posting it because I didn’t have a good solution to propose. These numbers are about 2 months old.)

I ran pyrefly on numpy and scipy-stubs’ typing tests with default semantics, then safe semantics, and examined new assert_type failures. There were 169 new failures, and the categories they fell into were:

  • In 81 cases (48%), one of the overloads returned a union containing the other return types. E.g., A | B, A, B => A | B is selected.

  • In 56 cases (33%), one of the overloads returned a union with Any, which caused the other return types to be assignable to it. E.g. A | Any, B => A | Any is selected.

  • 29 cases (17%) were the union case but with the union in a type argument. E.g., X[A | B], X[A], X[B] => X[A | B] is selected.

  • Finally, in 3 cases (2%), one of the overloads returned a variable-length tuple and the others fixed-length tuples. E.g., tuple[A, ...], tuple[A, A], tuple[A, A, A] => tuple[A, ...] is selected.

For what it’s worth, mypy and pyright seem to get the expected answer in these cases not by picking intelligently but by selecting the first matching overload [1], and the overloads are ordered so the most general one comes first. Regardless, the pattern seems to be that rather than all of the overload signatures being equal, so to speak, one of them is intended to be a fallback. Requiring that the inferred return type be assignable to all of the return types would prevent a type checker from choosing the fallback in the above cases. The mypy primer numbers suggest that this isn’t an isolated issue.

[1] Example of pyright picking the first overload in an ambiguous case:

from typing import Any, overload

class A[T]: ...

@overload
def f(x: A[int]) -> int: ...
@overload
def f(x: A[str]) -> str: ...
def f(x) -> Any: ...

def g(x: A[Any]):
    reveal_type(f(x))  # int

Example with mypy (note that the type alias seems to be needed to hide the Any from mypy):

from typing import Any

class A[T1, T2]: ...

type X[T] = A[Any, T]

from typing import overload
@overload
def f(x: A[int, int]) -> int: ...
@overload
def f(x: A[str, int]) -> str: ...
def f(x) -> int | str:
    return 0
    
def g(x: X[int], y: A[Any, int]) -> None:
    reveal_type(f(x))  # int
    reveal_type(f(y))  # but this is `Any`!
2 Likes