Specifying invalid signatures in overloads

Typing
1

During the last typing meetup @erictraut mentioned that for time to time people look for ways to mark a specific overload as invalid. It often leads to developers using NoReturn / Never for it which is incorrect. See also the great video by anthony on that topic: https://www.youtube.com/watch?v=WuXRn3euN8k

Example: How to annotate func so it doesn’t accept normal strings as input and only sequences?

def func(arg: Sequence[str]) -> int: ...

func(["Hello", "World"])  # ok
func("abc")  # should be an error

After thinking about it, I believe this might be fairly simple to archive if we’re to add a new SpecialForm which could be used as return type annotation for these cases, say NoMatch. Since the overload matching happens based on the arguments, it wouldn’t need to change. Type checkers just need to check afterwards if the matched return type is NoMatch and emit an error accordingly. The overloads for the case above would then look like this

@overload
def func(arg: str) -> NoMatch: ...
@overload
def func(arg: Sequence[str]) -> int: ...

An obvious advantage would be that this closely mirrors what people expect would work with NoReturn.

Open questions

  1. Is this something worth adding support for?
  2. Is the return type annotation the best option? In passing, Eric mentioned maybe adding an @add_error decorator.
  3. If return type annotation, is NoMatch the best name?

Prototype

I prototyped the implementation in mypy. For tests, it can be imported from from typing_extensions import NoMatch.

5 Likes
2

IMO, yes, but it shouldn’t be restricted to @overload, but also work for e.g. __bool__ implementations that raise an error, see a recent discussion on this forum. (Yes, this encourages violations of the LSP. But static typing needs to describe real world code, and such code does often violate the LSP.)

I think a decorator is cleaner than a type. After all, this isn’t really a type in the normal sense - the return type continues to be Never since the implementation presumably raises an error, i.e. “returns” the bottom type. And adding a third type to the list NoReturn, Never that seems very similar is going to be unnecessarily confusing. And we also already have something similar with @deprecated

For a name, I am honestly not sure. Maybe @error("Message"), which is similar to constructs in other languages with a related purpose? (See the C PreProcessor or nim pragmas). But having a public symbol just named error is a bit confusing.

1 Like
3

Note that there’s a need for this in the standard library.

This currently doesn’t give any type error from type checkers, but it crashes with TypeError:

from collections import Counter


my_counter = Counter({"a": "b"})
my_counter.total()
4

This isn’t incorrect, but the current consequences of it aren’t what people expect the outcome to be. The distinction here is important because it means we can fix this without needing any additional type-theory or LSP violations. I think it would be better to fix this by making the behavior people expect here for callers to actually happen.

There’s 3 ways I see to fix this, and 1 non-fix

  • In the prior thread about numpy and __bool__, I mentioned generalized algebraic effects. This would be the “ideal”, but it’s also not something that would be trivial to add to python’s type system.

  • Add a New special form that means “This is Never for the purposes of the type system, but when it’s the return type, The caller is using it wrong, and they should get an error about this.” (I explored this option as part of an on-hold experiment before, and believe it’s viable here)

  • Add intersections and type negation so that overloads can be specified more accurately to avoid some of the need. This won’t address every case of the use of Never, but will cover that of current use in only some overloads.

  • continue with the status quo, docs + runtime errors are enough, and Never is correct from the type-theory perspective.

5

Do you have any more motivating examples? Isn’t the string one handled by useful_types.SequenceNotStr?

6

useful_types.SequenceNotStr is problematic for multiple reasons, as the way it tries to get clever to avoid working with str breaks other checks by being too broad, and those are much more important than the str vs Sequence[str] check. I would suggest people not use this.

7

I think the current proposal is a bit too narrow. I think the feature would also be useful in non-overloaded functions.

When writing third party stubs for typeshed there’s a pattern I came across quite a few times, which also happens to match the use-case the prior thread about __bool__ inquired about. I.e. overwriting a method in a subclass with an implementation that always raises and having the type checker understand, that it’s an error to call that method on that type.

To be fair I think of this as an anti-pattern, since you’re potentially creating hidden mines in the supertypes for people to step on, but nevertheless, given that these types already exist, it would be more useful for type checkers to be able to emit a custom error message when one of these methods is called on a type, where it’s known to be an error.

So I would propose using the previously suggested @type_error decorator[1], which mirrors the behavior of @deprecated. Except it is treated as a generic error.

The other advantage of using a decorator, is that you can control what the fallback type will be when the type checker matches this overload[2].

A decorator like this also opens up some additional design space for making this feature even more flexible, a couple of possibilities come to mind:

  1. Optional level parameter. In order to be able to emit lower confidence/severity errors that can be turned off by default.

  2. Optional category parameter. In order to give people the ability to silence a specific kind of @type_error, e.g. str-as-iterable via type: ignore[str-as-iterable], but not others.

Beyond that I think it’s important to specify what happens with overlapping overloads, where only one of them is marked an error.

This sounds trivial at first, but has similar subtle complexities like the overload system already does. E.g. imagine a function with overloads for Iterable[T], str and Literal["foo"], but only str is an error. All three overlap, but intuitively you only want this to emit an error when you pass a str that isn’t "foo".

Maybe this is too specialized, so it’s not worth supporting complex cases like this, but either way there should at least be a sketch of the desired logic for emitting errors with overlapping overloads.

  1. I think Jelle first came up with that name, but I might be wrong, so please don’t be mad if you’re the person that actually came up with this idea ↩︎

  2. In most cases you would probably want it to be Never, but I can think of cases where you use @type_error for something that doesn’t actually produce a runtime error, like passing str into (Iterable[T]) -> list[T], you would still want the return type to be list[str], but you would also want to emit an error ↩︎

8

If the overloads were defined in the order as you’ve mentioned, I believe Iterable[T] is a full overlap (unless T is constrained) for str, and str is a full overlap for Literal["foo"]. A type checker should emit an error at the overload definition site, since the other two won’t be ever matched.

The solution should be to reverse the order, so the first overload catches Literal["foo"] (and emits an error due to @type_error)

9

I wasn’t really suggesting to define the overloads in that order, the most specific overload should of course generally come first if it overlaps with a broader one.

I agree that something clear-cut like this would be nice. Unfortunately, the order of the overloads matters remarkably little[1]. See Eric’s draft of the overloads chapter with a sketch of how overload resolution actually works today: https://github.com/python/typing/pull/1839

Whether or not a type checker would emit an error at the definition site for partially overlapping overloads depends on the return types of each overload, but a @type_error doesn’t necessarily result in a conflicting return type.

I mostly brought this up to illustrate that the rules for @type_error would need to be carefully drafted, so @type_error does the most useful thing, without making the overload matching algorithm even more complicated and difficult to understand.

At a glance, the most useful behavior to me would be, after eliminating all the non-matching overloads, to emit the error if the first remaining candidate is decorated with @type_error, regardless of whether or not it conflicts with one of the later remaining candidates, without changing how the return type is determined in the ambiguous case.

But I haven’t thought hard enough about it to be confident, that there’s no issues with doing things this way. PEP 702 appears to work that way, although the wording is so vague, I don’t really know what was intended to happen in the ambiguous case. For deprecations that seems fine, since it doesn’t seem very likely that you will hit the ambiguous case. But for type_error I expect ambiguous cases to be much more common.

  1. I think this is a real shame, a more simple rule like “First match” wins would have been a lot easier to implement and understand, but it’s kind of difficult to change course at this point, since it has worked differently for so long now ↩︎

10

I did read it prior, but I don’t think I understand your point.

That might be true, but it definitely matters in determining the validity of overlapping overloads, which is relevant to my previous comment.

Now, for my own sake I’ll try to go over the algorithm given the following overloads:

@overload
@type_error
def foo(s: Literal["foo"], /) -> Never: ...
@overload
def foo(s: str, /) -> list[str]: ...
@overload
def foo[T](s: abc.Iterable[T], /) -> list[T]: ...

…and a call of foo("foo").

  1. step shouldn’t eliminate any of the overloads, since all have 1 positional param, and there’s 1 positional argument.

  2. step evaluates the overloads like independent function calls, and records errors.
    Presumably, this is when the overload with @type_error would be noted as producing an error, but since the two other overloads match and do not produce one, evaluation continues.

  3. step doesn’t apply, since not all overloads result in an error.

  4. step doesn’t apply, since there’s no *args nor **kwargs.

  5. step replaces the T with str. Since the two remaining overloads have equivalent return type, go to 6.

  6. step chooses the 2nd overload ((str) -> list[str]) as winning match.

It seems to me that under those rules, the 2nd step would discard the first overload due to the error, and the evaluation proceeds normally.
The overload resolution would need to specifically check for @type_error at the 2nd step (and possibly later?) for this to work.

11

You basically illustrated my point with your example. @type_error would be pretty useless if it worked like that, that’s kind of what I was getting at.

So for the purposes of overload resolution a @type_error shouldn’t be able to eliminate an overload. Otherwise that overload would never be matched.

But there’s a problem with the ambiguous return types part in step 5. Since you stop the algorithm and return Any no overload gets chosen, so it would once again follow that no error is emitted.

But that’s also kind of bad[1], so you probably still want to do step 6 anyways for the purposes of emitting a potential @type_error or @deprecated.

But you could also make arguments for different behaviors. This is just the one that makes the most sense to me.

  1. even though you would get an error at the definition site for the partial overload with incompatible returns ↩︎

12

What if the rules were as follows:

  1. If you have a list of @overload and you mark one or more of them as @type_error, all of the overloads marked as @type_error must come before the overloads without @type_error.
  2. When matching a call to an overloaded function/method, if @type_error is present, in the overloads, the type checker first tries to match any of the ones labeled as @type_error using the same rules as is done today with overloads, but the overloads without @type_error are ignored. If any match, an error is produced.
  3. If none of the @type_error overloads match, then they are not considered further, and regular overload matching is applied with the remaining overloads.

So for this example:

@overload
@type_error
def foo(s: Literal["foo"], /) -> Never: ...
@overload
@type_error
def foo(s: list[Literal["foo", "goo"]], /) -> Never: ...
@overload
def foo(s: str, /) -> list[str]: ...
@overload
def foo[T](s: abc.Iterable[T], /) -> list[T]: ...

foo("foo")
foo(["foo"])
foo(["goo"])
foo(["foo", "goo"])
foo(["foo", "goo", "moo"])

Then the error would be found in the first 4 calls, but not the last one.

Also, the following code would be rejected by the type checker because all of the @type_error overloads are not first:

@overload
@type_error
def foo(s: Literal["foo"], /) -> Never: ...
@overload
def foo(s: str, /) -> list[str]: ...
@overload
@type_error
def foo(s: list[Literal["foo", "goo"]], /) -> Never: ...
@overload
def foo[T](s: abc.Iterable[T], /) -> list[T]: ...
13

Can this system address this one? Specifying invalid signatures in overloads - #3 by beauxq

The requirement to match errors first looks like it makes it less useful.

For Counter.__init__, it seems to me we would want to say that Mapping[T, int] succeeds, but after that doesn’t succeed, any other Mapping[T, object] fails.
But if we check for the failure Mapping[T, object] first, then Mapping[T, int] will fail when it shouldn’t.

14

I see your point. So maybe this would work:

  1. If you have a list of @overload and you mark one or more of them as @type_error, all of the overloads marked as @type_error must be in a single group, i.e., either:
  • the first set of overloads with @type_error is followed by overloads without @type_error; or,
  • all the overloads without @type_error come first, followed by a set of overloads without @type_error.
  1. When matching a call to an overloaded function/method, the type checker first tries to match everything in the first group, which will either have all @type_error overloads, or none. If there is a match among those overloads, then do not try to match the overloads in the second group. If there is no match among the first group, move to the second group, and no longer consider the first group.
  2. if @type_error is present, in the overloads (whether in the first group or the second group), the type checker tries to match any of the ones labeled as @type_error using the same rules as is done today with overloads. If any match, an error is produced.

So I think my example above will work with this rule, and you could do Counter.__init__() as follows (leaving out some of the other matches):

@overload
def __init__(self, Mapping[T, int]): ...
@overload
@type_error
def __init__(self, Mapping[T, object]): ...

So then you have 2 options as someone writing overloads. Either you put the group of overloads that are acceptable first, and errors second, or you put the ones that are errors first, and ones that are acceptable second.

15

Then where does the Iterable[T] go? (the reason that Mapping[T, object] matches)

If it’s in the first group, then Mapping[T, object] will still be accepted when it shouldn’t be.
If it’s not in the first group, then an Iterable[T] that should be accepted, will not be accepted.

16

OK, here’s yet another proposal. I also realized that putting the errors in the second group of two doesn’t really matter - if you don’t match the first group, it’s an error anyway.

  1. When using @type_error, the overloads with @type_error must be grouped. They can be in the first group, or the second group. If in the second group, a third group of overloads (without @type_error) is required.
  2. Overload matching is done by group, using the current rules. If there is no match within a group, then the next group is considered for matching, and matching continues as if the prior groups did not exist.
  3. If @type_error is present in the overloads (whether in the first group or the second group), the type checker tries to match any of the ones labeled as @type_error using the same rules as is done today with overloads. If any match, an error is produced.

So then I think Counter.__init__() could be done as:

@overload
def __init__(self, Mapping[T, int]): ...
@overload
@type_error
def __init__(self, Mapping[T, object]): ...
@overload
def __init__(self, Iterable[T]): ...

First it tries to match Mapping[T, int]. If that is not the case, the Mapping[T, object] is checked. If a match, then an error is emitted and matching stops. Then it tries matching Iterable[T].

The group concept is key. You don’t have to worry about overlaps across groups, as each group is considered independently, in order.

1 Like