# Specifying invalid signatures in overloads **URL:** \ **Category:** Typing\ **Created:** [December 27, 2024, 12:34am UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405 "2024-12-27T00:34:24Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![cdce8p](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cdce8p/32/3575_2.png) [@cdce8p](https://discuss.python.org/u/cdce8p)\ **Post date:** [December 27, 2024, 12:34am UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/1 "2024-12-27T00:34:24Z")
During the last typing meetup @erictraut mentioned that for time to time people look for ways to mark a specific overload as invalid. It often leads to developers using `NoReturn` / `Never` for it which is incorrect. _See also the great video by anthony on that topic: [https://www.youtube.com/watch?v=WuXRn3euN8k](https://www.youtube.com/watch?v=WuXRn3euN8k)_ Example: How to annotate `func` so it doesn’t accept normal strings as input and only sequences? ```py def func(arg: Sequence[str]) -> int: ... func(["Hello", "World"]) # ok func("abc") # should be an error ``` After thinking about it, I believe this might be fairly simple to archive if we’re to add a new SpecialForm which could be used as return type annotation for these cases, say `NoMatch`. Since the overload matching happens based on the arguments, it wouldn’t need to change. Type checkers just need to check afterwards if the matched return type is `NoMatch` and emit an error accordingly. The overloads for the case above would then look like this ```py @overload def func(arg: str) -> NoMatch: ... @overload def func(arg: Sequence[str]) -> int: ... ``` An obvious advantage would be that this closely mirrors what people expect would work with `NoReturn`. #### Open questions 1. Is this something worth adding support for? 2. Is the return type annotation the best option? In passing, Eric mentioned maybe adding an `@add_error` decorator. 3. If return type annotation, is `NoMatch` the best name? #### Prototype I prototyped the implementation in mypy. For tests, it can be imported from `from typing_extensions import NoMatch`. > > > https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405 ---
**Author:** ![MegaIng](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/megaing/32/16162_2.png) [@MegaIng](https://discuss.python.org/u/MegaIng)\ **Post date:** [December 27, 2024, 1:19am UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/2 "2024-12-27T01:19:02Z")
> [@cdce8p](#): > > Is this something worth adding support for? IMO, yes, but it shouldn’t be restricted to `@overload`, but also work for e.g. ` __bool__ ` implementations that raise an error, see a recent discussion on this forum. (Yes, this encourages violations of the LSP. But static typing needs to describe real world code, and such code does often violate the LSP.) > [@cdce8p](#): > > Is the return type annotation the best option? In passing, Eric mentioned maybe adding an `@add_error` decorator. I think a decorator is cleaner than a type. After all, this isn’t really a type in the normal sense - the return type continues to be `Never` since the implementation presumably raises an error, i.e. “returns” the bottom type. And adding a third type to the list `NoReturn`, `Never` that seems very similar is going to be unnecessarily confusing. And we also already have something similar with `@deprecated` For a name, I am honestly not sure. Maybe `@error("Message")`, which is similar to constructs in other languages with a related purpose? (See the C PreProcessor or nim pragmas). But having a public symbol just named `error` is a bit confusing. ---
**Author:** ![beauxq](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/beauxq/32/15042_2.png) [@beauxq](https://discuss.python.org/u/beauxq)\ **Post date:** [December 27, 2024, 1:28am UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/3 "2024-12-27T01:28:29Z")
Note that there’s a need for this in the standard library. This currently doesn’t give any type error from type checkers, but it crashes with `TypeError`: ```python from collections import Counter my_counter = Counter({"a": "b"}) my_counter.total() ``` ---
**Author:** ![mikeshardmind](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/mikeshardmind/32/14381_2.png) [@mikeshardmind](https://discuss.python.org/u/mikeshardmind)\ **Post date:** [December 27, 2024, 1:58am UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/4 "2024-12-27T01:58:04Z")
> [@cdce8p](#): > > It often leads to developers using `NoReturn` / `Never` for it which is incorrect. This isn’t incorrect, but the current consequences of it aren’t what people expect the outcome to be. The distinction here is important because it means we can fix this without needing any additional type-theory or LSP violations. I think it would be better to fix this by making the behavior people expect here for callers to actually happen. There’s 3 ways I see to fix this, and 1 non-fix - In the prior thread about numpy and ` __bool__ `, I mentioned generalized algebraic effects. This would be the “ideal”, but it’s also not something that would be trivial to add to python’s type system. - Add a New special form that means “This is Never for the purposes of the type system, but when it’s the return type, The caller is using it wrong, and they should get an error about this.” **(I explored this option as part of an on-hold experiment before, and believe it’s viable here**) - Add intersections and type negation so that overloads can be specified more accurately to avoid some of the need. This won’t address every case of the use of Never, but will cover that of current use in only some overloads. - continue with the status quo, docs + runtime errors are enough, and Never is correct from the type-theory perspective. ---
**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\ **Post date:** [December 27, 2024, 2:02am UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/5 "2024-12-27T02:02:53Z")
Do you have any more motivating examples? Isn’t the string one handled by `useful_types.SequenceNotStr`? ---
**Author:** ![mikeshardmind](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/mikeshardmind/32/14381_2.png) [@mikeshardmind](https://discuss.python.org/u/mikeshardmind)\ **Post date:** [December 27, 2024, 2:17am UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/6 "2024-12-27T02:17:08Z")
`useful_types.SequenceNotStr` is problematic for multiple reasons, as the way it tries to get clever to avoid working with str breaks [other checks](https://github.com/hauntsaninja/useful_types/issues/25) by being too broad, and those are much more important than the str vs Sequence[str] check. I would suggest people not use this. ---
**Author:** ![Daverball](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/daverball/32/15931_2.png) [@Daverball](https://discuss.python.org/u/Daverball)\ **Post date:** [December 27, 2024, 7:51am UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/7 "2024-12-27T07:51:19Z")
I think the current proposal is a bit too narrow. I think the feature would also be useful in non-overloaded functions. When writing third party stubs for typeshed there’s a pattern I came across quite a few times, which also happens to match the use-case the prior thread about ` __bool__ ` inquired about. I.e. overwriting a method in a subclass with an implementation that always raises and having the type checker understand, that it’s an error to call that method on that type. To be fair I think of this as an anti-pattern, since you’re potentially creating hidden mines in the supertypes for people to step on, but nevertheless, given that these types already exist, it would be more useful for type checkers to be able to emit a custom error message when one of these methods is called on a type, where it’s known to be an error. So I would propose using the previously suggested `@type_error` decorator\[1\], which mirrors the behavior of `@deprecated`. Except it is treated as a generic error. The other advantage of using a decorator, is that you can control what the fallback type will be when the type checker matches this overload\[2\]. A decorator like this also opens up some additional design space for making this feature even more flexible, a couple of possibilities come to mind: 1. Optional `level` parameter. In order to be able to emit lower confidence/severity errors that can be turned off by default. 2. Optional `category` parameter. In order to give people the ability to silence a specific kind of `@type_error`, e.g. `str-as-iterable` via `type: ignore[str-as-iterable]`, but not others. * * * Beyond that I think it’s important to specify what happens with overlapping overloads, where only one of them is marked an error. This sounds trivial at first, but has similar subtle complexities like the overload system already does. E.g. imagine a function with overloads for `Iterable[T]`, `str` and `Literal["foo"]`, but only `str` is an error. All three overlap, but intuitively you only want this to emit an error when you pass a str that isn’t `"foo"`. Maybe this is too specialized, so it’s not worth supporting complex cases like this, but either way there should at least be a sketch of the desired logic for emitting errors with overlapping overloads. * * * 1. I think Jelle first came up with that name, but I might be wrong, so please don’t be mad if you’re the person that actually came up with this idea 2. In most cases you would probably want it to be `Never`, but I can think of cases where you use `@type_error` for something that doesn’t actually produce a runtime error, like passing `str` into `(Iterable[T]) -> list[T]`, you would still want the return type to be `list[str]`, but you would also want to emit an error ---
**Author:** ![Eneg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/eneg/32/18177_2.png) [@Eneg](https://discuss.python.org/u/Eneg)\ **Post date:** [January 2, 2025, 2:04am UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/8 "2025-01-02T02:04:44Z")
> [@Daverball](#): > > E.g. imagine a function with overloads for `Iterable[T]`, `str` and `Literal["foo"]`, but only `str` is an error. All three overlap, but intuitively you only want this to emit an error when you pass a str that isn’t `"foo"`. If the overloads were defined in the order as you’ve mentioned, I believe `Iterable[T]` is a full overlap (unless `T` is constrained) for `str`, and `str` is a full overlap for `Literal["foo"]`. A type checker should emit an error at the overload definition site, since the other two won’t be ever matched. The solution should be to reverse the order, so the first overload catches `Literal["foo"]` (and emits an error due to `@type_error`) ---
**Author:** ![Daverball](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/daverball/32/15931_2.png) [@Daverball](https://discuss.python.org/u/Daverball)\ **Post date:** [January 2, 2025, 8:25am UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/9 "2025-01-02T08:25:59Z")
I wasn’t really suggesting to define the overloads in that order, the most specific overload should of course generally come first if it overlaps with a broader one. I agree that something clear-cut like this would be nice. Unfortunately, the order of the overloads matters remarkably little\[1\]. See Eric’s draft of the overloads chapter with a sketch of how overload resolution actually works today: [https://github.com/python/typing/pull/1839](https://github.com/python/typing/pull/1839) Whether or not a type checker would emit an error at the definition site for partially overlapping overloads depends on the return types of each overload, but a `@type_error` doesn’t necessarily result in a conflicting return type. I mostly brought this up to illustrate that the rules for `@type_error` would need to be carefully drafted, so `@type_error` does the most useful thing, without making the overload matching algorithm even more complicated and difficult to understand. At a glance, the most useful behavior to me would be, after eliminating all the non-matching overloads, to emit the error if the first remaining candidate is decorated with `@type_error`, regardless of whether or not it conflicts with one of the later remaining candidates, without changing how the return type is determined in the ambiguous case. But I haven’t thought hard enough about it to be confident, that there’s no issues with doing things this way. PEP 702 appears to work that way, although the wording is so vague, I don’t really know what was intended to happen in the ambiguous case. For deprecations that seems fine, since it doesn’t seem very likely that you will hit the ambiguous case. But for `type_error` I expect ambiguous cases to be much more common. * * * 1. I think this is a real shame, a more simple rule like “First match” wins would have been a lot easier to implement and understand, but it’s kind of difficult to change course at this point, since it has worked differently for so long now ---
**Author:** ![Eneg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/eneg/32/18177_2.png) [@Eneg](https://discuss.python.org/u/Eneg)\ **Post date:** [January 2, 2025, 7:14pm UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/10 "2025-01-02T19:14:19Z")
> [@Daverball](#): > > See Eric’s draft of the overloads chapter I did read it prior, but I don’t think I understand your point. > [@Daverball](#): > > the order of the overloads matters remarkably little. That might be true, but it definitely matters in determining the validity of overlapping overloads, which is relevant to my previous comment. * * * Now, for my own sake I’ll try to go over the algorithm given the following overloads: ```py @overload @type_error def foo(s: Literal["foo"], /) -> Never: ... @overload def foo(s: str, /) -> list[str]: ... @overload def foo[T](s: abc.Iterable[T], /) -> list[T]: ... ``` …and a call of `foo("foo")`. 1. step shouldn’t eliminate any of the overloads, since all have 1 positional param, and there’s 1 positional argument. 2. step evaluates the overloads like independent function calls, and records errors. Presumably, this is when the overload with `@type_error` would be noted as producing an error, but since the two other overloads match and do not produce one, evaluation continues. 3. step doesn’t apply, since not all overloads result in an error. 4. step doesn’t apply, since there’s no `*args` nor `**kwargs`. 5. step replaces the `T` with `str`. Since the two remaining overloads have equivalent return type, go to 6. 6. step chooses the 2nd overload (`(str) -> list[str]`) as winning match. It seems to me that under those rules, the 2nd step would discard the first overload due to the error, and the evaluation proceeds normally. The overload resolution would need to specifically check for `@type_error` at the 2nd step (and possibly later?) for this to work. ---
**Author:** ![Daverball](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/daverball/32/15931_2.png) [@Daverball](https://discuss.python.org/u/Daverball)\ **Post date:** [January 2, 2025, 9:29pm UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/11 "2025-01-02T21:29:07Z")
You basically illustrated my point with your example. `@type_error` would be pretty useless if it worked like that, that’s kind of what I was getting at. So for the purposes of overload resolution a `@type_error` shouldn’t be able to eliminate an overload. Otherwise that overload would never be matched. But there’s a problem with the ambiguous return types part in step 5. Since you stop the algorithm and return `Any` no overload gets chosen, so it would once again follow that no error is emitted. But that’s also kind of bad\[1\], so you probably still want to do step 6 anyways for the purposes of emitting a potential `@type_error` or `@deprecated`. But you could also make arguments for different behaviors. This is just the one that makes the most sense to me. * * * 1. even though you would get an error at the definition site for the partial overload with incompatible returns ---
**Author:** ![Dr-Irv](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dr-irv/32/16656_2.png) [@Dr-Irv](https://discuss.python.org/u/Dr-Irv)\ **Post date:** [June 18, 2025, 10:26pm UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/12 "2025-06-18T22:26:06Z")
What if the rules were as follows: 1. If you have a list of `@overload` and you mark one or more of them as `@type_error`, all of the overloads marked as `@type_error` must come before the overloads without `@type_error`. 2. When matching a call to an overloaded function/method, if `@type_error` is present, in the overloads, the type checker first tries to match any of the ones labeled as `@type_error` using the same rules as is done today with overloads, but the overloads without `@type_error` are ignored. If any match, an error is produced. 3. If none of the `@type_error` overloads match, then they are not considered further, and regular overload matching is applied with the remaining overloads. So for this example: ```python @overload @type_error def foo(s: Literal["foo"], /) -> Never: ... @overload @type_error def foo(s: list[Literal["foo", "goo"]], /) -> Never: ... @overload def foo(s: str, /) -> list[str]: ... @overload def foo[T](s: abc.Iterable[T], /) -> list[T]: ... foo("foo") foo(["foo"]) foo(["goo"]) foo(["foo", "goo"]) foo(["foo", "goo", "moo"]) ``` Then the error would be found in the first 4 calls, but _not_ the last one. Also, the following code would be rejected by the type checker because all of the `@type_error` overloads are not first: ```python @overload @type_error def foo(s: Literal["foo"], /) -> Never: ... @overload def foo(s: str, /) -> list[str]: ... @overload @type_error def foo(s: list[Literal["foo", "goo"]], /) -> Never: ... @overload def foo[T](s: abc.Iterable[T], /) -> list[T]: ... ``` ---
**Author:** ![beauxq](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/beauxq/32/15042_2.png) [@beauxq](https://discuss.python.org/u/beauxq)\ **Post date:** [June 18, 2025, 11:45pm UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/13 "2025-06-18T23:45:17Z")
Can this system address this one? [Specifying invalid signatures in overloads - #3 by beauxq](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/3) The requirement to match errors first looks like it makes it less useful. For `Counter. __init__ `, it seems to me we would want to say that `Mapping[T, int]` succeeds, but after that doesn’t succeed, any other `Mapping[T, object]` fails. But if we check for the failure `Mapping[T, object]` first, then `Mapping[T, int]` will fail when it shouldn’t. ---
**Author:** ![Dr-Irv](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dr-irv/32/16656_2.png) [@Dr-Irv](https://discuss.python.org/u/Dr-Irv)\ **Post date:** [June 19, 2025, 1:47pm UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/14 "2025-06-19T13:47:29Z")
> [@beauxq](#): > > The requirement to match errors first looks like it makes it less useful. I see your point. So maybe this would work: 1. If you have a list of `@overload` and you mark one or more of them as `@type_error`, all of the overloads marked as `@type_error` must be in a single group, i.e., either: - the first set of overloads with `@type_error` is followed by overloads without `@type_error`; or, - all the overloads without `@type_error` come first, followed by a set of overloads without `@type_error`. 1. When matching a call to an overloaded function/method, the type checker first tries to match everything in the first group, which will either have all `@type_error` overloads, or none. If there is a match among those overloads, then do not try to match the overloads in the second group. If there is no match among the first group, move to the second group, and no longer consider the first group. 2. if `@type_error` is present, in the overloads (whether in the first group or the second group), the type checker tries to match any of the ones labeled as `@type_error` using the same rules as is done today with overloads. If any match, an error is produced. So I think my example above will work with this rule, and you could do `Counter. __init__ ()` as follows (leaving out some of the other matches): ```python @overload def __init__ (self, Mapping[T, int]): ... @overload @type_error def __init__ (self, Mapping[T, object]): ... ``` So then you have 2 options as someone writing overloads. Either you put the group of overloads that are acceptable first, and errors second, or you put the ones that are errors first, and ones that are acceptable second. ---
**Author:** ![beauxq](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/beauxq/32/15042_2.png) [@beauxq](https://discuss.python.org/u/beauxq)\ **Post date:** [June 19, 2025, 2:14pm UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/15 "2025-06-19T14:14:39Z")
Then where does the `Iterable[T]` go? (the reason that `Mapping[T, object]` matches) If it’s in the first group, then `Mapping[T, object]` will still be accepted when it shouldn’t be. If it’s not in the first group, then an `Iterable[T]` that should be accepted, will not be accepted. ---
**Author:** ![Dr-Irv](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dr-irv/32/16656_2.png) [@Dr-Irv](https://discuss.python.org/u/Dr-Irv)\ **Post date:** [June 19, 2025, 2:38pm UTC](https://discuss.python.org/t/specifying-invalid-signatures-in-overloads/75405/16 "2025-06-19T14:38:30Z")
OK, here’s yet another proposal. I also realized that putting the errors in the second group of two doesn’t really matter - if you don’t match the first group, it’s an error anyway. 1. When using `@type_error`, the overloads with `@type_error` must be grouped. They can be in the first group, or the second group. If in the second group, a third group of overloads (without `@type_error`) is required. 2. Overload matching is done by group, using the current rules. If there is no match within a group, then the next group is considered for matching, and matching continues as if the prior groups did not exist. 3. If `@type_error` is present in the overloads (whether in the first group or the second group), the type checker tries to match any of the ones labeled as `@type_error` using the same rules as is done today with overloads. If any match, an error is produced. So then I think `Counter. __init__ ()` could be done as: ```python @overload def __init__ (self, Mapping[T, int]): ... @overload @type_error def __init__ (self, Mapping[T, object]): ... @overload def __init__ (self, Iterable[T]): ... ``` First it tries to match `Mapping[T, int]`. If that is not the case, the `Mapping[T, object]` is checked. If a match, then an error is emitted and matching stops. Then it tries matching `Iterable[T]`. The group concept is key. You don’t have to worry about overlaps across groups, as each group is considered independently, in order.