Type hints for bool vs int overload

Typing
21

Here’s a general purpose solution:

from __future__ import annotations

from typing import Protocol, overload, override

class A: pass
class B: pass

class Just[T](Protocol):
    @property
    @override
    def __class__(self, /) -> type[T]: ...  # pyright: ignore[reportIncompatibleMethodOverride]
    @__class__.setter
    @override
    def __class__(self, t: type[T], /) -> None: ...

@overload
def func(arg: bool) -> A: ...
@overload
def func(arg: Just[int]) -> B: ...
def func(arg: bool | Just[int]) -> A | B:
    if isinstance(arg, bool):
        return A()
    else:
        return B()

playground links:

This Just type, and several (runtime-checkable) aliases like JustInt and JustObject (useful for sentinels), are implemented (and tested) in the upcoming optype v0.9 release (docs), which I’ll release today or tomorrow.
I’ve been using this trick quite a lot in scipy-stubs, and I’ve had no problems with it so far.

2 Likes
22

Thanks!

I am struggling to understand what the code in the Just class means though…

I notice that there is a pyright: ignore so I am wondering if this offers any particular advantage over my current solution which is just type: ignore and seems to work as intended with mypy and pyright:

from typing import overload

class Boolean: pass
class Integer: pass

@overload
def f(a: bool) -> Boolean: ... # type: ignore
@overload
def f(a: int) -> Integer: ...

def f(a: bool | int) -> Boolean | Integer:
    if isinstance(a, bool):
        return Boolean()
    return Integer()

# The inference here is the most important thing:
reveal_type(f(True)) # Boolean
reveal_type(f(1)) # Integer

I took a look at optype and it looks very nice. I would need to take more time to understand it though. I haven’t quite got my head around a typing-only library yet. Would you use that as a runtime dependency or just a dev dependency?

23

Minor caveat: This would only work correctly if func() is invoked directly.

reveal_type(func(True))   # A

# Some code you don't control
def f(a: int):
    reveal_type(func(a))  # B

f(True)                   # !!!
1 Like
24

The overlapping overloads error might seem redundant at first glance, but the because of the way that unions work in python, we can get incorrect outcomes in situations like this:

def somewhere(x: bool | int):
    reveal_type(f(x))  # Integer

And what we actually want, is for f(bool | int) to return Boolean | Integer. Because if the input is either bool or int , then the output should also be either Boolean or Integer.

That depends on where you’re using it: If you only import optype as op in your dev code, then you can install it as a dev dependency. If you import it somewhere in the public code, then it should probably be a required dependency.

25

You use both func and f in your example, so I’m not sure what you’re referring to. It would also help if you could explain what you mean with the # !!! in the code.

26

They mean that the Just type doesn’t solve the underlying problem in a meaningful way, it just allows people to sometimes more accurately type their functions if they are aware of the problems. Continuation of your example:

def foo(arg: int) -> B:
    return func(arg)


reveal_type(foo(5))     # revealed type is B, runtime type is also B
reveal_type(foo(False)) # revealed type is B, runtime type is A

Which means that the issue of LSP violation still happens.

Ofcourse, the author of foo can also use Just[int], but that means that every downstream user needs to be aware of this.

27

I’d say that having a way to distinguish between int and bool, which was previously thought impossible, is pretty meaningful :man_shrugging:

Not every downstream user needs to know this; it’s mostly useful for maintainers of stub packages, and other typed libraries. And to me, spreading awareness sounds a lot easier than having to go through the PEP process. It’s also something that we can all help with help with.

28

And yet, it was already being done in OP. The point is your alternative solution is still masking the conceptual problem with the code, just in a slightly more ergonomic way (while probably relying on poorly defined type checking behavior, although I am not sure about).

1 Like
29

Ah apparently I failed properly explain what the Just type actually does. And this example pretty much sums it up:

from typing import Protocol, override

class Just[T](Protocol):
    @property
    @override
    def __class__(self, /) -> type[T]: ...  # pyright: ignore[reportIncompatibleMethodOverride]
    @__class__.setter
    @override
    def __class__(self, t: type[T], /) -> None: ...


only_int_tn: Just[int] = 1  # accepted
only_int_tp: Just[int] = True  # rejected

This shows that the (bool) -> A and (Just[int]) -> B overloads don’t overlap. So it doesn’t mask the problem, it actually solves it.

1 Like
30

I fully understand what the Just[T] type does.

You are failing to understand what the counterpoint is and where the LSP violations come from.

31 
def foo(arg: bool | int) -> A | B:
    out = func(arg)
    reveal_type(out)  # Revealed type is "Union[__main__.A, __main__.B]"
    return out

So because bool | int is equivalent to int, the outcome should be the same with arg: int. So I believe we’re dealing with a type-checker bug here.

32

That might be something you can argue for. However, what type checkers do instead is complain about the @overload definitions conflicting. You are tricking the type checkers into not complaining via, let’s say, poorly defined mechanics that happen to work somewhat, but it have surprising side-effects (like here). This also works in only some type checkers. Covering mypy and pyright is pretty good, but e.g. pyre doesn’t work.

1 Like
33

Just may also stop working when the behavior of things inherited from the data model, and the behavior of type at runtime are cleared up.

This definition relying on __class__ only “works” (to the extent which is does) because type checkers are encoding the information about the data model slightly incorrectly right now, and pyright is recognizing it enough that you’re having to ignore it.

34

I was under the impression that it works because T@Just is invariant.

35

T@Just shouldn’t be invariant here. the type of __class__ shouldn’t be overridable with use of type[T], and should just be type[Self], hence your need for the incompatible method override.

This dovetails with some other ongoing issues with the type-instance relationship not being encoded correctly with regard to runtime access to the relationship in python’s type system.

36

I’m entirely sympathetic to the need this is intended to fill, but I don’t think this is the right way here, and this needs an actual proposal to the type system for a Supported means of spelling “exactly this type, not some compatible type” (or type negation, which would allow int - bool, for example, here)

1 Like
37

Your use of “tricking” suggests that you thing that the overloads are actually overlapping. But the fact that a: Just[int] = True is rejected, which is a consequence of T@Just being invariant, proves that the overloads actually don’t overlap.

And what poorly defined mechanics are you talking about exactly?

That’s too bad. But in my limited experience, there are more things that don’t work in pyre.

Either way, I don’t see any reason why Just shouldn’t work as I intended. As far as I can tell, it doesn’t use any hacks and doesn’t rely on undefined behavior. It simply is uses the invariantly-bound __class__ type to reject everything whose __class__ isn’t of that same type.

38

This is the trick here. And your type ignore on this shows that at least one type checker knows enough to flag it as incorrect. __class__ is special, and isn’t invariant with respect to the given T in it’s behavior.

The data model even documents this as being special, and spells out that this should be type[Self], it’s the same relationship as using the single argument form of type at runtime.

1 Like
39

It actually must be invariant, otherwise it’d be type-unsafe, and would allow for thing like int.__class__ = bool.

The incompatible override error stems from the fact that type[T] isn’t (always) assignable to type[Self]. But that rests on the (IMO false) assumption that Self@Just is actually bound to the instance that matches Just, and that this instance is not always compatible with T. But in practice, T and Self are the very same.

40

Yes, and that’s totally right. But I see no reason why that makes it incompatible with Just.

Because in a way, you could see T@Just as an externalized Self.