Inferred type of function that calls dunder (`abs/__abs__`)

Typing
1

I was about to open a bug report with mypy but then I realised that both mypy and pyright have the same strange behaviour so I thought I would check here first.

In the following both mypy and pyright infer that abs(value) has a different type from value.__abs__():

from typing import Self

class Real:
    def __abs__(self) -> Self:
        return self

def my_abs[T: Real](value: T, opt: bool) -> T:
    reveal_type(abs(value))      # Real
    reveal_type(value.__abs__()) # T
    return value

This means that return abs(value) is reported as an error by mypy although pyright allows it. I don’t see why the inferred types are different at all though since abs just calls __abs__. The same behaviour is seen with __floor__ and math.floor.

Is this just a bug in both checkers or am I misunderstanding something?

What I was originally trying to do here is to have a protocol for abs but mypy rejects any code that uses it:

from typing import Self, Protocol

class HasAbs(Protocol):
    def __abs__(self) -> Self:
        return self

def my_abs[T: HasAbs](value: T) -> T:
    return abs(value)

# Incompatible return value type
# (got "HasAbs", expected "T")

Note that this is not the same as SupportsAbs[R] because the return type is Self. This is a non-generic protocol for types that are closed under abs.

Ideally it should be possible to write T: SupportsAbs[T] but type checkers don’t allow type constraints to be generic (I run into this limitation frequently).

2

The reason for this behavior is abs is annotated as follows in typeshed

@runtime_checkable
class SupportsAbs[T](Protocol):
    @abstractmethod
    def __abs__(self) -> T: ...

def abs[T](x: SupportsAbs[T], /) -> T: ...

So, when you call abs(value), type-checkers try to interpret your Real argument as a SupportsAbs[T], which gets resolved to SupportsAbs[Real], so the return type is Real.

On the other hand, if you call value.__abs__(), they use the method defined on the Real class, which is T@my_abs called with an argument of T@my_abs, so the return type is T@my_abs.

The type system currently just lacks the ability to precisely model the runtime behavior of many dunder methods and their associated operators.

In principle, I think we would need some new typing primitives like ReturnType[<function>, <args>] and Member[<type-variable>, <literal-string>] to access members on upper bounded type variables and reference the return type of functions provided their arguments. With such primitives, typeshed could annotate abs() like so, and the two different ways to call it should give the same result.

def abs[S: SupportsAbs](x: S, /) -> ReturnType[Member[S, "__abs__"], S]
3

This looks like a mypy false positive to me, and pyright considers this valid (pyright-play).

T@typing.SupportsAbs should resolve to whatever value.__abs__() returns, i.e. the Self of T: HasAbs, so it should resolve to T.

It’s a bit strange that pyright reveals abs(value) as HasAbs, and not as T, but I suppose it’s not incorrect.

4

It is a false positive but I think it is caused by the incorrect inference. It is correct to reject returning HasAbs rather than T and both checkers correctly reject this:

from typing import Self, Protocol

class HasAbs(Protocol):
    def __abs__(self) -> Self:
        return self

def my_abs[T: HasAbs](value: T, other: HasAbs) -> T:
    return other # error

What is incorrect is failing to recognise that abs(value) is of type T. I’m not sure how it is that pyright handles these two cases differently but it reveals the type as HasAbs* with an asterisk rather than HasAbs which presumably means something.

5

The asterisk is shorthand for something I call a “conditional type”. You can read about it here. You can think of it as an intersection. Pyright doesn’t have general support for intersections (since the concept hasn’t yet been spec’ed), but it does support the limited concept of a concrete type intersected with a type variable. In this case, the type could be written as HasAbs & T, but I chose to display it as HasAbs* because most Python developers are not familiar with intersections. Plus, conditional types differ from intersections in some subtle but important ways — notably, in how value-constrained type variables are handled during function calls.

6

If T is bounded by HasAbs should that intersection not simplify? Or does it not because of the subtle differences?

I guess this answers the original question then: pyright can infer the type correctly here so it would be possible for mypy to do the same.

7

In this particular example, it could be simplified. In many cases, it cannot. For example, if you use an isinstance pattern to narrow the type to some subtype of HasAbs, it cannot be simplified. Also, if it’s a value-constrained type variable (as opposed to a regular type variable with an upper bound), it cannot be simplified.

8

I have opened a mypy issue: