Compatability of descriptor objects in protocols

Typing
1

Right now, type checkers (pyright and mypy here) can correctly determine the resulting type is compatible, but still reject it for compatability


import asyncio
from collections.abc import Callable
import typing as t


class prop[T, R]:
    def __init__(self, fget: Callable[[T], R]) -> None:
        self.fget = fget

    def __call__(self) -> R: ...

    def __set__(self, instance: T | None, owner: type[T]) -> t.Never:
        raise AttributeError

    @t.overload
    def __get__(self, obj: T, objtype: type[T] | None) -> R: ...

    @t.overload
    def __get__(self, obj: None, objtype: type[T] | None) -> Callable[[t.Any], R]: ...

    def __get__(self, obj: T | None, objtype: t.Any):
        if obj is None:
            return self.fget
        return self.fget(obj)


class X:
    def __init__(self):
        self._future: asyncio.Future[None] = asyncio.Future()
        self._future.set_result(None)
        t.reveal_type(self.__await__)
        t.reveal_type(self._future.__await__)
        t.reveal_type(type(self).__await__)
        t.reveal_type(type(self._future).__await__)


    @prop
    def __await__(self):
        return self._future.__await__

async def example():
    x = X()
    await x  # this should be fine, and indeed works at runtime.
    print("It works!", flush=True)
    

if __name__ == "__main__":
    asyncio.run(example())

According to Eric in bringing this as an issue with pyright:

Pyright uses the Awaitable protocol to determine whether a type is awaitable. The type in question here is not compatible with this protocol, so I think it’s correct for pyright to emit an error here.

This appears to be a hole in the definition of compatibility then, as here’s the entire definition of Awaitable

@runtime_checkable
class Awaitable(Protocol[_T_co]):
    @abstractmethod
    def __await__(self) -> Generator[Any, Any, _T_co]: ...

I would say that any descriptor that results in equivalence to a method behaviorally should be compatible with a method in a protocol, as protocols are specifically for structural typing. Unlike @property, the descriptor here works on both the type and instances of the type behaviorally equivalently to methods.

4 Likes
2

There seems to just be a type checker bug happening here beyond just descriptor use, the following example when trying to track exactly why this fails works in pyright, a descriptor returning Any is compatible, so I’m not sure why a descriptor returning the exact type expected isn’t, but that’s what I was told by @erictraut

Code sample in pyright playground

import asyncio
from collections.abc import Callable
import typing as t


class prop[T, R]:
    def __init__(self, fget: Callable[[T], R]) -> None:
        self.fget = fget

    def __call__(self) -> R: ...

    def __set__(self, instance: T | None, owner: type[T]) -> t.Never:
        raise AttributeError

    @t.overload
    def __get__(self, obj: T, objtype: type[T] | None) -> R: ...

    @t.overload
    def __get__(self, obj: None, objtype: type[T] | None) -> Callable[[t.Any], R]: ...

    def __get__(self, obj: T | None, objtype: t.Any):
        if obj is None:
            return self.fget
        return self.fget(obj)


class X:
    def __init__(self):
        self._future: asyncio.Future[None] = asyncio.Future()
        self._future.set_result(None)
        t.reveal_type(self.__await__)
        t.reveal_type(self._future.__await__)
        t.reveal_type(type(self).__await__)
        t.reveal_type(type(self._future).__await__)


    @prop
    def __await__(self):
        return self._future.__await__

class Y:

    def __init__(self):
        self._future: asyncio.Future[None] = asyncio.Future()
        self._future.set_result(None)
        t.reveal_type(self.__await__)
        t.reveal_type(self._future.__await__)
        t.reveal_type(type(self).__await__)
        t.reveal_type(type(self._future).__await__)


    @prop
    def __await__(self) -> t.Any:
        return self._future.__await__

async def example():
    x = X()
    await x  # this should be fine, and indeed works at runtime.
    await Y()  # this works, both at runtime and type check, so it isn't the descriptor use
    print("It works!", flush=True)
    

if __name__ == "__main__":
    asyncio.run(example())
3

Something strange going on there that looks to be a bug, uncomment the below (incorrectly typed) __call__ definition in prop and it type checks:

import asyncio
from collections.abc import Callable
import typing as t


class prop[T, R]:
    def __init__(self, fget: Callable[[T], Callable[[], R]]) -> None:
        self.fget = fget

    # def __call__(self) -> R: ...

    def __set__(self, instance: T | None, owner: type[T]) -> t.Never:
        raise AttributeError

    @t.overload
    def __get__(self, obj: T, objtype: type[T] | None) -> Callable[[], R]: ...

    @t.overload
    def __get__(self, obj: None, objtype: type[T] | None) -> Callable[[t.Any], Callable[[], R]]: ...

    def __get__(self, obj: T | None, objtype: t.Any):
        if obj is None:
            return self.fget
        return self.fget(obj)


class X:
    def __init__(self):
        self._future: asyncio.Future[None] = asyncio.Future()
        self._future.set_result(None)

    @prop
    def __await__(self) -> Callable[[], t.Generator[t.Any, None, None]]:
        return self._future.__await__


async def example():
    x = X()
    y: t.Awaitable[None] = x
    await x  # this should be fine, and indeed works at runtime.
    print("It works!", flush=True)
    

if __name__ == "__main__":
    asyncio.run(example())

But prop.__call__ is never invoked.

4

That still fails Pyright Playground

__call__ definition there has no effect

5

If you uncomment the __call__ it passes… pyright playground

6

Sorry, I misread your first post and missed that uncommenting was what fixed that example, however I see no reason that should have any effect at all, especially when the original examples both had that (useless) __call__ definition and fail.

__call__ on this is not ever, and will not ever be invoked, I could put a os._exit(1) there without breaking anything.

7

Yes. That’s why I said:

My point is it might be a bug in pyright. The __call__ definition shouldn’t make a difference but does.

Edit: also the return type of my __call__ is slightly different to the original definition

1 Like