Compatibility of protocol class object with `type[T]` and `type[Any]`

Typing
1

The typing spec is clear that "variables and parameters annotated with type[Proto] accept only concrete (non-protocol) subtypes of Proto. This means the following code should result in a type error.

class Proto(Protocol):
    x: int

def func1(v: type[Proto]):
    pass

# Type error: Only concrete class can be given where "type[Proto]" is expected
func1(Proto)

And indeed, mypy and pyright both generate a type error here.

However, the typing spec is not clear what should happen when a protocol class object is assigned to a variable or parameter of type type[T] (where T is a type variable) or type[Any]. Mypy and pyright produce inconsistent results for type[T]. Mypy produces inconsistent results between type[T] and type[Any].

T = TypeVar("T")

class Proto(Protocol):
    x: int

def func1(v: type[T]):
    pass

# Mypy: Only concrete class can be given where "type[Proto]" is expected
# Pyright: No error
func1(Proto)

def func2(v: type[Any]):
    pass

# Mypy: No error
# Pyright: No error
func2(Proto)

Which behavior is correct and/or desirable?

Regardless of the answer we converge upon, this seems like something that should be made clear in the spec, so we should think about how best to word this.

1 Like
2

mypy’s behaviour is probably incorrect, but I’m less sure about which direction — whether it should match pyright and permit everything, or error in all cases. Note both pyright and mypy also permit type[object], so don’t think there’s anything particularly special happening with type[Any].

(more broadly, but also more off-topic-ly)

I’m not sure how successful “only concrete class can be given” check is at all. PEP 544 says it’s meant to disallow instantiation of abstract types. I think I’d prefer if direct instantiation of a cls: type[Proto] was what was disallowed and you were forced to pass in Callable[..., Proto] if you wanted to do cls(...). This would also help with __init__ unsoundness.

This came up recently in Reason given for disallowing non-concrete subtype assignment is unsound · Issue #1647 · python/typing · GitHub. Also see Use case for typing.Type with abstract types · Issue #4717 · python/mypy · GitHub where there’s pretty strong evidence users don’t like this check as currently implemented in mypy. (Note that mypy’s behaviour is different than pyright’s, e.g. it also subjects ABCs to this constraint, again as a means of disallowing instantiation of an abstract type)

Note this check does get you some other things not mentioned in PEP 544, like avoiding isinstance on non-runtime-checkable Protocols

9 Likes
3

My own experience is I have passed abstract classes to type[T] where function was able to handle that without an issue. Passing a type does not imply the use case is to instantiate it directly. I don’t think I have examples of passing protocol to a type[T], although that’s because I tend to see more libraries with abstract classes then protocols. There’s a lot of common abstract classses in standard library but few (not sure of any) protocols there. I do not see a strong argument for why protocols vs abstract classes should differ in rules here.

So my own preference would lean towards,

variables and parameters annotated with type[Proto] accept only concrete (non-protocol) subtypes of Proto

removing this entirely.

If not then how should code that does runtime type manipulations (singledispatch/config systems) handle this? We could have type[T] and ConcreteType[T] (or AbstractType[T]), but unsure extra special form is worth distinguishing the two. If answer is no option, then it mostly becomes noisy type ignore.

edit: context:global ": type[T… - Sourcegraph we can review usage patterns of type[T]. Too many to review but spot checking 4,

First one uses it for a cast and abstract/protocol would work fine. Second one modifies class attributes for documentation. That would work fine too with protocol. Third one is interesting. It does not work with protocols, but it also does not work with most concrete types. It is only for TypedDict types so really it’s closer to type[T: TypedDict] even though that’s not well defined in type system either. Fourth one looks like Self maybe written before Self existed?

3 Likes
4

In Haskell terms, I think of Protocol and type both being two different kinds. In this analogy, mypy gets it right with func1; if you think of func1 as a type-level function, its argument must have kind *. Proto, on the other hand, has kind Protocol.

Under this interpretation, type is a kind, and type[Any] is still equivalent to type but not to Any itself. If you want a function that that accepts a type or a protocol, you would need a union kind:

def func2a(v: type):  # only accepts "real" types

def func2b(v: Protocol): # only accepts subclasses of Protocol

def func3(v: type | Protocol):   # accepts real types or protocols

I realize this isn’t doable without introducing the kind of type hierarchy found in Haskell, but it might be useful for providing a basis for how to treat Protocol.

6

Callable is a way I have tried to go with this pattern, but mypy and pyright accept both ABC types and Protocol types as Callable.

import abc
from typing import Callable, Protocol


class B(abc.ABC):
    @abc.abstractmethod
    def f(self) -> None:
        """ f """


def g(x: Callable[[], B]) -> None:
    b = x()
    b.f()


g(B)


class C(Protocol):
    def f(self) -> None:
        """ f """


def h(x: Callable[[], C]) -> None:
    c = x()
    c.f()


h(C)

no type errors reported where I would expect them to be reported (using B and C as Callable)

2 Likes
7

The typing spec goes on to state following later in the same section:

Assigning an ABC or a protocol class to a variable is allowed if it is not explicitly typed, and such assignment creates a type alias.

English is not my first language, and I know it doesn’t say “only if”, but, to me, the spec is clear that assigning non-concrete type value to a variable is only allowed to create a type alias.
So, when a protocol class object is assigned to a variable or parameter of type type[T] or type[Any], type checkers should not allow such assignment and raise error.

Having said that, my experience of the concreteness restriction is really bad as hauntsaninja summarised, and wholly support removing (or type checkers not adding) any restriction like mdrissi suggests.

1 Like
8

I wound up at this thread from a GitHub thread which I wound up at from another GitHub thread which ultimately gos back to this note in the docs of the svcs package, which came up when searching for an error I had indeed encountered with using svcs.

Since there seems to be a repeated request for concrete (pun intended) use cases, I have a protocol – call it MyProto – and multiple classes which implement it. I use svcs as a service locator, and at runtime I want to select a particular implementation of MyProto (which will vary depending on information only available at runtime) and register a factory for it in my svcs.Registry. The problem, of course, is that a later get(MyProto) is flagged as a type error by mypy and pyright even though the whole thing can be proven sound statically.

I can use the suggested workaround in the svcs docs of calling get_abstract() instead of get(), but then of course I have to deal with the fact that now type checkers think I’m getting an Any instead of a MyProto. Or I could turn off this check. But I’m also writing an internal-use library which will be deployed into lots of other things that will use type checking, and I’d have to tell all of those people to apply a workaround (either turning off the check or always using get_abstract() to fetch this specific item from the service locator).

All of which is a bit annoying. I see there’s no activity in this thread for some time, and none in the GitHub threads either; is there anyplace where discussion on this is happening that I’ve missed, or would this be the right place to try to revive the request for the typing specs to change to accommodate this sort of use case?