# Protocol classes should not match \`Callable\[..., Proto\]\`

**URL:** <https://discuss.python.org/t/protocol-classes-should-not-match-callable-proto/75475>\
**Category:** Typing\
**Created:** [December 27, 2024, 9:15pm UTC](https://discuss.python.org/t/protocol-classes-should-not-match-callable-proto/75475 "2024-12-27T21:15:50Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![hauntsaninja](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/hauntsaninja/32/12451_2.png) [@hauntsaninja](https://discuss.python.org/u/hauntsaninja)\
**Post date:** [December 27, 2024, 9:15pm UTC](https://discuss.python.org/t/protocol-classes-should-not-match-callable-proto/75475/1 "2024-12-27T21:15:50Z")

</div>

The following code type checks under mypy and pyright, but raises an error at runtime.

```python
from typing import Callable, Protocol

class Proto(Protocol):
    def foo(self): ...

def foo(t: Callable[..., Proto]):
    t()

foo(Proto) # TypeError: Protocols cannot be instantiated

```

Should we specify that type checkers should error here?

Similar thing for abstract classes (although the spec doesn’t yet really talk about abstract classes):

```python
import abc
from typing import Callable, Protocol

class Abstract(abc.ABC):
    @abc.abstractmethod
    def foo(self): ...

def foo(t: Callable[..., Abstract]):
    t()

foo(Abstract) # TypeError: Can't instantiate abstract class Abstract without an implementation for abstract method 'foo'

```

My secret goal is to make headway on `T. __init__ ` unsoundness, by encouraging use of `Callable[..., T]` instead. A subgoal is preserving soundness on attempts to instantiate protocols or abstract classes. Some relevant [discussion here](https://discuss.python.org/t/compatibility-of-protocol-class-object-with-type-t-and-type-any/48442/2)

---

<div class="post-metadata">

**Author:** ![erictraut](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erictraut/32/15190_2.png) [@erictraut](https://discuss.python.org/u/erictraut)\
**Post date:** [December 27, 2024, 9:29pm UTC](https://discuss.python.org/t/protocol-classes-should-not-match-callable-proto/75475/2 "2024-12-27T21:29:48Z")

</div>

I don’t feel that strongly either way on this topic. I don’t think this is a source of bugs today, so I think that adding this rule would have little or no impact.

If we were to add this rule, I presume we’d need to carve out some special cases for protocol classes that define a ` __new__ ` and/or ` __init__ ` method?

---

<div class="post-metadata">

**Author:** ![mikeshardmind](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/mikeshardmind/32/14381_2.png) [@mikeshardmind](https://discuss.python.org/u/mikeshardmind)\
**Post date:** [December 27, 2024, 10:00pm UTC](https://discuss.python.org/t/protocol-classes-should-not-match-callable-proto/75475/3 "2024-12-27T22:00:58Z")

</div>

The signature for `foo` looks reasonable to me. I don’t agree with the error. As has been discussed in many places, the type system can’t always know if it has that type or a subtype. There are valid subtypes of Abstract that in some cases the type system would only have a bound of `type[Abstract]`

> [@hauntsaninja](#):
>
> My secret goal is to make headway on `T. __init__ ` unsoundness, by encouraging use of `Callable[..., T]` instead.

I don’t think this is the right approach. We’ve seen more and more impacts of LSP exceptions (most recently, accurate typing and static error detection of `copy.replace`), I think we should be looking at ways to narrow these exceptions to the bare minimum that are caused by `object` and `type` and get user code that creates unnecessary issues fixed over time.

---

<div class="post-metadata">

**Author:** ![hauntsaninja](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/hauntsaninja/32/12451_2.png) [@hauntsaninja](https://discuss.python.org/u/hauntsaninja)\
**Post date:** [December 27, 2024, 11:13pm UTC](https://discuss.python.org/t/protocol-classes-should-not-match-callable-proto/75475/4 "2024-12-27T23:13:48Z")

</div>

Quick and dirty mypy draft PR here: [Disallow assignment of protocol or abstract classes to Callable by hauntsaninja · Pull Request #18347 · python/mypy · GitHub](https://github.com/python/mypy/pull/18347) primer looks fine (if you ignore the prefect bug). Both hits are true positives in my opinion (though diagnostics need to be improved).

> [@erictraut](#):
>
> If we were to add this rule, I presume we’d need to carve out some special cases for protocol classes that define a ` __new__ ` and/or ` __init__ ` method?

Hm, good question. IIRC `Protocol. __new__ / __init__ ` failed at runtime prior to 3.11. Given that [it seems it was added to make mixins work](https://github.com/python/cpython/issues/88970#issuecomment-1093924058), I’d say we don’t need a carve out. This is similar to how pyright and mypy do not have a carve out for [direct instantiation](https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmUkQWEMoDCAhgDYlEBGJApgDRQAK4MYAxmCQFCevkDOfRszAAKJmBbsSASgBcnKIqgATasCgB9DaiQwtIvtRLB6wMGFmYUMOVAB0DhUs7iWIgKzSgA)

> [@mikeshardmind](#):
>
> The signature for `foo` looks reasonable to me. […] As has been discussed in many places, the type system can’t always know if it has that type or a subtype

To be clear, the request is for a call site error, not an error on definition of `foo`. The type checker can actually sort of know at the call site… The requested behaviour would be similar to the [behaviour here](https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmUkQWEMoDCAhgDYlEBGJApgDRQAK4MYAxmCQFCevkDOfRszAAKJmBbsSASgBcnKIqgATasCjAwovtRLA5UAHTHuq9RSIgRMWbETUA2uJYBdA8cOm1UCEVRjhAH0bOwRHZzA3eSUoCysI6QUlOICJMGDpIA), whose specification dates back to PEP 544. Note this specified behaviour means we actually do mostly know a `x is not Proto` where `x: type[Proto]`, so we don’t have transitivity issues.

(The PEP 544 behaviour can get annoying when applied to abstract classes like mypy does, because there are several possible uses of `type` beyond instantiation. But as I mentioned in the thread I linked in OP, I think more use of `Callable` can help here)

> [@mikeshardmind](#):
>
> I don’t think this is the right approach. We’ve seen more and more impacts of LSP exceptions

Ah interesting. I thought of my goal as complementary.

` __init__ ` LSP violations are endemic. If we can get to a point where `--strict-type-instantiation` is viable (that disallows instantiation of unknown non-final `type[C]` and assignment of unknown non-final `type[C]` to `Callable[P, C]`), maybe users can get soundness even if using the many libraries that have ` __init__ ` LSP violations. But this may not be viable (or may need intersections to be viable)!

(to avoid confusion for readers, my secret long term goal is technically a separate topic from OP)

---

<div class="post-metadata">

**Author:** ![mikeshardmind](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/mikeshardmind/32/14381_2.png) [@mikeshardmind](https://discuss.python.org/u/mikeshardmind)\
**Post date:** [December 28, 2024, 12:31am UTC](https://discuss.python.org/t/protocol-classes-should-not-match-callable-proto/75475/5 "2024-12-28T00:31:20Z")

</div>

> [@hauntsaninja](#):
>
> The requested behaviour would be similar to the [behaviour here](https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmUkQWEMoDCAhgDYlEBGJApgDRQAK4MYAxmCQFCevkDOfRszAAKJmBbsSASgBcnKIqgATasCjAwovtRLA5UAHTHuq9RSIgRMWbETUA2uJYBdA8cOm1UCEVRjhAH0bOwRHZzA3eSUoCysI6QUlOICJMGDpIA), whose specification dates back to PEP 544. Note this specified behaviour means we actually do mostly know a `x is not Proto` where `x: type[Proto]`, so we don’t have transitivity issues.

I’m a bit concerned that this is going to catch other things that should be valid, but I guess we can deal with that if it comes up that type checkers are emitting false positives as a result of this.

> [@hauntsaninja](#):
>
> > [@mikeshardmind](#):
> >
> > I don’t think this is the right approach. We’ve seen more and more impacts of LSP exceptions
> 
> Ah interesting. I thought of my goal as complementary.
> 
> ` __init__ ` LSP violations are endemic.

Yeah, and that’s a problem in it of itself. It creates more problems than just telling people to use `**_kwargs: object` if they want to allow subclasses to add kwargs (and other such things). If we were able to limit all of the exceptions to those that are part of the data model or from `object`/`type`, and model the exceptions so that they only apply to overriding a default implementation, we could get actual soundness of `type` I’ve been working on language towards this route with [` __hash__ `, ` __eq__ `, and LSP](https://discuss.python.org/t/hash-eq-and-lsp/68138)

> [@hauntsaninja](#):
>
> If we can get to a point where `--strict-type-instantiation` is viable (that disallows instantiation of unknown non-final `type[C]` and assignment of unknown non-final `type[C]` to `Callable[P, C]`)

I don’t see this as a goal worth pursuing, I think this is a configuration option that does more harm than good, much like those that flag Any/Unknown, they restrict valid code rather than gradually improve what is actually expressible and work towards making what exists possible to use soundly.

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [December 28, 2024, 8:55am UTC](https://discuss.python.org/t/protocol-classes-should-not-match-callable-proto/75475/6 "2024-12-28T08:55:36Z")

</div>

> [@hauntsaninja](#):
>
> My secret goal is to make headway on `T. __init__ ` unsoundness, by encouraging use of `Callable[..., T]` instead.

IMO, this would be an awesome step in the right direction.

For additional support, this problem is extensively discussed [here](https://github.com/python/mypy/issues/4717) in which many people don’t like how MyPy complains about abstract types not being passable (since MyPy assumes that any type can be instantiated). Separating callability from the concept of a type would eliminate this entire problem.

---

<div class="post-metadata">

**Author:** ![jorenham](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jorenham/32/27806_2.png) [@jorenham](https://discuss.python.org/u/jorenham)\
**Post date:** [January 5, 2025, 11:41am UTC](https://discuss.python.org/t/protocol-classes-should-not-match-callable-proto/75475/7 "2025-01-05T11:41:36Z")

</div>

A `Protocol` can’t be instantiated, but the things that are assigned to it at runtime can:

```py
import operator
from collections.abc import Iterable
from typing import Protocol

class CanPos[T, RT](Protocol):
    def __init__ (self, x: T, /) -> None: ...
    def __pos__ (self, /) -> RT: ...

def pos_map[T, RT](
    mapper: type[CanPos[T, RT]],
    things: Iterable[T], 
    /,
) -> Iterable[RT]:
    things_that_can_pos = map(mapper, things)
    return map(operator.pos, things_that_can_pos)

```

Here it’s valid to pass a `type[int]` to `mapper`:

```pycon
>>> list(pos_map(int, ["1", "2"]))
[1, 2]

```