@typing.protocol decorator to create a class of type Protocol

Instead of having typing.Protocol, which we inherit to define a Protocol class, we could have a global @protocol decorator that declares a class as a Protocol. And the class that adheres to that Protocol should be decorated with a decorator having the name of the Protocol class it adheres to.

Let’s see an example.

@protocol
class VehicleProtocol:
    def start(self):
        “””Start the vehicle.”””

and then

@VehicleProtocol
class Chevy:
    def start(self):
        print(“Igniting the spark plugs…”)

The reason for having the concrete implementations be decorated like this is to be more obvious to a developer that, as per our example, the class Chevy adheres to the VehicleProtocol Protocol.

Would you like to be able to do this in a future version of Python?

It is already possible to declare explicitly that a class implements a protocol by inheriting from it (class Chevy(VehicleProtocol):). This is optional, because it’s important that protocols you define can match classes in code that you don’t control.

You don’t provide any reasoning for using a decorator instead of a base class. Adding a new mechanism would have a cost (now users have to deal with two equivalent ways to do the same thing), and I don’t see any benefit that’s enough to offset the cost.

12 Likes

I’ve occasionally found myself wanting to verify that a function conforms to a callable protocol at definition site, which is I think about the only instance where a decorator would’ve come in handy.

Do note that Protocol inherits from abc.ABC which itself sets a metaclass, so not inheriting from it will change semantics as you can only set the metaclass during class creation which is before class decorators are called.

I’m trying to recall how we decided that dataclasses should use a class decorator instead of a metaclass.

A problem with metaclasses is that they are a pain to combine with another metaclass (you have to construct a new metaclass that multiply inherits from both metaclasses).

But I think that ABC and Protocol have a requirement that is hard to satisfy with a class decorator: If your class derives from an abstract base class and doesn’t implement all abstract methods, it should still be considered abstract.

At least, that is how it is for ABC. I’m not sure how strong the case is for doing the same for Protocol: If my class inherits from a protocol class, is my class a protocol itself or does this just declare that it implements that protocol? As it is, the intention is ambiguous and this is resolved the same way as abstractness – if not all abstract methods are implemented, it is still a protocol class.

Perhaps the main argument against the proposal here is backwards compatibility, and/or the way it works with ABCs?

3 Likes

I’m not terribly familiar with Python ABCs, but I don’t think that’s how it’s resolved for protocols - not by type checkers, at least. If a class does not directly subclass Protocol, it is not considered to be a protocol and all protocol members must be implemented. The class is not assumed to be a (sub-)protocol if not all members are implemented. (But I wouldn’t know why subclassing was chosen in preference to a decorator.)

Oh, you’re right – PEP 544 has “Subclassing a protocol class would not turn the subclass into a protocol unless it also has typing.Protocol as an explicit base class. Without this base, the class is “downgraded” to a regular ABC that cannot be used with structural subtyping.”

Alas, the PEP doesn’t seem to have considered a class decorator. We’ll have to ask @ilevkivskyi.

The reason dataclasses doesn’t use metaclasses is exactly this: to avoid conflicts with other metaclasses that were already being used. I wanted dataclasses to be able to be easily added to existing classes, even if they were already using metaclasses. I’m not sure protocols would have the same constraint.

3 Likes

I don’t remember all the details, but IIRC base class was chosen because:

  • We wanted protocols to also be “regular” ABCs (e.g. for compatibility with existing ones,like Iterable)
  • It would be similar to how Generic base works.

Btw how would you define a generic protocol using a decorator? You would need both the decorator, and the Generic[T] base. While having Protocol as a base class allows you a simple shorthand Protocol[T].

6 Likes

I see that I have sparked an interesting discussion, so I guess there is some interest in this.

I want to know whether you’re only interested in the @protocol part of the idea or also in the decorating the class that adheres to the protocol part of the idea.

My rationale for decorating the protocol-adhering class is that it becomes more obvious that a class is adhering to a protocol and to which one.

Just like in my example

@VehicleProtocol
class Chevy:
    def start(self):
        print(“Igniting the spark plugs…”)

it is obvious that

  1. the Chevy class adheres to a protocol, since the Protocol-based class was named as ...Protocol, revealing that fact, and
  2. the Chevy class adheres to the VehicleProtocol protocol.
2 Likes

Also, I am against littering the global interpreter namespace. So, I imagine this decorator living in typing. Just like the @dataclass decorator is not global. So, a simple from typing import protocol brings it to life, or import typing and then we can use it by writing @typing.protocol, of course.

Luckily, Python is case-sensitive, so Protocol (the type) and protocol (the decorator) can live side by side in typing. Then, a developer can decide how to create a class of the Protocol type: either by subclassing Protocol or using the @protocol decorator.

It is an essential property of Protocol that a class may implement a protocol without explicitly declaring so. There are situations where a class in a library over whose development you have no control conforms to a protocol in your own code (maybe you even define the protocol to match the library’s behavior). It is already the case that a class that implements a protocol may inherit from the protocol class (using it as an ABC). I see no advantage in requiring this.

1 Like

I see. Well, I was merely suggesting the second part of the idea to perhaps make it easier for developers reading someone else’s code not having to search through the structure of each class whether its methods adhere to the demands of a protocol to then understand the code in terms of aha, so this Foo class follows the BarProtocol protocol.

But having the @protocol decorator is something I’d really like to see in Python 3.12. If not, well, then I’ll just have to satisty my pedantic ego with the syntax class MyProtocol(typing.Protocol).

How about this?

@protocols(SomeProtocolClass, AnotherProtocolClass)
class MyClassAdheringToProtocols:
    pass

I agree it is of value to document protocols a class implements intentionally, yet multiple-inheritance seems enough if the base class has a ...Protocol suffix in its name.

It may be more pythonic, though, to be able to name protocols in the style of Sortable.

It is weird that a class must inherit from a class defined in typing, as in typing.Protocol. That makes the global @protocol decorator a good idea.

7 posts were split to a new topic: Rationale behind dataclasses using a decorator

I feel like this is your main point, right?—you feel that base class inheritance is “unobvious” because inheriting from them is optional?

I think you may be on to a reasonable problem. However, isinstance is the canonical way to check if something “is a” something-else. And inheritance is the canonical way to make something “is a” something else.

One general feature that would solve your problem is the static type assertion proposed here: Add static_assert to verify type constraints in mypy · Issue #5687 · python/mypy · GitHub

Then, you could do

static_assert(issubclass(Chevy, VehicleProtocol))

Later, if you make VehicleProtocol runtime checkable, you can make this into an ordinary assert.

The only problem with this compared to your decorator is that this would be done after the class. So I like your decorator idea if you call it something like static_assert_inherits_from and you could also have a assert_inherits_from decorator for runtime-checkable protocols and just ordinary base classes that you want to check.

What do you think?

Here’s my initial @protocol decorator implementation.

from typing import Protocol


class protocol:
    """A @protocol decorator that defines a Protocol-based class."""

    def __init__(self, class_type):
        self.class_type = class_type

    def __call__(self):
        new_class_name = self.class_type.__name__
        new_class_base = Protocol, self.class_type
        new_class_dict = dict(self.class_type.__dict__)
        new_class_type = type(new_class_name, new_class_base, new_class_dict)
        return new_class_type()


# Usage example

@protocol
class MathProtocol:
    def sum(self, x: int, y: int, /) -> int:
        ...

    def subtract(self, x: int, y: int, /) -> int:
        ...


class MathOperations:
    def sum(self, x: int, y: int, /) -> int:
        return x + y

    def subtract(self, x: int, y: int, /) -> int:
        return x - y