Protocol as a soft keyword, making structural typing a first-class citizen

Ideas
1

Hi everyone,

Building on Guido’s comments at the recent PyCon US Typing Summit regarding the evolution of type hints, I’d like to propose elevating structural typing to a first-class language feature by introducing protocol as a soft keyword. When PEP 484 was introduced, the guiding constraint was “no new syntax.” As a result, we had to bolt typing features onto existing runtime mechanics. For Protocols (PEP 544), that meant using class inheritance:

from typing import Protocol

class Drawable(Protocol):
    def draw(self) -> None: ...

The motivation for changing this is not to reduce boilerplate. A single inherited class is not the end of the world. Instead, it is confusing, since classes and “interfaces” are fundamentally different concepts that many modern type systems distinguish clearly. A Protocol is fundamentally a structural contract, not an object-oriented base class. Using subclassing syntax to define an interface conceptually conflates static structural typing with runtime inheritance. By introducing a protocol soft keyword, we clearly separate the two concepts:

protocol Drawable:
    def draw(self) -> None: ...

This follows the precedent set by the type soft keyword in Python 3.12 (PEP 695), which showed the value of elevating core typing constructs from module imports to native syntax. Before I draft a formal PEP, I would love to hear the community’s thoughts on the appetite for this, and any initial feedback from those familiar with the PEG parser’s soft keyword mechanics.

CC: @Jelle for hallway track conversations we had at PyCon

13 Likes
2

My immediate reaction is that this could be useful, and potentially opens the door to further improvements to structural typing. A few questions:

  • The Protocol class allows other classes to inherit from a Protocol, with the intent of explicitly declaring a given class adheres to the given protocol. Do I understand correctly that this runtime behavior would not be supported by your soft keyword?
  • What are the actual semantics of the protocol keyword? Is it creating a class? If so, can you explain in more detail what this addition would achieve versus the status quo? If not, how might it interact with runtime introspection?
3

The Protocol class allows other classes to inherit from a Protocol, with the intent of explicitly declaring a given class adheres to the given protocol. Do I understand correctly that this runtime behavior would not be supported by your soft keyword?

Standard classes will absolutely still be able to inherit from a protocol to explicitly declare adherence, exactly as they do today. The goal isn’t to restrict how protocols interact with classes, but to make protocol definition and composition cleaner. I should have included an example of protocol composition in my original post, but it would look like this:

protocol Sized:
   def size(self) -> int

protocol Drawable:
    def draw(self) -> None: ...

protocol SizedDrawable(Sized, Drawable): ...

# Explicit adherence remains unchanged
class Box(SizedDrawable):
    ...

I think this is easier to parse visually and explain to people new to the Python type system than the concept of a magical class with special rules and properties.

What are the actual semantics of the protocol keyword? Is it creating a class? If so, can you explain in more detail what this addition would achieve versus the status quo? If not, how might it interact with runtime introspection?

I mostly see this as purely syntactic sugar to make this proposal as widely agreeable as possible. During parsing, protocol Drawable: would effectively lower directly into the equivalent class Drawable(typing.Protocol): AST. Efforts to evolve Python’s approach to structural typing should probably be handled in a different PEP/proposal.

2 Likes
4

Any kind of work that improves the current situation on protocols has my support. I’m extensively using protocols and right now they could receive much more love. Also your syntax seems great to distinguish them from classes. There are a few things to consider.

What happens to @runtime_checkable?
What happens to the discernment between data and non-data protocols?
Does this proposal remove the coupling with _ProtocolMeta, the ABC class typing.Protocol currently requires to inherit from to make structural type checking work?
Will this mean promoting protocols to a C type? Will there be some form of protocol declaration from C-extensions? How will that work?
Do you plan to change how protocols are actually checked to make them more accurate / easy to deal with? Or is this a completely separate point?

1 Like
5

Is this just syntactic sugar? If so, I don’t really see any value in being able to write
protocol Foo: over class Foo(Protocol):. If this somehow enabled us to combine Protocols without subclassing though, then I would be in favor. I really want something like

def foo(fp: Reader + Writer) -> None: ...
def foo[T: Readable + HasName](fp: T)  -> T: ...

instead of the current subclassing approach.

8 Likes
6

While I like elevating protocols, I prefer elevating other constructs along with it.

The standard library contains various constructs that don’t exactly behave like classes: enums, dataclasses, namedtuples, etc.

Third-party packages might in their turn have important constructs that they’d like to treat as first-class citizens, for example when interacting with an external object system.

These are two of the reasons behind PEP 834 (currently still a PR), which introduces a more generic approach with the make soft keyword.

For example, that would turn:

from typing import Protocol

class Drawable(Protocol):
    def draw(self) -> None: ...

Into:

from typing import protocol

make protocol Drawable:
    def draw(self) -> None: ...

I don’t want to hijack this thread to discuss PEP 834 instead, but just wanted to mention it :slight_smile:

3 Likes
7

You mention PEP 384 but I assume it’s a typo and you mean PEP 834?

8

You’re right, thanks for noticing! Corrected the reply.

1 Like
9

Thanks for the feedback and questions!

@jacopoabramo
To keep this proposal focused and maximize its chances of acceptance, I currently intend for it to act strictly as syntactic sugar during parsing (lowering to a standard class inheriting from typing.Protocol). Because of that, the answers to your runtime questions are fairly straightforward:

  • @runtime_checkable: Decorators would work exactly as they do on classes today. For ex:
@runtime_checkable
protocol Drawable:
    def draw(self): ...
  • Data vs non-data protocols: This behavior remains unchanged and adheres strictly to the existing PEP 544 rules.
  • _ProtocolMeta, C-types, and checking accuracy: Because the AST lowers to a class under the hood, the current runtime machinery remains completely unchanged. Upgrading Protocols to a native C-type or rewriting how they are evaluated at runtime would be amazing, but that is a massive undertaking that warrants its own dedicated PEP. Since I am proposing a new keyword here, I think it is probably best to limit the scope as much as possible. If this PEP were accepted, I think it would be far easier to propose some of these changes you are talking about.

@Monarch

If this somehow enabled us to combine Protocols without subclassing though, then I would be in favor

I think this is where intersection types come in. I know this has been a forever ongoing topic, but I hope we are getting closer to formalizing it. This would allow you to do

def foo(fp: Reader & Writer) -> None: ...
def foo[T: Readable & HasName](fp: T)  -> T: ...

and even

type ReadWrite = Reader & Writer

Regarding the value of syntactic sugar, there is strong precedent for syntax changes that “just” sugar over existing behaviors when they clarify intent. For example, the currently active PEP 823 and PEP 824 (None-coalescing operators) are explicitly designed as pure syntactic sugar for common is not None patterns because the readability and ergonomic wins are high.

Because we currently use the class keyword, protocols are shackled to class mechanics (MRO, standard inheritance) in developers’ minds and in our tooling. Carving out a dedicated protocol keyword establishes the strict grammatical boundary required for exactly the kind more expansive changes you’re talking about. It gives us a dedicated syntax space to build upon for future without having to rewrite the entire runtime engine in one massive PEP today.

@monster
Thank you for pointing out PEP 834. I remember talking to @Jelle specifically about this, but I didn’t realize it went anywhere. I am not opposed to pushing forward with just PEP 834, but I think it would further lock in protocols’ relationship to Python’s class system, even though it’s a fundamentally different concept.

In my ideal world, I would love to actually have both changes to the language. The protocol keyword could designate a structurally typed interface, while the make keyword could create “special” classes.

protocol Drawable:
    def draw(self) -> None: ...

protocol Debug:
    def debug(self) -> None: ...

make dataclass Square:
    length: int
    def draw(self) -> None: ...

def debug_render(obj: Debug & Drawable) -> None:
    ...
1 Like
10

My instinct is that this idea in the current form of syntactic sugar for a single type of declaration is not enough, in the absence of other features.
The argument is that structural typing is fundamentally different enough that it should have a different-looking declaration and not be tied to normal class semantics, but without any changes to how protocols are actually handled at runtime, or even really offering specific ways that protocol semantics should be different, the proposed change doesn’t actually address the motivation.

7 Likes
11

As stated by @a-reich, if the change doesn’t bring any additional features or changes to protocols that make them worthwhile, what’s the use of a shorter syntax?

In all fairness I would have expected a dedicated keyword to inherently bring the protocols as part of the builtin C-types given that this was done in the past for TypeVar. I do not think a new keyword can be snuck in without this, and it wouldn’t make sense if it weren’t.

2 Likes
12

I get where you’re coming from, but protocols are still classes. You can provide default implementations for its members and treat it like an optional superclass.

1 Like
13

I like the current inheritance syntax of protocols, which also shares ‘syntax’ with abc and (kind of) metaclasses

2 Likes
14

I strongly support this. I think the addition of the type soft keyword shows that small changes to spelling can go a long way – even (especially?) when they don’t have any runtime impact (edit: this is wrong, see below). The argument that classes and “interfaces” / Protocols are fundamentally different lands with me too. I know they’d still be classes under the hood, but that doesn’t take away from the benefit of making structural subtyping more natural.

1 Like
15

Sorry, but I think the analogy to the type statement isn’t accurate. It introduced a new runtime semantics, namely it creates aliases that are instances of the new TypeAliasType class.

2 Likes
16

That’s a good point, thank you for the clarification.