Functional Protocols

Ideas
1

Topic Introduction

Has anyone ever considered adding protocols to Python that fit the more functional style? For example, we already have the built-in len which simply calls the __len__ magic method on implementers of the “Sized” protocol. Let’s say I want to create a new protocol, “Renderable”, for a UI components. Currently I have two options:

Structural Protocols (PEP 544)

I could create a Protocol from PEP 544 that requires all classes to have a render method. This works, but has a couple downsides

  1. If I have a component that implements multiple protocols, then the names of the members must be distinct
  2. I must call the object method inst.render(...) rather than render(inst, ...). This creates a strange asymmetry between built-in protocols that are based on magic methods, and user-created protocols.
  3. There cannot be a default implementation of the method

Functools Singledispatch

I could instead create a render function using functools.singledispatch. This addresses my concerns from above, but has its own problems.

  1. The implementation of the function lives outside of the class body, but is often dependent on the class implementation. For example, you might want to reference “private” variables prefaced with an underscore. Doing this outside of the class body is considered bad practice. Overall, we have coupling between the class and function implementation
  2. There is no way to type-hint that the argument to some other function implements the protocol in the same way that PEP 544 provides.
  3. If we want to ensure at runtime that a class implements a protocol with multiple members, we must check that an implementation exists for every member.

Proposal

I believe there should be a version of protocols that take the best of both approaches.

A protocol is a function or collection of functions that a class must implement corresponding methods for.

@generic
def render(self):
  # default implementation

Renderable = Protocol(render)

class UIComponent:
  @render.implementation
  def _render(self):
    # class-specific implementation

render(UIComponent())

isinstance(UIComponent(), Renderable)  # True

Here the Renderable protocol has only one member, but we could have many members, and the idea is that protocols could be combined.

This addresses my concerns from above:

  • We do not rely on the name of the methods like magic methods or PEP 544 protocols, instead we use decorators.
  • These protocols mirror built-in ones in that we can call using a functional approach.
  • We can have a default implementation easily and transparently.
  • The implementation lives directly within the class body, and is explicit in which protocol it is implementing.
  • We should be able to easily type hint and perform runtime type checks on the protocol
    • I suspect that static type checkers like MyPy might have some issues handling this type of thing out of the box, but I am unfamiliar with how they are implemented.
1 Like
2

You can already accomplish the proximity of a render() implementation and class details with singledispatch():

import functools


@functools.singledispatch
def render(self):
    print("default implementation", self)


class UIComponent:
    # cannot @decorate because class is not yet defined; perhaps with Python 3.11 Self?
    def _render(self):
        print("class-specific implementation", self)


render.register(UIComponent, UIComponent._render)  # therefore, register after definition

render(1)  # generic
render(UIComponent())  # specific

print(any(isinstance(UIComponent(), cls)
          for cls in render.registry))  # actually default handler makes *any* class renderable
print(any(isinstance(UIComponent(), cls)
          for cls in render.registry 
          if cls is not object))  # but you can require specificity if you want

That resolves (although not so elegantly) the mentioned problem #1 for singledispatch(), I think. Problem #3 shouldn’t happen too often if we rely on duck-typing, or should it? But indeed problem #2 remains, as type checkers are left completely in the dark.

3

Yeah I think this is probably the best way to recreate what I’m describing given what we currently have. The main issues are readability/boilerplate and the inability to add type-hints. Not deal breakers, but not ideal either.

I’m sure people would say that we should just use normal protocols from PEP 544, and I will admit that my main issue with them is they are inherently object-oriented where I tend to lean for a more functional approach where possible