Anonymous protocols, objects

Imagine if you will a statically-typed language mode for Python, the important idea for this topic being type elision, i.e. types don’t exist at runtime.

Let’s say additionally that for larger applications, protocols play a considerable role.

In this language mode, we want to be able to define protocols that can be typed statically and without incurring a runtime impact – without having to define a class and inherit from Protocol.

Here’s a possible syntax which will probably be quite divisive for reasons that will become more clear later:

type Proto = \
    meth(self) -> int

# ^ More or less equivalent to a `Protocol` subclass with a
#   single method defined.

class C:
    i = 0
    def meth(self):
        return self.i

def func(x: Proto):
    return x.meth()

func(C())  # Passes static type check

The protocol exists as a type alias – it’s essentially anonymous.

The syntax actually builds on the general idea of anonymous block statements which is nothing new, stemming back from at least 2005 (see PEP 340).

The rule is that everywhere an expression is allowed today, we’ll now allow an anonymous block as well. The value of the expression is an anonymous object with whatever local variables as the “class”.

Thus, we can rewrite the last bit of the example above:

func(\
    i = 0
    def meth(self):
        return self.i
)

(And this of course works with return as well, assignments; as stated, everywhere an expression is otherwise allowed).

Now the benefit is that protocols and objects are really the bread and butter of Python – they should be easy to both define and implement. And why allow such a powerful syntax for type definition when it works just as well for creating objects.

Going further, we’ve got TypedDict now, but the more obvious syntax to type a dictionary is arguably:

type Movie = {
    "name": str,
    "year": int,
}

And this actually motivates the anonymous block statements syntax, too: we can’t use something like curly braces because they’re for dictionaries and sets; and brackets are for lists.

For objects, we want something more succinct: essentially nothing (for a single statement, we don’t even need the backslash).

Both the original Protocol proposal and TypedDict were designed around classes and subclasses in particular. But there’s an intersection proposal coming along which lets use take types further with just the type keyword; the proposal presented here builds on that momentum.

Finally, a note on the dictionary typing example above: for an optional key, instead of having to use TypedDict and total=False, a syntax such as "year"?: int could be devised – it’s more flexible and arguably more readable than the somewhat outlandish total.

1 Like

This isn’t really a concrete proposal right now, more like a stream of ideas, which is fine.

However, the central idea of “anonymous block statements syntax” is almost surely never going to happen, or if does, you are going to need a lot more motivation than “it allows us to add minimally better syntax for Protocols”. It has basically the same problems as MultiLineLambdas, which, as you might have noticed when you did your research have a long history of not being added. The central problem (to me at least) is that it adds indentation based syntax in the middle of expressions, which is a fundamental break from python syntax up until now. You will need to do a lot of work to convince many different people that this is a good idea.

An alternative and presumably more palatable spelling of the protocol type could be:

protocol Proto:
    meth(self) -> int

(Or interface which I prefer, but perhaps that’s from having spent too much time using Zope.)

Either term could be introduced as a soft keyword – just like type.

At the surface, it doesn’t look very Pythonic, not having def in there – and where is the optional docstring? But that’s code and not strictly speaking a type definition.

Ok, maybe I just don’t understand what your design goal is. Can you be clear about what problem you are trying to solve? What exactly is the drawback of writing class Proto(Protocol): ...? Since you clearly aren’t attached to the “anonymous” part of the post title.

Note that I don’t think there will be support for adding syntax that has no runtime access method. If you want that, use comments.

Thanks, the design goal was motivated by how TypeScript provides a typing system that exists outside of the target language which has the benefit that at runtime, types simply don’t exist.

In Python, as implemented in the current typing standards track, types are defined within the language such that it’s not exactly trivial to remove types from a program (i.e. in a runtime situation where type information was not useful, which is probably the most common situation). The stream of ideas was coming from this direction.

But with your input and having thought about it some more, even if at the surface, types such as those deriving from Protocol or TypedDict are an active part of the program, they can in principle be elided by a transpiler just as easily as the types specified in a TypeScript file.

This is easily true for Python too. Just define your protocols in their own file, import behind an if TYPE_CHECKING: guard, and use from __future__ import annotations. You can even use linters like Ruff to enforce the style.

1 Like

It’s worth noting that the type annotations I contributed to Chameleon already do this. They use flake8-type-checking to enforce the style, it’s a little more fully featured currently, compared to the Ruff implementation, but I do eventually want to port my improvements to Ruff, if someone doesn’t beat me to the punch.

1 Like

Is it possible to express something like implements Proto to the effect that during type-checking (i.e. with typing.TYPE_CHECKING set), you can declare that a class implements the protocol without having to inherit from it.

Something like:

if typing.TYPE_CHECKING:
    from .protos import Proto

class MyProto:  # type: Proto
    def meth(self):
        ...

Even inheriting won’t really give you the same thing as for example a zope.interface.Interface, since the error usually only pops up when you try to instantiate the class, at which point it will detect missing method implementations and warn you that the type is still abstract.

You can add type tests to your test suite to ensure things like that[1].

I.e. do something like

x: Proto = MyProto
MyProto()

and run the type checker on it, if MyProto doesn’t implement Proto or is abstract, then the first or second line will give you an error.

Also note that you don’t have to inherit from a Protocol to implement it, it’s a fully structural type, so as long as your type provides a compatible structure, then it is considered an implementation of that Protocol.


  1. similar to ensuring the interface was implemented in tests using things like assert interface.implementedBy(klass) and zope.interface.verify.verifyClass ↩︎

That makes sense and I suppose you’ll eventually trigger the issue when your code instantiates the class – but it does seem helpful to be able to declare that a given class implements some protocol, just to get the error at the definition site rather than when you’re trying to use it. Can’t have it all …

You could probably achieve something like that using a class decorator, although currently you would have to define a new one for each protocol you want to verify, due to the lack of higher kinded type vars:

FooT = TypeVar("FooT", bound=Foo)
def implements_foo(cls: FooT) -> FooT: ...

@implements_foo
def MyFoo:
    pass

This would raise an error if MyFoo wasn’t compatible with Foo.

Technically you could get away with a generic version in mypy, because the return type of class decorators is ignored, but in pyright you would be changing the type MyFoo to Foo using this version:

def implements(proto: T) -> Callable[[T], T]:
    return lambda x: x

@implements(Foo)
class MyFoo:
    pass

Ideally the implements function could be written out using a generic upper bound, but this doesn’t seem to work:

def implements[T, V: T](x: T) -> Callable[[V], V]:
#                    ^  "T" is not defined
    return lambda x: x

It would seem that within a type parameter list, you’re not able to refer to already defined parameters.

In TypeScript, you can say <T, V extends T> and it just works.

Yes, even if you write it out with TypeVar you are not allowed to use another TypeVar for the bound argument (unless it is bound to the current scope), conversely you are allowed to do that however with the default argument introduced by PEP 696, that doesn’t really help in this case though.

Once we have intersections we could spell this another way:

def implements[ProtocolT: type[Protocol]](
    proto: ProtocolT
) -> Callable[[T & ProtocolT], T]:
    return lambda x: x