A missing counterpart of `override`? (a.k.a. `virtual`)

In Python 3.12, we have:

  • abc (PEP 3119) and typing.Protocol (PEP-544)
  • typing.final (PEP 591)
  • typing.override (PEP 698)

But is there a way to express the idea of something similar to, say, C++ virtual, C# virtual, or VB Overridable?

Consider the following toy example:

class Parser[T: Format](Protocol)

    def pre_parse(self) -> None:
        pass
        
    def post_parse(self) -> None:
        pass

    def parse(self) -> T:
        self.pre_parse()
        # do the job
        self.post_parse()


class FunkyFormatParser(Parser[FunkyFormat]):

    @override
    def pre_parse(self) -> None:
        """Makes extra steps required for this
        funky (no pun intended) format.
        """
        # do extra steps

Here we have two overridable methods — pre_parse() and post_parse() — and one method that is not intended to be overridden — parse().

There are at least two ways to express our intentions:

  1. We could mark parse as a final method, but we really don’t want to do it, if someone really needs to extend the parse() method in a subclass, we prefer to let they do it.
  2. We could mark both pre_parse() and post_parse() as abstractmethods. We don’t want to do it either, it would be very cumbersome for end users because all abstractmethods must be overridden (even such no-op/default implementations methods).

Basically, what I’m looking for is like a less restrictive version of abstractmethod, a clear marker that some method/property MAY (not MUST!) be overridden:

class Parser[T: Format](Protocol)

    @overridable
    def pre_parse(self) -> None:
        pass
      
    @overridable
    def post_parse(self) -> None:
        pass
        
    def parse(self) -> T:
        self.pre_parse()
        # do the job
        self.post_parse()

I avoided well-known final in favor of made-up overridable because final (at least in C++) is more than a mere “it could be overridden” marker, it also means dynamic dispatch.

Is there a way to express my idea using the standard library? Am I missing something? Or maybe it’s just a silly idea that tries to solve the problem that doesn’t even exist?


offtopic:

An error occurred: Sorry, new users can only put 2 links in a post.

It’s funny that links to PEPs are counted too. Had to delete not only all links to C++/C#/VB docs
(understandable), but all PEP links too :slight_smile:

So in C++ you need the virtual marker to get dynamic dispatch right? That’s not necessary in Python, all methods are automatically dispatched dynamically and are overrideable by default. So I don’t see what this marker would add.

3 Likes

We essentially have four ways you can mark as method now:

  • @final: this method may not be overridden
  • @abstractmethod: this method must be overridden
  • @override: this method must be an override. (This is essentially orthogonal to the rest; you could meaningfully combine with @final.)
  • The default: this method exists, you can choose whether or not to override it.

What you’re asking for is something that says that you don’t have to override a method, but you might want to. What would that actually do in a type checker? You say that users usually shouldn’t override the parse method, but they can if they want to; and they often should override the pre_parse method, but they don’t have to. That sounds to me that there isn’t a difference in how type checkers should treat parse and pre_parse, so I don’t see a case for adding a feature to the type system.

9 Likes

I’ve understood my mistake. This is a wrong category for my question. The “Typing” category is all about, well, typing system and type checkers. What I asked has nothing to do with typing or type checking, it is but a human-readable marker, some sort of documentation. As @jelle noted, it has zero sense for static type checkers, there are no sound rules how type checkers should handle all of this mess (“what was the user’s intention?”).

Hence, there is no place for such a thing in the typing module (and to an extent in the stdlib as a whole). If someone really wants something like this (instead of, for example, comment blocks and/or docstrings), they can easily use a simple identity function as such a marker-decorator:

def overridable[T, **P](meth: Callable[P, T]) -> Callable[P, T]:
    return meth


class Parser[T: Format](Protocol)

    @overridable
    def pre_parse(self) -> None:
        pass

Thanks for the discussion and sorry for bothering.

4 Likes

You didn’t bother anyone!

6 Likes

A good use case for this is detection of “method may be static” situations. A base class may not use self, but an override might. As I understand it, this is an inspection done by PyCharm, but in my experience, PyCharm will quickly get with the program and make the most of a @virtual decorator. However, I believe the warning disappears once there is such an override, so it has limited value, but it would be a nice indicator to the reader that the method is intended to be overridden and to get rid of the warning right away.