Abstract Base Class Safety Enhancements

Ideas
1

Hello, first time posting so let me know if I could do anything differently.

I recently was messing around with a custom metaclass which does registration-time validation of ABC implementations (ensuring that an implementation of the base virtual functions actually exists and is type-correct.) This also catches missing implementations when using virtual inheritance, which iirc doesn’t happen otherwise (only with regular inheritance.)

Overall I think this would be a simple but incredibly powerful addition to the standard library. I think the most questionable part of it might be the type normalization… open to ideas on how to make that more conformant.

It’s here or on PyPI if you want to mess around with it: GitHub - ralian/abcreg: Stricter Abstract Base Classes for Python · GitHub

2

What’s missing from regular ABCs that ABCReg brings additionally? From the README, ABCReg

provide(s) additional interface checking at registration time to ensure that registered virtual and ordinary subclasses are valid.

What kind of interface checking does it do exactly?

Can you make an example?

1 Like
3

Sure, it is maybe a bit unclear what I was trying to do. Consider this bad example, which executes without issue:

from abc import ABC, abstractmethod

class LogHook(ABC):
    @abstractmethod
    # Perform some action on a log message, and return whether we should propogate to other hooks.
    def handlemsg(self, msg: str) -> bool:
        ...

# Client code: a lazy implementation which does not return the correct type!
@LogHook.register
class LogWriter(object):
    def handlemsg(self, msg):
        with open('logfile', 'a') as fout:
            fout.write(f"LogWriter: {msg}\n")
        pass

# In fact, with virtual inheritance, we could skip implementing handlemsg altogether:
@LogHook.register
class LogWriter2(object):
    ...

# We can construct this list without issues, but there are no guarantees about the functionality of the implementation.
hooks : list[LogHook] = [LogWriter(), LogWriter2()]

There is tooling in ABC to catch this if you use regular inheritance, but not with virtual inheritance. IMHO, this is an inconsistency in design. I added extra checks on registration in my design, so if you switch to AnnotatedABC you get the following errors when the registration is attempted:

Traceback (most recent call last):
File "/home/ralian/abcreg/main.py", line 11, in
@LogHook.register
^^^^^^^^^^^^^^^^
File "/home/ralian/abcreg/abcreg/annotated_abc.py", line 36, in register
cls._validate_implementation(subclass)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/home/ralian/abcreg/abcreg/annotated_abc.py", line 46, in _validate_implementation
raise TypeError(msg)
TypeError: <class 'main.LogWriter'> does not fully implement 'LogHook': missing or mismatched 'handlemsg'

Yes, this might not be useful if you control all the code in your environment. But, if you are writing something like a plugin API as I am, it’s a great way to ensure consumer code which is incorrectly implemented gets clear and obvious error reporting, while still using virtual inheritance.

4

Oh, and you can see the flow of how exactly this mismatch is detected here: abcreg/abcreg/annotated_abc.py at main · ralian/abcreg · GitHub

If all attributes marked with __isabstractmethod__ match, this of course just calls super().register(subclass) on ABC.

5

Ah, now I see. So with regular ABCs virtual registration will still occur even if a virtual subclass doesn’t strictly implement the method’s signatures. This is actually quite nice, thanks.

EDIT: I’m also working on a plugin API of sorts so I might be interested in this. Could you expand on the project you’re working on?

6

Of course! It’s for a code forge I’m working on with FastAPI. It’s in the very early stages, but there are going to be several plugin interfaces, namely for (1) custom actions and (2) custom pages provided by loaded plugins.

And, it’s really a minimum viable example right now, but I do have an example plugin that uses this api:

edit: sorry, I have to restart the server, so those links might not work for a few minutes.

1 Like
7

While doing more digging on this, I think one possible solution is to just use Protocols. It’s not perfect because you don’t get checking at module load time, but it is more ergonomic. Mypy isn’t very happy when I switch my plugin api to this, even if I turn on runtime checking.

8

I did some more research and released a new version of abcreg (0.2) which has support for checked protocols. Protocols were a bit trickier than ABCs due to needing implementers to be direct subclasses. But, if you look at the new example on github, you can now create runtime checked protocols which also check on registration:

from typing import Protocol, runtime_checkable
from abcreg import AnnotatedProtocolMeta

@runtime_checkable
class LogHook(Protocol, metaclass=AnnotatedProtocolMeta):
    def handlemsg(self, msg: str) -> bool: ...

@LogHook.register
class LogWriter:
    def handlemsg(self, msg: str) -> bool:
        return True

Why? Again, it’s a niche use case, but practically for me this means once my plugin system is switched over to checked protocols, the user will get an error about mismatched interfaces at plugin load time rather than a potentially nasty runtime error when the plugin is invoked, or even logic errors which don’t raise any exceptions if the type mismatch is subtle enough.

9

I have to check but I’m leaving this comment as a reminder that this might actually be useful for me as well. Thanks for sharing, I was indeed thinking that you could have used protocols with all it’s quirks and limitations but you beat me to it

1 Like