Provide a canonical way to declare an abstract class variable

There’s a recent help post of Abstract variables in abc that asks about how an “abstract variable” can be declared such that it is required for a subclass to override the variable, to which @drmason13 replied:

Although this approach of abusing an abstract property as an abstract variable works and is widely accepted, it feels like a workaround and an unnecessary learning hurdle to newcomers.

I think it would make the code more readable if the abc module officially supports a more pronounced and intuitive way of declaring an abstract class variable

My first thought was to assign to an abstract variable a sentinel object abc.ABSTRACT_VARIABLE:

from abc import ABC, ABSTRACT_VARIABLE

class BaseClass(ABC):
    required_attr = ABSTRACT_VARIABLE

class SubClass(BaseClass):
    pass

SubClass()
# TypeError: Can't instantiate abstract class SubClass without assigning a value to abstract variable 'required_attr'

However, a sentinel object can’t be declared with a specific type, so my second idea is to add a new generic alias such as typing.AbstractVar (or maybe abc.AbstractVar makes more sense, or both) for this purpose:

from abc import ABC
from typing import AbstractVar

class BaseClass(ABC):
    required_attr: AbstractVar[str]

class SubClass(BaseClass):
    pass

SubClass()
# TypeError: Can't instantiate abstract class SubClass without assigning a value to abstract variable 'required_attr'
2 Likes

I’d probably still use this, but I’d prefer to also allow consumers of my ABCs to set required_attr as an instance variable in their __init__, which precludes using an ABC altogether.

Protocols work a lot more smoothly with typing. They even have:

https://typing.readthedocs.io/en/latest/spec/class-compat.html#classvar

What’s wrong with:

from abc import ABC

class BaseClass(ABC):
    required_attr = None

    def __init__(self):
        if required_attr is None:
            raise TypeError('Subclasses must set required_attr to a non-None value')

I wonder if syntax similar to dataclass could be used here, and an abstract variable could be defined as simply as required_attr: str in the definition of the ABC. I guess the question is whether there’s any other use for such a variable definition–I can’t think of one in the moment but I’m not a huge user of ABCs.

Not much, but it relies on the subclass’s __init__ calling super().__init__

__init__ will never be called. And even if called in subclass initiliazer, base initiliazer should be called before any subclass actions

True but would a required_attr as defined need to be defined ahead of the __init__ call? That’s how I would interpret it. Not as something that is set by __init__ but as something that is prior to it being called.

No, in both @abstractmethod and AbstractAtrribute examples, it would fail before any memory for an instance is allocated. SubClass() would first reach ABCMeta.__call__ and fail inside it. ABC-s work on classes, not instances.

Pointess memory allocation is not really a big of a deal, but this solution just begs for missuse. This check should fail ragardless of state of instance:.

if getattr(type(self), "required_attr", None) is None:
    raise

See [abc] Add abstract attributes via `abstract` type-hint which has essentially this same request. Basically attributes that are not defined on the parent but which MUST be defined on the subclass. In that post I share my favorite workaround:

from abc import ABC

class Foo(ABC):
    myattr: int

    def __init__(self, myattr: int):
        self.myattr = myattr

class Bar(Foo):
    def __init__(self):
        super().__init__(myattr=15)

The parent class simply has my_attr as a required constructor argument so subclasses can’t get around defining it. Though they could include my_attr as a required argument into their own constructor which is kind of like saying the subclass also has that attribute as abstract.

3 Likes

Could we not have something like ?

class A:
name = required_property() # property decorator that throws error by default when it is accessed

class B(A):
def init(self):
self.name = “Frodo” (this property is now set so won’t throw error)

I didn’t find any other language that allows you to create abstract class attributes. However, each language has its own way of creating abstract instance attributes. In Python, an instance attribute can be a property or a method. I believe the current approach is the canonical way to declare abstract instance attributes.

I don’t think the original poster in the other thread meant to refer to abstract class attributes but rather to abstract instance attributes.

Runtime checking isn’t a problem, as AttributeError gets the point across nicely.

Similarly, the property version is the correct approach for instance variables (it even has the nice property of making the instance variable covariant)

What’s technically missing is a way to tell typecheckers (and the ABC metaclass) that concrete ABC subclasses must provide a particular class variable.

I haven’t found that to be a major problem in practice (unit tests tell you pretty quickly if something is wrong), but the descriptive gap does exist (I’m not counting “define a custom metaclass and a regular required property” as a reasonable solution)

5 Likes

Oh I see, what I posted realizes an abstract instance attribute, in a sense. But if you really want an abstract class attribute then yeah, there’s not really a semantic way to do that.

Yes that’s exactly what’s being proposed here, an abstract class variable, not an abstract instance variable.

A property is supposed to be bound to an instance, which a class variable is not, but the current workaround of an abstract class variable using an abstract property happens to work because ABCMeta.__new__ only checks if the name of an abstract property is defined at all in the subclass, and doesn’t care if it’s also defined as a property in the subclass, which is why I find the workaround to be counter-intuitive and implementation detail-reliant.

It isn’t a major problem because we can either use the said workaround or simply choose to not declare the class variable as abstract at all, and rely on a custom runtime check in a class method that needs the class variable, while an actual abstract class variable would’ve made the same code that much more elegant.

A good example is the fixer_base.BaseFix.PATTERN class variable in the now deprecated lib2to3 package, which is really meant to be an abstract class variable but is instead commented as such because there is no good official way to make it one:

I also occasionally miss the possibility to declare an abstract class variable. What I usually do in such cases is just using bare ClassVar[...] annotations + adding a suitable comment, e.g.:

class MyAbstract:
    # Required to be provided by concrete subclasses:
    foo: ClassVar[int]
    spam: ClassVar[str]
    name_to_parrot: ClassVar[Mapping[str, Parrot]]

# And somewhere later...

class MyConcrete(MyAbstract):
    foo: ClassVar[int] = 42
    spam: ClassVar[str] = 'ham'
    name_to_parrot: ClassVar[Mapping[str, Parrot]] = {"Polly": Parrot()}

That’s not perfect, as it does not provide any runtime abc.ABCMeta-like validation, but usually I don’t bother enough.

If I did, I’d probably implement something along the lines of the following (pseudo-)code:

import abc
import functools

class EnhancedABC(
    # Just providing *also* the normal ABCMeta's validation:
    metaclass=abc.ABCMeta,
):
    # My custom extra validation:

    def __init_subclass__(cls, /, **kwargs):
        super().__init_subclass__(**kwargs)
        orig_init = cls.__init__
        if not getattr(
            orig_init,
            "_EnhancedABC__abstract_class_attr_verification_provided",
            False,
        ):
            @functools.wraps(orig_init)
            def __init__(self, /, *args, **kwargs):
                self.__verify_abstract_class_attributes()
                orig_init(self, *args, **kwargs)
        __init__.__abstract_class_attr_verification_provided = True
        cls.__init__ = __init__

    @classmethod
    @functools.cache
    def __verify_abstract_class_attributes(cls):
        attrs: Set[str] = (  # here: pseudo-code for brevity
            <"scan cls.__mro__ for runtime attr names">)
        clsvars: Set[str] = (  # here: pseudo-code for brevity
            <"scan cls.__mro__ for ClassVar-annotated attr names">)
        if missing := clsvars - attrs:
            raise TypeError(
              f"Can't instantiate abstract class {cls.__qualname__} " 
              f"with unassigned abstract class attribute(s): "
              f"{', '.join(sorted(missing))}")

Then I could use EnhancedABC like so:

class MyAbstract(EnhancedABC):
    foo: ClassVar[int]
    spam: ClassVar[str]
    name_to_parrot: ClassVar[Mapping[str, Parrot]]

class StillAbstract(MyAbstract):
    foo: ClassVar[int] = 42
    spam: ClassVar[str] = 'ham'

class MyConcrete(MyAbstract):
    foo: ClassVar[int] = 42
    spam: ClassVar[str] = 'ham'
    name_to_parrot: ClassVar[Mapping[str, Parrot]] = {"Polly": Parrot()}

# This should be ok:
obj = MyConcrete()

# But each of these should cause TypeError:
StillAbstract()
MyAbstract()
1 Like

Yes it’s certainly doable to extend ABCMeta and implement our own logics that validate abstract class variables, though if Python has official support for abstract class variables the devs of major IDEs and extensions would be encouraged to incorporate such validations into the IDEs like they do for abstract methods:

1 Like

I wonder if something like this would be useful:

import abc
from typing import (
    Abstract,  # <- would be a new thing
    ClassVar,
)

class MyAbstract(abc.ABC):
    foo: Abstract[ClassVar[int]]
    spam: Abstract[ClassVar[str]]
    name_to_parrot: Abstract[ClassVar[Mapping[str, Parrot]]]

Static type checkers (and IDEs) would complain if a (sub)class was instantiated without having assigned all attributes marked with this new qualifier.

Perhaps, ABCMeta could also implement a suitable runtime check…

That isn’t what I meant, I meant just declaring the variable on the base ABC, and then letting the AttributeError escape at runtime if the subclass doesn’t define the value.

A specific example from my current project (involving an ABC that is also a data class, hence the use of ClassVar):

class _PythonEnvironmentSpec(ABC):
    "Common base class for Python environment specifications"

    # Optionally overridden in concrete subclasses
    ENV_PREFIX = ""

    # Specified in concrete subclasses
    kind: ClassVar[LayerVariants]
    category: ClassVar[LayerCategories]
    ... # Rest of class definition
@dataclass
class RuntimeSpec(_PythonEnvironmentSpec):
    kind = LayerVariants.RUNTIME
    category = LayerCategories.RUNTIMES
    ... # Rest of class definition
@dataclass
class _VirtualEnvironmentSpec(_PythonEnvironmentSpec):
    # Intermediate class for covariant property typing (never instantiated)
    runtime: RuntimeSpec = field(repr=False)
@dataclass
class FrameworkSpec(_VirtualEnvironmentSpec):
    ENV_PREFIX = "framework"
    kind = LayerVariants.FRAMEWORK
    category = LayerCategories.FRAMEWORKS
    ... # Rest of class definition
@dataclass
class ApplicationSpec(_VirtualEnvironmentSpec):
    ENV_PREFIX = "app"
    kind = LayerVariants.APPLICATION
    category = LayerCategories.APPLICATIONS
    ... # Rest of class definition

The type system won’t actually prevent you from instantiating _PythonEnvironmentSpec or _VirtualEnvironmentSpec here, and it won’t complain directly if you comment out the kind or category variables in one of the concrete subclasses.

But if you tried to actually do that, you’re going to get an AttributeError at runtime from the code that actually references those class attributes.

Could type checkers potentially do a better job of detecting this situation and complain if you try to instantiate the classes with missing class variables? Probably, but that’s arguably just a typechecker implementation issue, rather than necessarily being a change to the type system specification itself.

Yes that’s what I was alluding to in the OP. Wrapping ClassVar in Abstract may be unnecessary unless it’s in a data class though.

I see. But the whole point of an abstract base class, like the typing system, is to catch errors as early as possible. If we rely on runtime AttributeError or NotImplementedError or other misbehaviors to occur only when access to a missing or non-overriden attribute or method is attempted, we would likely have to spend much more time debugging because the error would possibly occur much later at downstream, and possibly only conditionally.

But how can a type checker know if a class variable defined in a base class is “missing” in a subclass if there is no good way to indicate that the class variable is abstract? The language should provide an official framework so the tools have a standard to follow.

C++20 concepts can requier existence of static variable, but that’s more of a side effect of templates. C++ templates can fail to type substitute, if no checked type has it, but that’s itself is more of a side effect.

2 Likes