Typing factory functions that are instance methods of a metaclass

Python Help
,
1

Consider

from typing import Any, TypeVar, reveal_type
T = TypeVar('T')
class Meta(type):
    def deserialize(cls: type[T], spec: dict[str, Any]) -> T:
        cls._munge_spec(spec)
        return cls(**spec)

    def _munge_spec(cls, spec: dict[str, Any]) -> None:
        # to be overridden by classes with this metaclass,
        # using @classmethod
        pass

class Blah(metaclass=Meta):
    def __init__(self, **kwargs: Any):
        pass

    @classmethod
    def _munge_spec(cls, spec: dict[str, Any]) -> None:
        spec["blah_added"] = True

# usage:
blah = Blah.deserialize({"foo": 23})
reveal_type(blah) # must be Blah

(They’re not visible in this cut-down example, but I do have a good reason to write a metaclass, and I also have a good reason to define class methods using instance methods of the metaclass.)

This works, but it doesn’t typecheck, because the type checker has no way of knowing that, at runtime, type[T] will in fact always be an instance of Meta. But if I change the annotation from type[T] to "Meta", then I no longer have a way to name the type of the return value, so I just trade one type checker error for another.

Can this code be made to pass mypy --strict somehow? Needs to work, and typecheck, in Python 3.11 and later.

2

Is there a good reason why you need a custom meta class at all?

_munge_spec is apparently meant to be an abstract class method, so you can define an abstract base class and use typing.Self to refer to the type of the instance:

from typing import Any, reveal_type, Self
from abc import abstractmethod, ABC

class BlahABC(ABC):
    @classmethod
    def deserialize(cls: type[Self], spec: dict[str, Any]) -> Self:
        cls._munge_spec(spec)
        return cls(**spec)

    @abstractmethod
    @classmethod
    def _munge_spec(cls, spec: dict[str, Any]) -> None:
        # to be overridden by a subclass
        # using @classmethod
        pass

class Blah(BlahABC):
    def __init__(self, **kwargs: Any):
        pass

    @classmethod
    def _munge_spec(cls, spec: dict[str, Any]) -> None:
        spec["blah_added"] = True

# usage:
blah = Blah.deserialize({"foo": 23})
reveal_type(blah) # main.py:27: note: Revealed type is "__main__.Blah"

mypy playground

3

Yes. The reasons are not visible here, but it’s absolutely necessary that there be a metaclass.

You’re correct that _munge_spec is an abstract class method, but, given that there’s a metaclass in play, and that _munge_spec’s caller is a concrete method of the metaclass, I’m 99% sure _munge_spec does in fact have to have its base definition be an instance method of the metaclass.

… Hm, writing that down made me think of a potential workaround (move both methods to a mixin base class) but I would still like to know if there is an answer to the question I originally asked.

4

No, _munge_spec does not have to be an instance method of a meta class.

You just need to make your custom meta class inherit from abc.ABCMeta instead of type:

from typing import Any, reveal_type, Self
from abc import abstractmethod, ABCMeta

class BlahMeta(ABCMeta):
    # your own metaclass attributes and methods here
    # call super() when overriding methods of ABCMeta
    pass

class BlahBase(metaclass=BlahMeta):
    @classmethod
    def deserialize(cls: type[Self], spec: dict[str, Any]) -> Self:
        cls._munge_spec(spec)
        return cls(**spec)

    @abstractmethod
    @classmethod
    def _munge_spec(cls, spec: dict[str, Any]) -> None:
        # to be overridden by a subclass
        # using @classmethod
        pass

class Blah(BlahBase):
    def __init__(self, **kwargs: Any):
        pass

    @classmethod
    def _munge_spec(cls, spec: dict[str, Any]) -> None:
        spec["blah_added"] = True

# usage:
blah = Blah.deserialize({"foo": 23})
reveal_type(blah) # main.py:31: note: Revealed type is "__main__.Blah"
5

That’s (approximately) the workaround I mentioned.

I am still looking for an answer to the question I originally asked.

6

You can use a Protocol as the type bound

from typing import Any, TypeVar, reveal_type, Protocol

class HasMungeSpec(Protocol):
    @classmethod
    def _munge_spec(self, spec: dict[str, Any]) -> None:
        pass

T = TypeVar("T", bound=HasMungeSpec)

class Meta(type):
    def deserialize(cls: type[T], spec: dict[str, Any]) -> T:
        cls._munge_spec(spec)
        return cls(**spec)

    def _munge_spec(cls, spec: dict[str, Any]) -> None:
        # to be overridden by classes with this metaclass,
        # using @classmethod
        pass

class Blah(metaclass=Meta):
    def __init__(self, **kwargs: Any):
        pass

    @classmethod
    def _munge_spec(cls, spec: dict[str, Any]) -> None:
        spec["blah_added"] = True

# usage:
blah = Blah.deserialize({"foo": 23})
reveal_type(blah) # Blah

Then

$ mypy --strict t.py 
t.py:30: note: Revealed type is "t.Blah"
Success: no issues found in 1 source file