Typing the _missing_ method of an StrEnum’s sub-class

Typing
1

Hi,

Somewhat related discussion on mypy: Confusing error message for a missing classmethod decorator · Issue #11791 · python/mypy · GitHub

An Enum’s class method _missing_() (docs) is typed in Typeshed as follows (code):

@classmethod
def _missing_(cls, value: object) -> Any: ...

That means that sub-classes like StrEnum and IntEnum also inherit that same signature. However, I expected those signatures to be adjusted for the sub-class, for example (code)

class StrEnum(str, ReprEnum):
    def __new__(cls, value: str) -> Self: ...
    _value_: str
    @_magic_enum_attr
    def value(self) -> str: ...
    @staticmethod
    def _generate_next_value_(name: str, start: int, count: int, last_values: list[str]) -> str: ...

seems to miss

def _missing_(cls: value: str) -> Self  # or return `str`

I’m also a little unclear on the return type here. The docs say

By default it does nothing, but can be overridden to implement custom search behavior

which I assume means that the method searches for and returns instances of _value_ or itself. See also discussion Enum classes and Self.

All this head-scratching because I’m trying to type such a custom _missing_() method of a StrEnum sub-class and I get conflicting errors. When typing value: str then

error: Argument 1 of "_missing_" is incompatible with supertype "enum.Enum"; supertype defines the argument type as "object"  [override]
        def _missing_(cls, value: str) -> typing.Self:
                           ^~~~~~~~~~
note: This violates the Liskov substitution principle
note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides

or when typing value: object then

error: Argument 1 to "MyStrEnum" has incompatible type "object"; expected "str"  [arg-type]
            obj = cls(value)
                      ^~~~~
error: Incompatible types in assignment (expression has type "object", variable has type "str") [assignment]
            obj._value_ = value
                          ^~~~~

So I changed the code in our sub-class to

@typing.overload
@classmethod
def _missing_(cls, value: str) -> typing.Self: ...

@typing.overload
@classmethod
def _missing_(cls, value: object) -> typing.Self: ...

@classmethod
def _missing_(cls, value):
    # Implementation goes here.

And this now complains about

error: Function is missing a type annotation  [no-untyped-def]
        def _missing_(cls, value):
        ^

Hm, any recommendations?

2

I’m not conversant with typing, so I might be misunderstanding, but the return type for _missing_ is `None`, an instance of the enum, or raising an exception.

3

making the input type str is incorrect - For any StrEnum S, S(1) is perfectly valid code that results in _missing_ being called with argument type int.

You need to explicitly handle non-str arguments inside of the function.

I am less sure about the return type. I think it should be None | Self, not sure why they are doing Any. Might be that Self has problems if used in conjunction with Enum?

4

Does the argument need to be convertible to str? Like for IntEnum does value need to accept Any (or narrowed: object) or can it accept something with a __int__ only? If so, we could annotate value: ConvertibleToStr or value: ConvertibleToInt with some simple protocols.

5

No. _missing_ is going to receive arbitrary objects. You yourself can then restrict the acceptable types, but that is not reflected in the signature of this method.

(Also, every python object is convertible to str)

6

I guess they inherit __str__ from object, but what’s about some protocol for IntEnum where they have to have some __int__? Also, does that account the possibility of __str__(self) -> Never?

7

You are still missing the point.

It doesn’t matter at all if the objects are convertible to str or int. They are going to be passed to _missing_ and it’s _missing_'s job internally to reject them.

The correct signature is _missing_(cls, value: object) -> Self | None. Use it (or well, you might need to use -> Any instead, not sure). Internally, reject unknown objects with if not isinstance(value, whatever): return None.

8

So then some overload like

@overload
@classmethod
def _missing_(cls, value: whatever) -> Self: ... # Self | Any too?
@overload
@classmethod
def _missing_(cls, value: object) -> None: ... # Here some Not[whatever] would fit best

would be best?

9

Yeah, I guess that would be accurate (assuming you accept all of whatever). Not sure if type checkers are going to respect it.

10

Perhaps turning `Enum` into an generic type with optional invariant type parameter could work?

1 Like
11

Well having it generic at runtime wouldn’t benefit anyone I guess, but generic for type checkers should work. (Only generic in stub file)?!

12

You mean Enum._missing_? I would think that for StrEnum._missing_ it would be:

_missing_(cls, value: str) -> Self|None  # (or whatever the return type should be)

Sure it’s valid code, but from a typing standpoint it’s wrong.

1 Like
13

Then propose those changes to typeshed. Currently they are strongly expressing the opinion that Enum._missing_ == StrEnum._missing_, both having the same signature.

Note that for Enums implementation this just doesn’t matter - they always reject.

This is most similar to object.__eq__ - another method for all subclasses are expected to also use the signature (self, other: object), making no type-level assumptions about the objects involved.

Subclasses of Enum are perfectly free to accept any objects they want. E.g. a Flag subclass might decide to accept a tuple of values and combine them into a singl Flag value: SF(('A', 'B', 'C')).

14

I don’t understand what you mean by this?

15
16

Ah, I see.

So if I understand correctly, only Enum._missing_ is typed, and it’s typed loosely enough to work with any enum subclass. If it was strictly typed (-> None), then every subclass would have to also be typed or get errors from a type checker.

Does that sound right?

17

If it was strictly typed in the sense of -> None, then no subclass would ever be allowed to return anything but None - which is obviously not desirable. Ideally it would be -> Self | None, but I suspect that has some weird issues related to Self, so they had to settle for Any.

1 Like