Enum classes and Self

[InSync]

The 4 major type checkers are handling this snippet very differently:

(playground links: Mypy, Pyright, Pyre; Pytype was run with -k -n)

from enum import Enum, auto
from typing import Self, final

class C: ...
class CEnum(C, Enum): ...

class CI(CEnum):
    A = auto()
    B = auto()

    @classmethod
    def return_member(cls) -> Self:
        # mypy     => Incompatible return value type (got "CI", expected "Self")
        # pyright  => Expression of type "Literal[CI.A]" is incompatible with return type "Self@CI"
        # pyre     => Expected `Variable[_Self_playground_input_CI__ (bound to CI)]` but got `CI`.
        # pytype   => fine
        # expected => fine
        return cls.A

@final
class FinalCI(CEnum):
    A = auto()
    B = auto()

    @classmethod
    def return_member(cls) -> Self:
        # mypy     => fine
        # pyright  => Expression of type "Literal[FinalCI.A]" is incompatible with return type "Self@FinalCI"
        # pyre     => Expected `Variable[_Self_playground_input_FinalCI__ (bound to FinalCI)]` but got `FinalCI`.
        # pytype   => fine
        # expected => fine
        return cls.A

# mypy     => Cannot extend enum with existing members: "CI"
# pyright  => Enum class "CI" is final and cannot be subclassed
# pyre     => fine
# pytype   => fine
# expected => error
class CIChild(CI):
    ...

@final
class D:

    @classmethod
    def return_member(cls) -> Self:
        # mypy     => fine
        # pyright  => Expression of type "D" is incompatible with return type "Self@D"
        # pyre     => Expected `Variable[_Self_playground_input_D__ (bound to D)]` but got `D`.
        # pytype   => fine
        # expected => fine
        return D()

Results:

• Mypy: 3/4
• Pyright: 1/4
• Pyre: 0/4
• Pytype: 3/4

According to the specs:

An Enum class with one or more defined members cannot be subclassed. They are implicitly “final”.

Perhaps a clarification like this could be added along with corresponding tests?

When used as the return type of an enum class’s method, Self means the same as the class itself:

class Shape(EnumWithNoMembers):
    SQUARE = 1
    CIRCLE = 2

    def m(self) -> Self:  # OK (there can be no subclasses of Shape)
        return Shape.SQUARE

    @classmethod
    def cm(cls) -> Self:  # OK (there can be no subclasses of Shape)
        return Shape.SQUARE

[erictraut]

I recommend against using Self in this case. The Self type is designed to support subclassing, and as you’ve pointed out, enums cannot be subclassed. All of the type checkers work fine if you switch to Shape.

[LewisGaul]

I just ran into this, using Self here seems like the right thing to do, avoiding needing to specify the class name twice? Why should Enum be an exception to the normal rule here?

[erictraut]

@LewisGaul

Why should Enum be an exception to the normal rule here?

Which rule are you referring to? Or are you just saying that you’ve used Self in similar situations and intuitively expect it to work here?

Enums require significant special handling in type checkers. The typing spec includes a lengthy chapter on this topic. The key here is the statement “Type checkers should infer a literal type for all members.”. In the code sample above, A and B are members, so their types are inferred to be Literal[CI.A] and Literal[CI.B], respectively. Their types are notably not inferred as Self.

[MegaIng (Cornelius Krupp)]

But surely Literal[CI.A] is a subtype of CI? And since the sensible choice for final classes is Self = CI, then surely Literal[CI.A] should also be compatible with Self?

Which step in this (very simple) logic chain is incorrect to justify not supporting this? Yes, Self would just be a very small convenience features in final classes/Enums, but I don’t think implementation effort justifies surprising and confusing users in this case.

[erictraut]

Yes, Literal[CI.A] is a subtype of CI, but it is not assignable to Self.

For details about Self and how type checkers should treat it, refer to this section of the typing spec. In short, you can think of Self as a type variable whose upper bound is the class that contains it. It is not simply a shortcut for specifying the class itself.

This code sample may help clarify.

from typing import Self

class Foo:
    def ok1[T: Foo](self: T) -> T:
        return self  # OK

    def ok2(self) -> Self:
        return self  # OK

    def bad1[T: Foo](self: T) -> T:
        return Foo()  # Type error

    def bad2(self) -> Self:
        return Foo()  # Type error

[MegaIng (Cornelius Krupp)]

Ok, then the spec should be update to match mypys behavior, where the final declaration changes how Self is interpreted. In mypy, the last line is not an error if the class is annotated with @final.

[erictraut]

@MegaIng, you’re welcome to propose spec changes. Here’s the process. If you (or anyone else) would like to pursue that, I recommend starting a new thread. I don’t think this change would require a PEP, so if you can get consensus from the community on your proposal, the TC could approve such a change. Getting the wording right might be tricky, so it would likely require some iteration and feedback.

What you’re proposing seems sound from a typing standpoint, but I don’t understand why such a rule would be specific to Self and not other type variables. I think mypy is inconsistent to honor @final in the bad2 method but not in the bad1 method.

I’ll also note that Enum classes are not typically marked @final explicitly, so if this were specified, you would probably want the spec’ed behavior to also handle “implicitly-final” classes.

[LewisGaul]

Which rule are you referring to? Or are you just saying that you’ve used Self in similar situations and intuitively expect it to work here?

The first line of PEP 673 is:

This PEP introduces a simple and intuitive way to annotate methods that return an instance of their class.

Here we have a class with a method returning an instance of the class (note that isinstance(MyEnum.FOO, MyEnum) holds true) and Self is not allowed as annotation.

I’m surprised there’s any debate here, this seems like a clear typing bug just by considering the end-user experience (whether it’s an issue with a spec or in individual implementations, and regardless of how much special handling enums already require).

Your suggestion was to switch to using the class name as the annotation (which currently requires stringifying or from __future__ import annotations). But what does this annotation mean, doesn’t that mean that the method returns an instance of the class? Isn’t that exactly what the Self annotation is intended to indicate?

[LewisGaul]

Interestingly this seems fine in VSCode (Pyright?):

class Mode(enum.Enum):
    FULL = "full"
    MINIMAL = "minimal"

    @classmethod
    def default(cls) -> Self:
        return cls("full")

This seems like a better workaround than using the class name as an annotation, but it’s not ideal needing to use the initialiser like this rather than the cls.FULL form.

[beauxq (Doug Hoskisson)]

@LewisGaul

The first line of PEP 673 is:

This PEP introduces a simple and intuitive way to annotate methods that return an instance of their class.

This quote alone doesn’t make it clear enough which class it’s talking about.
It’s not talking about the class that the method is defined in, it’s talking about the class of the object that the method is called on, which in most cases, is not guaranteed to be the same class.

Self is not allowed as annotation.

I hope you noticed here Enum classes and Self - #10 by LewisGaul that this statement is wrong. It’s not about whether Self is allowed. It’s about which class the Self is referring to.