Unbound __init__ of subscripted generic base class - CPython bug?

I wish to call a method of a generic base class in a way that passes strict type checking.

The following code runs fine, but Pyright strict mode reports an error:


from typing import Generic, TypeVar

T = TypeVar("T")

class A(Generic[T]):
    def __init__(self, x:T):
        self.x = x

class B(A[int]):
    def __init__(self, x:int):
        # Pyright strict mode reports: Type of "__init__" is partially unknown
        A.__init__(self, x) 

I had a discussion with the Pyright author, who suggests the subscripting the base class i.e.

class B(A[int]):
   def __init__(self, x:int):
        A[int].__init__(self, x) 

This is now reported correct, but at runtime A.__init__ isn’t called! A[int].__init__ seems to silently do nothing. I’m using Python 3.11.7. Could this be a bug?

For context, the reason that I want to explicitly call the base class rather than use super() to achieve type checking with multiple inheritance. The standard trick of passing along extra arguments to the next class in the MRO using **kwargs means forgoing type checking on the second and subsequent base classes.

Taking a look at A[int].__init__:

>>> print(A[int].__init__)
<bound method _GenericAlias.__init__ of __main__.A[int]>

which is indeed not related to A.__init__, nor calling it at any point, contrary to “normal” methods

>>> A[int].foo
<function A.foo at 0x...>
>>> A[int].foo is A.foo
True

Testing a few examples, the same behavior applies to __str__, __eq__ and some other – certainly the methods defined on Generic. I suppose that’s by design?


>>> inspect.signature(A[int].__init__)
<Signature (origin, args, *, inst=True, name=None)>

Looks like this method fortunately wants two positional arguments, so does A.__init__, so it just “fails silently” – but that would not have been the case without your x parameter!

Thank you for the investigation. That’s interesting about the magic methods. I tried to find out if this is by design. I found nothing definitive, but the acceptance of this pull request suggests that it is not. From the description, it seems as if the intention is for subscripted generics to function as proxies at run time.

I don’t think you should ever do this. Call super().__init__ or else your class won’t work if someone inherits from it. Unless your class is final, you have no way of knowing that A is the superclass.

It always is a superclass, but it’s true that we don’t know that A is next in the MRO (is that what you meant?). However, for my problem this doesn’t matter, because I do not use duplicate inheritance of base classes. (I use Protocol and structural subtyping to define interfaces, which has largely obviating my need for duplicate inheritance.) The absence of repeated base classes means that the superclass graph of any given class is a statically-defined tree. Each __init__ can call the __init__ of its base classes without risk of breaking the C3 rules. There is no need for super() i.e. for dynamic C3 linearization. I find this restriction makes the code more comprehensible while still allowing for multiple inheritance. Another advantage is that it allows for static type checking of the arguments to all base class __init__ methods, not just the next one in the MRO.

This does feel like a bug in the runtime. I haven’t looked at the implementation recently, but this may not be easy to fix.

After looking at _GenericAlias implementation, the “shadowing” methods:

  • Magic: __init__, __eq__, __hash__, __or__, __ror__, __getitem__, __repr__, __reduce__, __mro_entries__, __iter__, __call__, __getattr__, __setattr__, __instancecheck__, __subclasscheck__, __dir__
  • “Private”: _determine_new_args, _make_substitution
  • “Public”: copy_with

I see no easy fix for this, indeed. A comment states

[…] for simplicity we don’t relay any dunder names

:sweat_smile:

By “duplicate inheritance”, I think you mean diamond patterns?

If you allow multiple inheritance then not calling super can still break code—even if you avoid diamond patterns.

class B:
  def __init__(self):
    pass
class A(B, C):
  def __init__(self):
    B.__init__(self)  # C's init is not called!

I think you should just call super.

By “duplicate inheritance”, I think you mean diamond patterns?

I mean where the (undirected) inheritance graph is cyclic. Most writeups of the diamond pattern refer to the simplest case (where the base class is a grandchild of the duplicated class). I wanted to be more general.

In your example, I would just call C.__init__(self) afterwards.

To use super I’d have to put it in B.__init__. To pass values from A.__init__ to C.__init__, I’d have to route them through B.__init__ using **kwargs , like this.

class B:
  def __init__(self, **kwargs):
    super().__init__(**kwargs)

class C:
  def __init__(self, x):
      self.x = 1

class A(B, C):
  def __init__(self):
   super().__init__(x=1)

This works, but now we’ve lost static typing of the arguments passed from A.__init__ to C.__init__. Contrast this with

class B:
  def __init__(self):
    ...

class C:
  def __init__(self, x):
      self.x = 1

class A(B, C):
  def __init__(self):
   B.__init__(self)
   C.__init__(self, 1)

Both __init__ calls are now amenable to type checking.

Sorry, I don’t see the difference. How can you have a cycle without a diamond pattern?

Let me also say that I love type annotations too, and it’s a real joy when everything is type checked.
However, in this case, you’ve chosen to write code that has stricter type checking rather than idiomatic code that:

  • is understandable to other Python programmers and
  • works with multiple inheritance.

In my opinion, the latter is better. As for the type checking, I proposed SuperKwargs to deal with this one day.

How can you have a cycle without a diamond pattern?

If the duplicated base class is a great-grandparent, then some texts would call that a diamond pattern, but others not. I just wanted to use unambiguous terminology - I think we understand each other :wink:

I agree that super() & co-operative inheritance have the advantages you mentioned. Thanks for pointing me to SuperKwargs. It certainly looks interesting. For me, the hope of something like that coming along one day swings the balance in favor of super(), so I’ve decided to switch back to using it.

1 Like