The problems you are referring to are all to do with inheritance and multiple inheritance. My response above was to you wondering if you know what the signature of super().__init__(...) should be. Speaking as someone who does not generally use multiple inheritance the answer to that question seems obvious to me: just look at the __init__ method in the parent class.
To be in a situation where you don’t know the signature of super().__init__() you have to suppose that downstream code might subclass your class using multiple inheritance. I don’t see why it is necessary to worry about that and I would not contort any of my own code to try to support it.
Earlier I asked this question and you answered:
Only the last point is not related to inheritance and multiple inheritance.
Since this is all about inheritance then I think it is worth spelling out what are the reasonable designs for inheritance that are actually worth supporting. There are many ways to make these things work by imposing constraints in a given class or class hierarchy but what there is not is a fully general subclassing design that can allow any arbitrary classes to be meaningfully combined with multiple inheritance.
I think you have a fundamental misunderstanding of Python’s multiple inheritance here. Multiple inheritance in Python is co-operative, meaning that users aren’t free to inherit from what they want. Much of the time it’s fine, but you should only multiply inherit within a class structure where all classes are aware of the other members. The behaviour of __init__ is one example where this is the case. Of course, you can propose to change Python’s multiple inheritance rules to not require co-operation, but it’s a change to the design, not just some bug fixes.
To be clear, the above is all about multiple inheritance - single inheritance doesn’t have the complexities that require the co-operative model.
Also, for what it’s worth, I’m not interested in theoretical debates about whether co-operative multiple inheritance is “real” MI, or whether you “have to” behave in a certain way in a MI system. If people want to have that sort of debate, that’s fine but I won’t get involved.
I have a similar opinion about inheritance as other people with detracting arguments for this.
Allowing arbitrary non-cooperative subclasses means that every addition, even by convention private ones (single prefix underscore), is breaking. This isn’t a useful definition of breaking, and it only comes about through what I view as irresponsible subclassing. Subclassing, with multiple inheritances or not, should only happen cooperatively, and any conflict with future additions that come from not using it cooperatively lies solely with the person who made a subclass of something not truly designed for it.
While I see the problem in the type system this is meant to address, I’d rather not introduce new syntax for a type system hole that is as narrow as that.
Can you point to any documentation supporting this contention? I don’t believe there is any such rule.
The whole point of cooperative multiple inheritance is that super-calls ensure that all classes are initialized regardless of whether they “know about each other”. That’s what I think cooperation refers to: the __init__ methods cooperate to do the initialization.
For example, there’s nothing wrong with this:
class A: ...
class B(collections.abc.Iterable[T], A): ...
Clearly, Iterable isn’t aware of A, but I think it would be a design error if this didn’t work.
iterable is an abc, which is explicitly for implementing via subclassing, and there are both implicit and explicit promises attached to it.
A better example would be me taking a random lock class from another library, and then subclassing it. There’s no reason for the developer of that library to consider (or have to consider) that someone else would do such a thing, and any problems that creates would be solely of my own making, not of the library developer for not considering that usage.
Iterable is a bad example for whatever point you are trying to prove:
it isn’t a normal class, it’s an ABC (essentially a Protocol, but predates that concept getting added to the language), which are inherently design to be inheritance, potentially in a MI way with all other subclasses of object[1]
It doesn’t initialize anything, it doesn’t have an __init__ defined. [2]
It doesn’t have any state, or even any meaningful methods made available.
as long as those other subclasses don’t use a conflicting metaclass ↩︎
the behavior of disallowing instantiation actually comes from object.__new__/object.__init__↩︎
Iterable is an example of a class that participates in multiple inheritance without knowing about the other base classes. That’s what I set out to illustrate. It is an example that disagrees with the contention that base classes must be “aware of each other”. Clearly they do not need to be aware of each other in all cases.
It’s not “cheating” just because it’s an ABC. Yes, an ABC is an interface by definition. Yes, it needs to support multiple inheritance. That’s why I chose it as an example.
Yes, they don’t have state. However, if we were to add state to an ABC (for example, to do some kind of logging), then we would simply do the forwarding. This pattern of forwarding parameters in __init__ is ubiquitous in Python’s ecosystem.
A couple random examples:
_ProtocolMeta does for example to ensure it can work with metaclasses that it doesn’t know about, and
Scipy’s new DistributionInfrastructure forwards parameters after doing some initialization.
No, there is nothing wrong with that. I would consider this different from multiple inheritance with concrete classes especially for Iterable since it provides nothing but a single abstract method although some ABCs work more like mixins.
One thing worth noting though is that using these ABCs, mixins or even just the typing things like Generic does actually create a multiple inheritance style MRO so it does bring in some of the Python-specific considerations around multiple inheritance.
There was a bug in SymPy where __init_subclass__ was defined for Expr but without calling super().__init_subclass__. Someone downstream did something like
class MyThing(Generic[T], sympy.Expr):
...
and this actually broke because then you have two bases that define __init_subclass__ and that need to call each other cooperatively. It would be better if these typing things did not insert actual runtime classes into the MRO but apparently that still happens even with the 3.12 syntax:
In [1]: class A: pass
In [2]: class B[T](A): pass
In [3]: B.mro()
Out[3]: [__main__.B, __main__.A, typing.Generic, object]
In [4]: B.__bases__
Out[4]: (__main__.A, typing.Generic)
Another way to look at it is that Iterable does know about all other objects. It has been designed to be compatible with object and all of its subclasses.
They need to be aware of each other in the generic sense that they need to be aware that they are participating in some inheritance structure that has some rules and may or may not involve multiple inheritance. There needs to be conventions for whether particular base class methods should call each other cooperatively or whether a given class can assume to be the provider of a method etc. All classes involved need to be designed with these conventions in mind.
In the case of a stateless ABC or mixin the convention is that if the class provides any non-abstract methods then those are expected to be either identical or orthogonal to methods provided by other bases so that there is no diamond for any particular method and no expectation of cooperatively calling each other with super(). Constraining base classes this way provides one sensible scheme for multiple inheritance.
Right, __init_subclass__ is another method that follows the augmenting pattern. It should always call super IMO. Not only do you break MI if you do, you break single inheritance too.
Seems to me like a bug in whoever forgot to call super.
Sure, and this is what I’m arguing for. In general, if you want your class to be able to participate in MI, then you should delegate to super. This is the state of Python today. There is no requirement that classes be explicitly aware of each other.
Right, that’s what I’m arguing for. The “awareness” is not some explicit awareness. It’s a just blind delegation to support a broad range of MI patterns in case a user wants to do that.
And if you do this blind delegation, then users are fairly free to inherit from your class.
Typically, it is clear who is the “root” of any particular method: object provides __init__, and type provides __init_subclass__. Another augmenting pattern is AbstractContextManager , which provides __enter__ and __exit__. The reason that it provides these is so that child classes can implement them, and then delegate to super. This way, you can use multiple inheritance to combine context managers if you want. In general, when you want to start this “augmenting pattern” on a method, you provide the stub in the base class.
It’s not that they’re “non-abstract”. It’s that they’re non-augmenting. The augmenting pattern exists for a handful of methods in Python (I’ve mentioned 4 in this comment), and for that pattern, you should always delegate to super.
Yes, but we are arguing that not all classes should generally be compatible with all other classes. Classes are free to define a subset of usecases they are explicitly supporting, beyond that, it’s “undefined behavior”.
ABCs by design support all subclasses of object[1]. This is an exceptional case, and most classes are not as compatible, nor is there a fundamental need for this to be the case. If you don’t agree with this, please provide concrete arguments for why, preferably with actual examples from real world code were classes not being compatible outside of design considerations was an issue that prevented a nice solution from working.
If you do agree that classes are free to define subsets of the global type hierarchy they are compatible with, and still are of the opinion that something can be improved here, please make it clearer what usecases you want to improve.
However, note that even if you call super, you can’t easily support arbitrary MI. If two different superclasses __init_subclass__ need the same keyword argument, it is not possible to provide it without breaking in some situations since object.__init_subclass__ expects no keyword arguments (ofcourse, this could have it’s behavior changed, but that would reduce error catching or introduce more complex edge cases).
I did, I am just not sure it actually helps with the issues you are outlining. Your proposal is to create an entirely new category of classes, c_classes, which are presumably incompatible with most normal classes and which have improved compatibility with each other?
I mean, sure, but I believe that there are already enough existing mechanisms to reduce the pain if all base classes agree and that adding yet another way is not a benefit. Instead energy is IMO better spend trying to improve the existing situations with better tooling and typing support.
I don’t think that you can make these things work usefully in general by supporting a “broad range of MI patterns”.
Also allowing users to freely inherit your classes may be a goal that you aspire to but it is not actually something that I want users to do in general. Subclassing across code boundaries creates very strong coupling and can only work properly with a particular design for how the subclassing is supposed to work.
I’ll give a simple example for why I don’t want this but in general there are many more reasons along these lines. Suppose I have this class:
So I have added a new method h and I have changed f so that it now calls h instead of g. Generally adding a new method like this is not considered a breaking change and the public interface of all methods still returns the same result so this should be a compatible change.
Now suppose that someone downstream has done:
class Down(Up):
def g(self): # Override Up.g
return 3
d = Down()
print(d.f())
Now my change to Up that seemed compatible is going to break them. They have managed to depend not just on the public API of the Up class but also on every piece of internal code that calls its g method. Making this subclass was a huge overreach beyond using what I would consider to be the public API. I don’t even want to have to think about whether this kind of change will break someone so I don’t want them to subclass my class.
If I had documented that Up can be subclassed and that someone can override g and expect that to affect f then the situation is different. This is what I mean by having a “subclassing design”. Subclassing only works if there is a contract about which classes can be subclassed, which methods can be overridden, what effects that will have etc. Without that as far as I am concerned the downstream people should consider Up to be final. They can subclass if they want just like they can monkeypatch Up.g but as Mike says any breakage is on them.
But the problem isn’t that someone inherited from your class. The problem is that their override did something you never promised to support. That’s okay. User beware.
You don’t need to do very much to allow inheritance. Just make sure to delegate in all augmenting methods. And you’re free not to do that if you don’t feel like it.
Personally, yes, I think it’s good practice unless you have a good reason not to. I don’t think “someone might create a bug in their own code when they inherit” is a good reason.
I think the essence of the point being made here is that, by default, there are no promises to support any kind of override, so by default subclassing is not supported at all. It only when explicit guidance is given in the docs (e.g., “users can/should subclass MyClass and override my_method in order to…”) that users should expect to be able to subclass anything whatsoever. This is in opposition to your view that (as I understand it) it would be better if people could assume they could subclass any class.
Personally, I’m not sure I agree with either position at the extreme, although I’m more towards the “subclassing is by default unsupported” end.