No, I never suggested that anything like that was true in Python today. Are you referring to the idea’s text, or some claim I made about Python?
Here’s what I’ve already said in the thread.
Subclassing is the idiomatic way to extend the behavior of a class while maintaining the is-a relationship, i.e., the ability to pass an object with extended behavior to functions that accept instances of that class.
Therefore, there are very few good reasons to prevent people from subclassing. This is the consenting adults principle. Just like we don’t block people from assigning to constant members—even if the class wasn’t designed to change those members, we also don’t block people from inheriting except in extraordinary cases.
The idea in the OP repairs some of the problems with subclassing so that it is easier to do, and with fewer pitfalls. This would mean that it would be even easier for subclasses to just work without needing enormous cooperation between superclass and subclass.
But your proposal already requires superclasses to be rewritten to use your new proposal - this already is a significant cost of the idea and means that it wont “just work”. And it still requires people to keep this in mind in situations like the one described by @oscarbenjamin above since those kind of changes will break subclasses.
So the situation continues to be that people need to make a conscious decision to support subclassing as something they are going to keep in mind when writing the code. You need to make a convincing argument that given this decision by the author, the new proposal is a significant improvement. To me it looks like a side-grade. Slightly more complex and magical code[1] with a few benefits, less repetition and slightly better init-focused MI behavior, at the cost of clarity, flexibility and familiarity. [2]
Your idea of it “just working” is never going to happen within Python - python is too flexible of a language for this with too many ways to break this in new and exciting ways.
Ignoring the potential performance pitfalls I am seeing in your proposal - suddenly way more objects are created during the process of a simple dataclass init. ↩︎
I don’t want to put words in Oscar’s mouth (or anyone else’s), but the contrary point of view that I see being expressed here is “You should never assume you can pass anything to functions that accept instances of class C except instances of exactly class C”. Or again, you should never assume you can create a new class with an is-a relationship to another class.
As for myself, I don’t really see how your last sentence I quoted follows from what precedes it. You keep talking about the LSP and the is-a relationship, but the question for me is why. Why do you (or why would anyone) want an is-a relationship in the first place? This isn’t to say that one is never wanted, but I think it makes more sense to consider why and when it is wanted rather than just starting from the premise that allowing is-a subclassing of anything anytime is automatically desirable.
I’m not suggesting that anything be rewritten. But yes, the benefits are only available if you use the feature.
I don’t think this is a particularly noteworthy issue. In general, the kinds of overriding that people tend to do is either:
abstract methods that are implemented by one derived class
augmenting methods that are implemented by many derived classes, wherein each implementation calls super
Overriding a non-abstract method is already something that tends to be rare and demand a lot of thought. I don’t think this is something that you need to worry about.
I’m not suggesting that anyone do that. Just like you don’t worry about whether someone will edit your constants.
Inheritance isn’t about “avoiding repetition”.
Okay, is that what you believe?
Because no static type checker even has the option to enforce that.
I mean this is the sort of thing you would have learned in school, or from any objected oriented programming textbook. It seems odd that I presented an idea about how to make inheritance easier, and the questions in the thread are about “but why do we need inheritance at all?” I don’t want to spend too much time recapping the basics of objected oriented programming, and I don’t think you find it all that interesting either? Surely, there are thousands of better sources that you could read for that information?
Here’s a famous one:
Gamma, Erich, et al. “Design patterns: Abstraction and reuse of object-oriented design.” ECOOP’93—Object-Oriented Programming: 7th European Conference Kaiserslautern, Germany, July 26–30, 1993 Proceedings 7 . Springer Berlin Heidelberg, 1993.
Here’s another.
Weisfeld, Matt. The object-oriented thought process . Pearson Education, 2008.
When I worked at Google, they gave us a textbook on OO on our first day.
I never said anything like that.
Also, I don’t think it makes for a very productive discussion to recast what someone is saying hyperbolically and then argue that it’s ridiculous. It would be better if you just quote me directly if you want to respond to something I said than this kind of loose “paraphrasing”.
Yes, sure, and there’s no way to guarantee that. The best you can do is to add it as a requirement in the base class’s docstring and hope that people follow it.
Well, recasting our criticisms as asking “why do need inherticance at all” also isn’t fair.
My point is “why do we need to make inheritance even easier?”
It often isn’t a great tool in general. OOP has been overused, there is a reason more modern languages tend to shy away from it.
Python is a multi-paradigmen language, OOP is not at it’s heart more than functional Programming is at it’s heart. (i.e. the central requirements for both are full filled, it’s just style which one you prefer for the most part.)
In python, LSP is not enforced nor enforcable, for various reasons. Ofcourse, it is a pretty central property, but it is not a fundamental requirement for making inheritance useful (despite what you are claiming, sometimes inheritance is about code reuse. It’s a tool that can be used for more than one thing.)
In python, there are central abilities of types that are incompatible with the idea of LSP - most notably hashability.
Inheritance is already pretty easy. It is also decently easy to write and document superclasses that can be inherit from easily, safely and with predicable behavior. It just requires “consenting adults” working together through documentation, and yes, sometimes mistake happen.
I am deliberately ignoring a few of the more rude parts of your latest comment. If you are getting frustrated, step back and come back to this later and reread the TOS. I am also going to go to bed now ↩︎
I’m sorry, are we looking at the same text? This is what I responded to:
An “is-a” relationship is by definition inheritance. So, I don’t see how this is an “unfair” reading of what is being asked.
I explained that in the OP.
Yes, it’s often overused. It can also be underused (situations where people avoid it when it is the best candidate). I don’t see how this is an argument against making it easier to do.
I disagree with that. I would say that idiomatic Python is imperative rather than functional, but do as you like.
I don’t know what “making inheritance useful” means. LSP is a fundamental property of inheritance after all. The whole point of the is-a relationship means that you can use subclasses in places where you use superclasses. That is, by definition, what LSP verifies.
If it’s just about code reuse and not about modeling is-a relationships, then I suggest that your suggestion is an example of inheritance overuse. You should prefer composition for this case. I don’t agree that this is “just a tool” and that “it can be used for more than one thing”. Here’s a discussion that says exactly what I’m saying, and here’s a SO answer. This is a classic topic in OO theory.
I don’t see how it’s “incompatible”. Do you want elaborate?
Sure, it’s pretty easy except for the problems that I identified in the OP. But I agree that Python doesn’t do too bad a job as an OO programming language as it already stands.
Well, things may have drifted a bit in the discussion. But let’s go back to your original post:
I don’t see that this a problem. In particular, the second point is easy to avoid if, as has been suggested multiple times in this thread, you don’t attempt to subclass without knowing everything about the class hierarchy you’re subclassing from. Then you just choose parameter names that don’t conflict.
The post also makes several references to the LSP as if it is self-evident that getting closer and closer to “complete” satisfaction of the LSP is a good thing, and several of your later posts repeat that LSP is “fundamental”. I don’t see it that way. I view the LSP as a sort of rule of thumb that guides us in designing classes, not an inviolable dogma. The mere fact that something (like __init__ or __replace__) doesn’t satisfy the LSP is not in itself evidence that any problem exists.
This is why I’m getting the vibe that you want “totally free subclassing” (along the lines of what you quoted me saying above). Because if you don’t want that, the current situation is fine: when subclassing, a Python user has to either accept that they may violate the LSP, or they have to check that the particular way they’re subclassing will not violate the LSP. All is well.
Well, there’s a lot more than that that can’t be guaranteed. You can reassign __class__ at runtime.
At one point in a later post you said:
Then later you said:
How are these two cases different? If someone forgot to call super when they were supposed to. . . that’s a bug, right? So why do we need any more complications than people just calling super when they are supposed to (which may include knowing the class hierarchy to know when they are supposed to)?
On rereading a few posts in the thread, I saw a few references to static typing as well, which made me realize I was perhaps confused. Your original post makes no reference to static typing (other than I guess saying constructors return Self), but then in a later post you seem to indicate that your proposal could already be implemented with metaclasses except for static typing. Is static typing support an integral part of the desiderata of your proposal?
EDIT: In particular this, which I somehow couldn’t find when writing this post:
It’s a bit more extreme than what I believe, as I mentioned in another post. And yes, no static type checker can do it, and that’s fine, because I wasn’t making any claims about static type checking and don’t care about static type checking. The text you quoted above is just my half-baked statement of a rule of thumb that Python users might keep in mind when considering creating or using a subclass of some other class.
Ok, one last comment, then I am probably bowing out of this topic for good: I do not care for theory. I want to write actual programs for my usecases.
In python “class” and “inheritance” are very versatile tools that can be used for a variety of different things. To name stdlib uses:
builtin datatypes
OOP-java style classes
OOP-smalltalk style classes (which includes no inheritance!)
composition based OOP
ABC
Protocols
MixIns/Addons
dataclasses
TypedDict
Generic
NamedTuple
Enum
custom data containers, e.g. range
stateful iterators, e.g. list_iterator
Even if you don’t want it to be, what python calls “inheritance” is actually just a convenient syntax sugar that can be used for all sorts of interesting stuff. Your narrow java-OOP based view is just one of many usecases, please don’t call others incorrect for using a tool differently than your world view allows.
I tried to explain (in the part of my post after what you quoted) what I meant by this, but looking back I see I probably was still unclear. “Why would anyone want this” was a poor phrasing because it sounds like I mean it’s something that would never be wanted. But what I mean is that it is not automatically something that is always wanted. That is, in each specific case where someone is considering subclassing, it behooves them to think about whether they specifically need an is-a relationship, and whether they specifically need it to strictly satisfy the LSP, and so on. That is why I’m not convinced by general arguments to the effect of “the LSP is fundamental”. My question is basically “what specific use cases are not currently doable but would be possible with some changes”? Or is it only that they cannot be currently done in a way that statically guarantees the LSP?
This is not reasonable to me. The theory is based on “writing actual programs” in a way that is clear to readers, and easy for writers. Theory is, by definition, the underpinning of good practice.
What you are postulating in this thread is your own theory. And of course, you’re free to do that, but I don’t think it has much value when put alongside the theory that already exists in the field of computer science, and has existed for literally decades.
Yeah, I think you’re not going to find answers you like if you reject orthodox theory that doesn’t match your intuitions.
You’re also going to have trouble being convincing when you don’t even know orthodox theory. It would have been different if you had said: “I know what the experts think, and I know why they think that, but I have a different viewpoint because…” But that’s not what you’re saying. You’re saying “I don’t know what the experts think, but I have arrived at my own conclusions”. And I think that comes across as extremely unconvincing. Like walking in to a physics lecture and sharing your heterodox “thoughts” without knowing their orthodox theories.
Yes, you can do anything you like, but using inheritance to model non-is-a relationships is neither idiomatic nor good code—according to experts. (I understand that you don’t care about experts.)
My viewpoint has nothing to do with “Java”. Did you look at my citations?
I’m saying that it’s incorrect within the established theory of OO programming. (You are free to reject that theory and do as you like.)
I don’t want to get too far into this kind of debate, but a simple way to characterize my response to this is: Python already deviates a fair amount from a strict interpretation of the established theory of OO programming. So there is necessarily a gap between talking about OO theory and talking about writing actual code in Python.
No, I don’t think it does. And again, if you want to argue that inheritance in Python should not respect OO theory, then I think you should still learn OO theory, learn its motivations, and then make an educated case about why the reasons don’t apply here.
I remember watching a talk by Brandon Rhodes I think where he talked about design patterns, and the gang of four book came up a lot. The gist is that the design patterns presented in that book came into place because C++/Java style inheritance is not that great (paraphrasing) and many of the patterns are either built into Python or can be represented better in other ways. I think this is why Cornelius says that you focus on Java-style inheritance when you cite a book on that subject. There’s nothing inherently wrong with that style of inheritance, but as was said earlier many newer languages go for traits/dataclasses instead, rejecting OOP entirely sometimes.
Also, I think that while inheritance is the canonical way you’re taught is-a relationships, the actual way it’s done in practice in Python is through duck typing, now formalized through Protocols. I’d argue that if you don’t have a good reason to do normal inheritance (which I find is rare, duck-typing and composition works wonders and removes all inheritance worries) you should use Protocols instead.
Lastly, one thing that I haven’t seen brought up in this thread is that instead of calling super().__init__ I was taught that you almost always actually want to call the explicit constructors of the superclasses. To me that seems like the best way to get around the quirks of super, but maybe there’s something I misunderstand?
In today’s Python? You shouldn’t do this, and I explained why here:
But if you mean as a solution to the problems of cooperative MI, then I think that may actually be the way to go. It’s probably a nonstarter since it’s so far from today’s Python though, and people don’t seem to think the problems justify such a significant cure.
I’d go the other way. Protocols should be reserved for when it’s too onerous to force derived class to inherit from a common parent. For example, SupportsInt is prevalent, but it’s too much of a pain to force everyone who is SupportsInt to inherit from it.
Protocols:
Make isinstance and issubclass both brittle and slow.
They can never add data members.
They can’t add behaviour.
Rather than explicitly declaring X < Protocol, you have to assert_type, which is awkward, and produces poor errors if it’s not. (Just says no—doesn’t tell you why.)
The advantage to protocols:
They don’t force derived classes to change anything in order to support them.
They don’t add to the MRO, which may make certain lookups very slightly faster.
Protocols were a great idea, but just like inheritance, they can be overused.
I am under the impression this is exactly “today’s Python”. The super() documentation makes this explicit:
[…] Good design dictates that such implementations have the same calling signature in every case […]
The problematic cases of __init__() that you are describing do not have the same calling signature in every case, so diamond-pattern super() does not apply. (And I would also have called the explicit __init__() functions of all superclasses, today.)
I had read that comment about explicitly calling superclass constructors bit forgot about it, thanks for pointing to it.
Regarding your list of disadvantages of protocols:
I think we’re thinking of these things very differently (if that wasn’t obvious already :)). I disagree that points 1-3 are bad in general (in certain circumstances maybe but typically no) and I don’t understand point 4, could you elaborate a bit?
Right, that’s how it is today. But that limitation essentially forces classes that want to participate in cooperative MI to accept a **kwargs: Any argument so that they do have compatible signatures.
The proposal essentially swaps that for calling explicit parents, which liberates the init from needing kwargs and forwarding everywhere. It would be closer to C++'s style.
If you want to guarantee that X < Y, with inheritance, you do:
class X(Y): ...
With protocols, you do:
class X: ...
assert_type(X, Y) # Or something like that, I rarely use protocols
If the assertion fails, it will just say it failed, but won’t tell you why. In the inheritance case, you would get a clear reason like “method incompatible with superclass definition”, etc.
