A little background - “Extension methods” is a .NET feature. Introduced with C# version 3 in November 2007, it is essentially syntactic sugar that enables turning the following code:
namespace NS
{
public static class Cls
{
public static void SomeMethod(int i, string j)
{
Console.WriteLine("{0}, {1}", i, j);
}
public static void Main(string[] args)
{
int num = 3;
Console.WriteLine(SomeMethod(num, "s"));
}
}
}
To this:
namespace NS
{
public static class Cls
{
public static void SomeExtensionMethod(this int i, string j)
{
Console.WriteLine("{0}, {1}", i, j);
}
public static void Main(string[] args)
{
int num = 3;
Console.WriteLine(num.SomeExtensionMethod("s"));
}
}
}
Which makes it look like you “extended” the int class with a custom method (even though in reality the method is completely external to the int class), hence the name “Extension methods”.
Personally, I think that extension methods will be a great addition to Python, and I was somewhat disappointed every time I felt that I needed this feature and it wasn’t there. I’ve tried searching in the PEP index in the past to see if maybe this feature was suggested and turned down, but this didn’t seem like the case. So I want to write a PEP to suggest this feature, but I would like some community feedback before I do.
Why I like this feature
It’s convenient - I work with researchers and they appreciate the ability to discover new functions of the library using autocomplete/IntelliSense, and it can look nicer for some applications.
I don’t want to break the language to achieve the same goal - I saw people equating this feature to monkey-patching, and it is not that - the original .NET feature states that a class function will always take precedence on an extension method, and that’s the correct behavior in my opinion.
It works in a static context - while I can just add a function on any instance I want, for static type-checking I won’t be able to see those functions, which is kind of annoying.
I don’t need to create a child class - I don’t think anyone wants to inherit from list or int, and it will be a headache to try and use a custom class for those types because every time you’ll want to use a number of a list you will have to wrap them in a custom class, and it won’t work inside other libraries.
Common criticism against this feature
Somewhat implicit - it could be more difficult to track down whether a function that seems to be “of an object” is actually defined on an object (or one of its parent classes) or if it is an extension method.
Somewhat unnecessary - as mentioned, it is just syntactic sugar for a function class so it isn’t strictly needed.
Not pythonic - this is somewhat subjective, but it might just not fit the language and I can understand that.
There is a thread on the old forum (linked here) on this topic, many of the responses there seemed a bit confusing to me. The reason I decided to post this topic again is because I think this is a good feature, and I want a definitive answer regarding it - even if it a rejected PEP.
It’s called monkeypatching, and you can only do it on user-defined types in Python - a necessary limitation of the built-in types being implemented on a different level.
class ToPatch:
pass
def SomeExtensionMethod(self, j):
print(f"{self}, {j}")
# this is the monkeypatch / method extension-ing
ToPatch.SomeExtensionMethod = SomeExtensionMethod
C# is a statically compiled language. While it has some dynamic features, its compiler knows exactly what is going it in cases such as your example. Features based on similar features in statically compiled languages often can’t be implemented with the same ergonomics in Python, and attempts to do so result in bad features.
The reason extension methods work in C# is that they are scoped – integers appear to have the extra method, but only in parts of the program where the extension method is in scope, visible to the compiler and the user alike. This does not translate well to Python, where the “compiler” (by design of the language) doesn’t know such things, even if it seems clear to the user.
I think it’s unreasonable to expect a PEP to be written just to explain why a misguided proposal is misguided. I am sure others will jump on this, but I think you should just accecpt that this particular idea does not translate well to Python.
(Some dynamic languages like Ruby allow user to extend built-in classes, but this is something we don’t want in Python. It would be quite confusing for a user to find that integers suddenly have a “spam” method because some library had a need for that. There is widespread agreement about this.)
That’s an implementation option, but there are issues with this. Mainly, you can override existing “native” functions this way, which is something I want to avoid.
class B:
def g(self, j: int):
return j + 5
b = B()
B.g = lambda self, j: j + 3
print(B.g(6)) # prints 9, with a proper extension function it would print 11 because the original function takes precedence
To your first point, I can understand that but I still think there can be a good way to introduce this feature. When compiling C# to CIL and back, you’ll see that the extension method call is converted to a regular function call, so it is just syntactic sugar. The reason why I would rather see it as a part of the language is because this way it wouldn’t require me to “break” the language to achieve the same result.
As for the scoping issue, in C# the extension methods can only be accessed in files where the namespace of the extensions is imported with a using statement, and I think that a similar scoping mechanism can be used here - so to use an extension method you would have to import the function to the file you are calling it from.
And to your final point - a post from a community member describing the idea in a searchable way (that got an answer from one of the core members of the dev team) is a good official response in my book. I had a hard time finding real opposition to this feature as there was no rejected PEP or recent forum post (and as I’ve mentioned, the old one was a bit confusing to me), so a comment from a core dev is awesome, and I appreciate it. Thank you!
(And I completely agree that “spam” methods on builtin types would be annoying as heck )
Perhaps extending classes requires the introduction of an additional extension mechanism, which is different from inheritance. The extension should result in another class that is equivalent to the original one (it is not strong subclass). Perhaps a closer concept might be a category from the Objective-C language
The . operator currently has a clear meaning: it calls getattr(obj, attr_name) on the object on the left, which doesn’t depend on anything else of the surrounding context. How would you introduce this syntax sugar in a way that wouldn’t break this definition or modify builtin types? If you have a simple solution to this, describe it.
Just saying “I still think there can be a good way to introduce this feature” isn’t gonna do anything to convince anyone. People have thought about it and haven’t come up with a good solution.
If you really, really want to you can already curse the object.__getattr__ implementation to look at the scope where the attribute is accessed from.
And even if there was an ok solution, I don’t really think obj.some_extension_method(*args) is that much better than some_extension_method(obj, *args). As you said, it would just be syntax sugar. It would have to be a zero cost solution to even be considered IMO, and I am basically sure that that is impossible without being statically compiled.
Inheritance in Python might be more powerful than in other languages, since I’m not seeing the limitation here - it should be entirely possible to add these methods with a simple mixin!
In [1]: class MyInt(int):
...: def something_new(self):
...: print(f'something new for {self}')
...:
In [2]: x = MyInt(2)
In [3]: x.something_new()
something new for 2
I think the problem is that functions accepting MyInts can’t accept ints, from a typing perspective at least. You also have the overhead of constructing the MyInt compared to constructing regular int. But I don’t think those problems warrant extension methods since using regular functions do the same (except the discoverability part but tbh that’s what docs are for)
As mentioned before, the C# extensions methods apply only in contexts in which the extension is known.
Without an practical application context or use case this kind of proposals turn into a “nice to have” request.
In TatSu there’s inheritance from list just to distinguish the creation context (to have a different type), and there’s inheritance from dict to provide features that became available only in recent versions of Python.
I don’t think that the OP’s suggestion over int makes for a good example about usefulness, because, as you say, it’s too easy to write SomeExtensionMethod(num, "s") and use it the context in which it’s required.
Python programming is multi-paradigm, unlike languages in which every solution must look object-oriented.
I think this feature – extending existing classes or types with new methods – is not just syntactic sugar or just a convenience. In languages like Rust that do support this (in Rust by using traits), it allows a different form of API design. For a very nice intro, see for instance: “Type-Driven API Design in Rust” by Will Crichton (https://www.youtube.com/watch?v=bnnacleqg6k).
I can see why in a non statically typed language like Python (with a very different data model) it’s not really possible or desirable to support this, but I kind of regret that.
Traits in are essentially the same as typeclasses in Haskell IIUC, which you could possibly implement in Python by having a class represent the trait, and then implement the trait specializations/methods (I don’t know the proper terminology) as staticmethods with multiple dispatch on the trait object. Not sure how it’d work in practice and you don’t get the rust thing where you can access trait methods through the dot operator.
Isn’t it easily implementable with operator overloading?
class some_extension_method:
def __init__(self, j):
self.j = j
def __rmatmul__(self, real_self):
print(f'{real_self}{self.j}')
num = 5
num@some_extension_method('s')
Exponent operator is another good alternative, with better precedence, but worse assosiation, and it looks worse. Anyway, looking at this and some other threads, it seems like some people really-really want infix functions.
)