Supporting patched methods to classes

Typing
1

I’d like to see the ability to add methods to classes and have the type hinters/LSPs keep track of the new methods.

Subclassing isn’t always possible because not everything else will be the subclass. This is particularly evident in polars with its ability to add plugins.

One of the main components of polars is their Expr (Expression) class. Expressions are used inside a DataFrame context like this:

import polars as pl

def func(df: pl.DataFrame):
    a: pl.Expr = pl.col("a")
    df.select((a*2).sum())

Notice how the Expression is chainable so that (a*2) returns an Expr which can then be used to call sum.

The intention is that plugins will patch pl.Expr with their functionality so that we can do:

import polars as pl
import some_plugin

def func(df: pl.DataFrame):
    a: pl.Expr = pl.col("a")
    df.select((a*2).plugin_feature())

The above works at runtime but there’s no way to have the checker know about plugin_feature.

2

Can plugins subclass pl.Expr, and users can type func to a: ExprSubclass, or is the question how to statically type, directly assigning a new unbound method plugin_feature to it?

3

Subclassing does not work. The question is how to make typing work with something like pl.Expr.plugin_feature=plugin_feature in one library’s init such that when that is imported and a user does pl.col("a").plugin_feature( that they’ll get type hints?

I am 99.9% sure the answer right now is that you simply can’t so this post is meant as a ‘please can this functionality be added?’ request but I’d love to be wrong and that there is a way to do it now.

4

As I understand it, this:

pl.Expr.plugin_feature=plugin_feature

requires dynamic typing. Type checkers would have to essentially run the entire application to check it, and be sure all such additions are accounted for, and that’s no longer static type checking. So I don’t think making typing work with this is going to happen any time soon.

5

Admittingly, I don’t know how typers and LSPs work so I know I’m over trivializing but even still, it doesn’t need to run any part of the code. As a more simple example, does it seem more tenable like this:

class Myclass:
    def __init__(self):
        self.s="s"
    def hello(self, to:str)->None:
        print(self.s + to)
        
def foo(self, bar: str)->None:
    print(self.s + bar)
    
Myclass.foo=foo
6

While it wouldn’t be impossible to support something like this, it makes type checkers’ jobs incredibly difficult, since all types would essentially have to be considered temporary until all modules have been analyzed, since you wouldn’t be able to exclude the presence of additional attributes. Anything other than that would result in part of your program being type checked against the wrong type until it sees that there are additional properties.

Type checkers would also have to decide on how to deal with accesses to this type before it has been augmented. You would also need strategies for conflict resolutions, when the same attribute has been assigned multiple times to different types. So it quickly becomes a complex mess where you can’t make many solid inferences about the state of things. So it’s more reliable to just not try to handle deeply dynamic stuff like this.

That being said, I can understand the demand for facilities to allow type checkers to understand some more dynamic features.

What I have done in some projects of mine is write customized stubs that contain some of this dynamic information, that I know to be true for my specific configuration, statically. Although it can be quite a pain to maintain, so it’s usually only really worth it for code that doesn’t have any inline type annotations to begin with, or where you only have to replace one or two modules with custom stubs.

I have also previously made a proposal for type placeholders, that would allow library users to replace some of the more looser/gradual types in a third party module with hardened types that better reflect your specific use/configuration of a library. But there didn’t appear to be any interest from the community.