@extends decorator to explicitly marking an overriden function to accept additional variables

Ideas
1

Hey there :smiley:
I’m not sure if there is already an existing solution or proposal for this (the closest thing I saw was https://peps.python.org/pep-0698/ but it is not quite the thing I need. I also, tried to achieve something with ParamSpec and Concatenate but it didn’t quite work for me either), but i encounter this problem quite a lot.

Whenever I create a class or a function, I like to have full documentation and annotations for each parameter to ease the work of users when using it. The users can have autocompletion from their IDE and a convinient popup with the parameter documentation as they are writing the values.

However, when inheriting from the class (in order to add additional attributes) or wrapping the function (once again, to add an additional parameter).

I would need to either:

  1. copy and paste every single parameter annotation and documentation onto the new function, which causes code duplications and a large risk of outdated documentation whenever changes are done to the original signature.

  2. Define the original parameters as *args, **kwargs and refering the user to a separate documentation in which case, the user loses the convinience of having the relveant information infront of their eyes as they are using the new function. (After all, IDEs have no way of knowing that these dynamic parameters exist solely to be passed down to the original function)

I suggest we have some kind of an @extends decorator, similar to the @wraps decorator in functools, which can be used as follows:

class A:
    def __init__(self, param1: int, param2: str):
        """
        :param param1: Documentation for param1
        :param param2: Documentation for param2
        """
        pass

class B(A):
    @extends(A.__init__)
    def __init__(self, param3: int, *args, **kwargs):
        """
        :param param3: Documentation for param3
        """
        super().__init__(*args, **kwargs)

The usage of the decorator will tell IDE’s that the decorated function is expecting to receive the exact same arguments of the original function (documentation, and annotations are the exact same), but with the addition of the new static parameters. IDE’s will be able to show the combined documentation and annotations of all the parameters, and would be able to warn when a incorrect dynamic parameter is passed (when the parameter doesn’t belong to the original function or the wrapped function.)

The decorator is meant to specifically be used when there are absolutly nothing done with the dynamic parameters except for passing them on to the original callable. Any other usage could be result in a warning.

There are some edge cases in regards to positional only arguments, but we can either forbid cases that cause ambiguity or forbid their usage with the decorator alltogether for simplicitly sake.

2

I think this is an important problem, but I don’t think the extends idea is going to solve it. In general:

  • derived classes might also want to synthesize parameters for superclasses, and
  • non-final derived classes don’t know which parameters their superclasses accept.

I’ve been waiting for TypedDict to be more established to suggest something, but I think we may need something a bit more flexible. Maybe something like

from typing import SuperKwargs

class B(A):
    def __init__(self, param3: int, **kwargs: SuperKwargs):

And some notation for synthesized parameters, say SuperKwargs['param1'] to indicate that param is synthesized.

The problem with extends are that it doesn’t have notation for synthesized parameters and you don’t know which method you extend (you don’t know it’s A.__init__).

3

Perhaps the name “extends” does not quite fit the kind of feature im looking for. But the idea is that by definition, a callable using extends is explicitly saying there is no processing being done on the kwargs before passing them to the other callable. Every attempt to process them would go against the meaning of “extending another callable” and would therefore cause IDEs to warn the user.

As for the second point, I haven’t thought about the super problem but we could say that calls to the extended callable must be explicit (using super should cause a type warning).
We could also decide that it is the IDE’s responsiblity to warn when one of a class’s multiple inheritors is using super in such a context.

As for the suggestion, I don’t see how SuperKwargs avoids the same problem (which class are SuperKwargs relate to? A or the hypothetical A2 class that might be a base class in multiple inheritance?) I’d love to hear further about the suggestion :slight_smile:

4

You mean explicitly calling A.__init__? That’s just bad design.

It avoids it by not specifying the superclass explicitly. Your type checker or IDE would figure out the parameters for any given class.

5

At least PyCharm inherits parameter annotations: perhaps you could submit a feature request to your IDE to replicate that functionality.

I think there’s merit to have a way to document only added parameters (perhaps with typing.Annotated).

This is not compatible with the signature of the parent class, meaning you could not use B in place of A. I wouldn’t use extends if I was forced to do this.

6

Typically, __init__ doesn’t have to obey the LSP. And if you don’t do something like this, your class won’t work with cooperative multiple inheritance with superclasses that add parameters.

1 Like
8

Please don’t resurrect old/abandoned/settled topics. If you have a new idea or want to devote your own effort towards reviving a proposal, please start a new topic with up-to-date proposal details.