I keep writing the following construct when I want to say that my __init__ or any other method has the same arguments as the same method in the parent class. It is mostly useful when you want your library to be fully compatible with any breaking changes of another library’s interface so you use **kwargs but you still want your users to have nice IDE support.
I understand that it’s not a complex approach so people could just write it themselves but I have never seen it used. Instead, people just write **kwargs: Any and give up. So having support for this in the standard library would serve as a nice guide for anyone wanting to write such code.
Example
Let’s say I’m using FastAPI and I want my own FastAPI class with post-processing after init. I do:
from fastapi import FastAPI
from typing import TypeVar, Callable
_T = TypeVar("_T", bound=Callable)
def same_definition_as_in(t: _T) -> Callable[[Callable], _T]:
def decorator(f: Callable) -> _T:
return f # type: ignore
return decorator
class MyFastAPI(FastAPI)
@same_definition_as_in(FastAPI.__init__)
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# some post processing here
This grants me the best of both worlds: no matter which breaking changes fastapi releases, they won’t affect my library but at the same time my users get full type hinting for MyFastAPI.__init__.
Let’s say I’m using FastAPI and I want my own FastAPI class with post-processing after init. I do:
from fastapi import FastAPI
from typing import TypeVar, Callable
_T = TypeVar("_T", bound=Callable)
def same_definition_as_in(t: _T) -> Callable[[Callable], _T]:
def decorator(f: Callable) -> _T:
return f # type: ignore
return decorator
class MyFastAPI(FastAPI)
@same_definition_as_in(FastAPI.__init__)
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# some post processing here
This grants me the best of both worlds: no matter which breaking changes fastapi releases, they won’t affect my library but at the same time my users get full type hinting for MyFastAPI.__init__.
This is typable properly with ParamSpec. You haven’t included enough information here to give you the correct answer, but decorators that return the same interface while wrapping are well-supported by ParamSpec
@Ovsyanka isn’t really asking for a way to make his example working, in fact it already type checks correctly (without ParamSpec). He was asking for a built in way of doing what he is currently doing manually.
One idea with ParamSpec would be:
from typing import Generic, ParamSpec
P = ParamSpec("P")
class MyFastAPI(FastAPI, Generic[P]):
def __init__(self, *args: P.args, **kwargs: P.kwargs):
super().__init__(*args, **kwargs)
# some post processing here
MyFastAPI[int, str](1, "1") # Less than ideal, what we want here is to have the init signature automatically inferred from the base class
It already exists, but as I said, I don’t have enough context around the problem space. You can do what was asked for already without new type syntax, generally through class decorators with paramspec rather than just subclassing.
@mikeshardmind I am sorry for not providing a more specific use case. Let me make it a bit less abstract.
I have multiple libraries that subclass some classes from other libraries so that we can use the extended functionality conveniently. For example, when versioning, we do something along the lines of:
from fastapi import APIRouter, FastAPI
class VersionBundle:
...
class MyFastAPI(FastAPI):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.version_bundle = VersionBundle()
def include_versioned_router(self, router: APIRouter):
generated_routers = generate_versioned_routers(router, versions=self.version_bundle)
for router in generated_routers:
self.include_router(router)
I have the following requirements at the same time:
If FastAPI deletes/renames/adds a parameter, I would like my library to continue functioning without modifications.
My users have full IDE support (autocomplete, type checking) for the __init__ method. Even if FastAPI updates and adds/removes/renames some parameter, and my library has not received any updates – my users must still see the full updated type signature as if they were using FastAPI directly.
Would you help me understand how I can use ParamSpec here to fulfill these two requirements without compromising on the interface?
I’ll do what I can here, but part of this is less trivial with the full context (and would likely require type intersections for ergonomic support via a mixin and composition rather than direct inheritance) As it isn’t just postprocessing on existing attributes, but adding attributes and also exposing their use in what appear to be public functions you intend to have typed as well
For instance, what if FastAPI adds an attribute named version_bundle that conflicts with yours?
Is subclassing the best way for your users to use this? (What about WSGI middleware?)
With intersections, you could do this more ergonomically, but until then, assuming subclassing remains the best option:
import types
from collections.abc import Callable
from typing import ParamSpec, Protocol, TypeVar, runtime_checkable
T = TypeVar("T")
P = ParamSpec("P")
class FastApi:
...
class VersionBundle:
...
@runtime_checkable
class HasVersionBundle(Protocol):
version_bundle: VersionBundle
def _post_processing_init(f: Callable[P, None]) -> Callable[P, None]:
def wrapped_init(*args: P.args, **kwargs: P.kwargs) -> None:
f(*args, **kwargs) # call the original __init__
instance, *_args = args
instance.version_bundle = VersionBundle() # type: ignore
return wrapped_init
def post_processed(typ: type[T]) -> type[T]:
name = f"Postprocessed{typ.__name__}"
nt: type[T] = types.new_class(name, (typ,) ) # type: ignore
# types.new_class should probably have it's type in the typeshed updated to be
# (name: str, bases: tuple[*Ts], kwds: ..., exec_body: Callable) -> type[*Ts]
nt.__init__ = _post_processing_init(nt.__init__)
return nt
base_with_modified_init = post_processed(FastApi)
class AfterMuchWork(base_with_modified_init):
def include_versioned_router(self, router: ...):
# doing this as an assertion means users using -O don't pay a runtime cost
assert isinstance(self, HasVersionBundle)
...
This isn’t an obvious or easy-to-come-across solution, so I can sympathize with the existing tools here not being enough. I think intersections would be more appropriate as a long-term solution for improving this than adding same_definition_as_in, and you’ll note this still requires type ignores, though they are less impactful in scope, and we can reason should be fine and that could even be improved by improving the typeshed
@mikeshardmind
I feel like this approach would be clunky even with type intersections. Inheritance is a normal mechanism that we can use to achieve the same results without as much magic.
A single type ignore is not too scary – typing itself is full of magic. If it was fully typed, we would have a lot of type ignores there. It could be considered a form of casting here.
As for other breaking changes in FastAPI – I usually use prefixes to prevent that. And the chance of init signature changing is infinitely higher than a chance of FastAPI using the exact same attribute name as me.
For what it’s worth, this isn’t the approach that would be used with intersections. This is what works Without that. With intersections, you could simply do:
@runtime_checkable
class Augmented(Protocol):
version_bundle: VersionBundle
def include_versioned_router(self, router: APIRouter):
...
class _MyFastAPI:
""" Private implementation,
since you don't want to have to keep
`*args, **kwargs` in sync with upstream
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.version_bundle = VersionBundle()
def include_versioned_router(self, router: APIRouter):
generated_routers = generate_versioned_routers(router, versions=self.version_bundle)
for router in generated_routers:
self.include_router(router)
def _get_class() -> type[FastAPI & Augmented]:
return _MyFastAPI # type ignore
MyFastAPI = get_class() # export this
I love type intersections and have even contributed a little to its unreleased PEP but I really don’t agree that this is the case for type intersections. This is a case for regular inheritance.
And I would argue this isn’t a case for inheritance at all. your shared logic here could just as easily be functional over an instanceof FastAPI rather than an instance method and from what you have here. It could also be a mixin (composition over inheritance) and the responsibility of your users to subclass Fast API and your mixin.
intersections would give you a way to express what you said want to do, but there are several other ways to do it which work now and which are arguably better design.
Your same_definition_as_in looks very similar to our copy_signature. As the docstring explains, our decorator was based on a recipe in a comment for a typing issue proposing this same functionality. The issue was closed when ParamSpec was added even though it didn’t really solve the issue.
Having something like this in typing or supported otherwise would be great. It would be especially convenient if it would be possible to add new parameters to the signature and not only copy the exact same signature.
So robotframework, Cadwyn, and apparently cattrs use this construct. These are libraries, yes, but regular business logic needs a similar structure for handling subclassing and super().__init__(*args, **kwargs) too.
It feels to me like enough of a reason to add it to typing. If we were to ever add it to typing, what name would you folks prefer?
same_signature_as_in
copy_signature
same_definition_as_in
wraps (please, don’t pick this one. I expect a lot of confusion from people, especially with auto-imports)
There is also an option of just improving support for functools.wraps. However, functools.wraps does not exactly do the same thing because it changes runtime attributes which might be an unwanted behavior in use cases of copy_signature.
Additionally, there is a more complex option: we could draft how the more advanced function signature modification (people have mentioned it above in this thread) would look like in the future and then implement a small portion of that to just support copy_signature. Though this one is probably the most dangerous option because if we get the future interface wrong, we are in for a lot of trouble, bad interfaces, and nasty deprecations. It is much easier to support a three-liner function that essentially does nothing at runtime.