# Allow self binding for generic ParamSpec **URL:** \ **Category:** Typing\ **Created:** [April 14, 2024, 1:29pm UTC](https://discuss.python.org/t/allow-self-binding-for-generic-paramspec/50948 "2024-04-14T13:29:47Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![cdce8p](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cdce8p/32/3575_2.png) [@cdce8p](https://discuss.python.org/u/cdce8p)\ **Post date:** [April 14, 2024, 1:29pm UTC](https://discuss.python.org/t/allow-self-binding-for-generic-paramspec/50948/1 "2024-04-14T13:29:47Z")
We ran into some issues while trying to use `ParamSpec` to type `lru_cache` in typeshed. See > > > This was originally added in #6221 but later reverted due to an incomplete Param…Spec implementation in mypy at the time. Maybe it works now. > > Refs: > \- #6356 > \- #6347 Basically, while `self` / `cls` is bound in case the `ParamSpec` is directly passed through as `Callable`, that isn’t the case for the generic version. ```py from typing import Callable, Generic, ParamSpec P = ParamSpec("P") class Wrapper(Generic[P]): def __call__ (self, *args: P.args, **kwargs: P.kwargs) -> None: ... def decorator(f: Callable[P, None]) -> Callable[P, None]: ... def lru_cache(f: Callable[P, None]) -> Wrapper[P]: ... class A: @decorator def method1(self, val: int) -> None: ... @lru_cache def method2(self, val: int) -> None: ... def test(self) -> None: reveal_type(self.method1) # def (val: int) reveal_type(self.method2) # Wrapper[[self: A, val: int]] self.method1(2) self.method2(2) # Error: Missing positional argument "val" ``` Although all type checkers seem to agree, this makes using generic `ParamSpec` in these cases difficult (if not impossible). One option would be to remove the first argument with `Concatenate` and using overloads but that creates additional issues with `@staticmethod` and `@classmethod`. The solution IMO would be to allow self binding even for generic ParamSpec, so that it doesn’t matter if the return type is a `Callable` or a custom generic. I.e. ```py def test(self) -> None: reveal_type(self.method1) # def (val: int) reveal_type(self.method2) # Wrapper[[val: int]] self.method1(2) self.method2(2) ``` A reference implementation for mypy can be found here: > > > Required for https://github.com/python/typeshed/pull/11662 ---
**Author:** ![Daverball](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/daverball/32/15931_2.png) [@Daverball](https://discuss.python.org/u/Daverball)\ **Post date:** [April 14, 2024, 2:28pm UTC](https://discuss.python.org/t/allow-self-binding-for-generic-paramspec/50948/2 "2024-04-14T14:28:21Z")
I believe this is one part the typing spec currently doesn’t cover almost at all. So it may be more helpful if we first add specification for how self-binding in conjunction with `ParamSpec` is supposed to work and then can extend that from just an explicit `Callable` type or function declaration to arbitrary types satisfying the `Callable` protocol, including descriptors that return such types from ` __get__ `, where the headaches usually really start to happen. Part of the problem certainly is ambiguity, since the type system is trying to abstract away Python’s object model\[1\] through special-casing callables. So it’s more of a question of finding out where the abstraction currently holds up and where it doesn’t and whether or not it can be fixed to cover more or all the possible cases accurately, without creating other problems. * * * 1. which creates `MethodType` and other wrapper types ---
**Author:** ![erictraut](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/erictraut/32/15190_2.png) [@erictraut](https://discuss.python.org/u/erictraut)\ **Post date:** [April 14, 2024, 5:15pm UTC](https://discuss.python.org/t/allow-self-binding-for-generic-paramspec/50948/3 "2024-04-14T17:15:08Z")
This is a long-standing problem, so thanks for starting the conversation. However, I think the problem is more complicated than you’ve indicated. Here’s some additional context. In your code example, the signature captured by `P` includes the `self` parameter of `method1` and `method2`. In both cases, `P` captures the signature `(self: Self@A, val: int)`. When this captured signature is applied to a `Callable` (as with `decorator` in your example), the specialized callable type is: ```python def (self: Self@A, val: int) -> None: ... ``` When you bind this to an instance of `A`, it works as you would expect for an instance method. When the same captured signature is applied to the `Wrapper` class (as it is with `lru_cache` in your example), the specialized type of the `Wrapper. __call__ ` method becomes: ```python def __call__ (__self: Self@Wrapper, self: Self@A, val: int) -> None: ... ``` When you call `self.method2(2)`, the first parameter (which I renamed `__self`) is bound to an instance of `Wrapper`. There is no opportunity to bind the second parameter (`self`) to an instance of `A`. I’m not sure what you’re proposing when you say “allow self binding for generic ParamSpec”. Are you suggesting that the `self` parameter of an instance method should never be captured by a `ParamSpec`? If so, that would break the `Callable` case. Are you suggesting that the extra `self` parameter should be dropped under certain circumstances when applying a captured signature during specialization? If so, under what circumstances? I’ll note that this gets even more complicated when a return type is captured along with a signature and that return type depends on the type of the `self` parameter. For example: ```python class Wrapper[**P, R]: def __call__ (self, *args: P.args, **kwargs: P.kwargs) -> R: ... def decorator1[**P, R](f: Callable[P, R]) -> Callable[P, R]: ... def decorator2[**P, R](f: Callable[P, R]) -> Wrapper[P, R]: ... class A: @decorator1 def method1(self) -> Self: return self @decorator2 def method2(self) -> Self: return self ``` In this case, the type of `self` (which is implicitly `Self`) must be preserved in the transformed method type to allow the return type (`Self`) to be solved at the time the method is called. It’s not clear how this would work in the case of `decorator2`. ---
**Author:** ![cdce8p](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cdce8p/32/3575_2.png) [@cdce8p](https://discuss.python.org/u/cdce8p)\ **Post date:** [April 14, 2024, 10:16pm UTC](https://discuss.python.org/t/allow-self-binding-for-generic-paramspec/50948/4 "2024-04-14T22:16:23Z")
> [@erictraut](#): > > When you call `self.method2(2)`, the first parameter (which I renamed `__self`) is bound to an instance of `Wrapper`. There is no opportunity to bind the second parameter (`self`) to an instance of `A`. > > I’m not sure what you’re proposing when you say “allow self binding for generic ParamSpec”. I think of it as implicitly passing `Self@A` as argument if `method2` is called with `self`, `type[Self]@A` in case the method is a `@classmethod` and not passing it for `@staticmethod` (same as for `A.method2`). > [@erictraut](#): > > I’ll note that this gets even more complicated when a return type is captured along with a signature and that return type depends on the type of the `self` parameter. For example: Return `Self` is indeed an interesting case. Notably, it doesn’t work currently with neither pyright nor mypy. For `reveal_type(self.method2()` pyright returns `Unknown`, whereas mypy even complains about a missing positional argument “self” and infers it as `Never`. This At least for mypy this suggests to me that the `Self@A` instance isn’t passed along which this proposal aims to resolve. With the mypy change above, it would be inferred as `R'5` i.e. `Any`.