Parameters with ParamSpec and unions that contain callable, and a tuple containing the callable

For pandas-stubs, we are trying to come up with a stub for DataFrame.pipe() . The function signature is DataFrame.pipe(func, *args, **kwargs) . The parameter func is described as:

Function to apply to the Series/DataFrame. args , and kwargs are passed into func . Alternatively a (callable, data_keyword) tuple where data_keyword is a string indicating the keyword of callable that expects the Series/DataFrame.

A contributor came up with a signature that works for mypy, but not pyright. I came up with the following example, that passes mypy, but not pyright

from typing import Callable, Concatenate
from typing_extensions import ParamSpec

P = ParamSpec("P")

def func1(x: int):

def func2(x: int, /, y: str):

def accept_func(
    f: Callable[Concatenate[int, P], None] | tuple[int, Callable[..., None]],
    *args: P.args,
    **kwargs: P.kwargs,
) -> None:
    if isinstance(f, tuple):
        if args:
            f[1](f[0], args)

accept_func((2, func1))
accept_func((3, func2), "abc")

pyright reports the following: - error: Arguments for ParamSpec "P@accept_func" are missing (reportGeneralTypeIssues) - error: Arguments for ParamSpec "P@accept_func" are missing (reportGeneralTypeIssues) - error: Arguments for ParamSpec "P@accept_func" are missing (reportGeneralTypeIssues)

mypy reports that the above is acceptable.

Based on @erictraut response here: Paramspec used within union reports errors that paramspec is missing · Issue #6796 · microsoft/pyright · GitHub

there seems to be some ambiguity from PEP 612 if this should be accepted or not. He suggested getting advice from this community as to a way forward.

I see I should’ve first read the link, as expected Eric has come up with the same solution… I’ll still leave my answer below for postery and also as my stance on whether or not this should just work with all the uses of P.args/P.kwargs being substituted with Any, if P has nothing to bind to.

I think it’s better to use overloads, because pyright is kind of correct that when you pass a tuple, then P has nothing to bind to, so it’s ambiguous what should happen. Similar things can happen with other function signatures that use a type var in a union with another type.

def accept_func(
    f: Callable[Concatenate[int, P], None],
    *args: P.args,
    **kwargs: P.kwargs,
) -> None: ...

def accept_func(
    f: tuple[int, Callable[..., None]],
    *args: Any,
    **kwargs: Any,
) -> None: ...

While more verbose, this is very explicit about when P actually applies, and when it’s just Any.

It’s also worth noting, that while ParamSpec might be a relatively unambiguous case right now[1], what about once it gains bound or default, what should happen then? The bound and default should probably still apply, that’s what they’re for after all, but was that actually what you intended to express?

I do like the use of singular type vars in functions as a gradual type[2] with some additional constraints placed upon it, so I’ll say that I don’t fully agree with how strictly this is handled by pyright and prefer the more loose approach by mypy, because it’s more expressive, but I also think it’s fair to say that if it’s singular only some of the time, because you put the type var into a Union, then it probably is a mistake and not actually intentional, especially if it’s an overlapping Union[3], which member do you bind more strongly to? The type var or a concrete matching type in the union? Do you bind to both? What if you have more than one type var in the union?

  1. i.e. you can come up with a simple, consistent rule, that should result in what most people would expect to happen ↩︎

  2. i.e. something like Any ↩︎

  3. i.e. at least one of the types in the union could also fulfill the constraints of the type var ↩︎