Preface
Before I begin, please note that this is the first time that I have considered and researched the process of proposing a PEP. The purpose of this post is to get some feedback from the community on a PEP I plan to propose. The format of this post does not necessarily represent the final format of the PEP, and, based upon other PEPs I have viewed, this PEP (at least in its current state) is quite bare-bones. Please read through this whole post before asking questions or giving feedback. The work-in-progress title of this pep is: Type Transformations on Variadic Generics [1].
Abstract
This PEP, which is a follow-up to PEP 646, would add specifications to allow for type transformations/mappings on variadic generics (typing.TypeVarTuple).
Motivation
This is something that I had originally believed to already be a part of PEP 646. It was only until recently that I found out that, as noted by Pradeep Srinivasan (one of the authors of PEP 646), this was excluded from the aforementioned PEP due to the level of complexity that the PEP had already reached. In the aforementioned notation by Srinivasan, it was mentioned that plans were in place to release a follow-up PEP that incorporated these changes by late 2022 - although it appears as though that never ended up happening [2].
This PEP would allow for transformations similar to those available to typing.TypeVar, such as below:
def foo[T](x: Type[T]) -> T: ...
to be available for variadic generics as well:
def foo[*Ts](*x: *Type[Ts]) -> Tuple[*Ts]: ...
Examples
NOTE: The syntax used in these examples is not concrete. I will go over the proposed syntax later.
Below is a condensed version of the builtins.map stub in Python 3.12:
class map(Iterator[_S]):
@overload
def __new__(
cls,
func: Callable[[_T1], _S],
iter1: Iterable[_T1],
/
) -> Self: ...
@overload
def __new__(
cls,
func: Callable[[_T1, _T2], _S],
iter1: Iterable[_T1],
iter2: Iterable[_T2],
/
) -> Self: ...
# ...and so on
@overload
def __new__(
cls,
func: Callable[..., _S],
iter1: Iterable[Any],
iter2: Iterable[Any],
iter3: Iterable[Any],
iter4: Iterable[Any],
iter5: Iterable[Any],
iter6: Iterable[Any],
/,
*iterables: Iterable[Any],
) -> Self: ...
def __iter__(self) -> Self: ...
def __next__(self) -> _S: ...
This could, as mentioned by Randolf Scholz, be more accurately (and concisely) typed using the following:
class map[F: Callable[[Ts], S], *Ts]:
@overload
def __new__(cls, func: F, /) -> Never: ... # need at least 1 iterable.
@overload
def __new__(cls, func: F, /, *iterables: *Iterable[Ts]) -> Self: ...
def __iter__(self) -> Self: ...
def __next__(self) -> S: ...
Not only is this much less cumbersome than before, but it also allows for any number of types to be used. I.e. it is not limited by the number of __new__ overloads you wish to write.
This is just one of many examples. Links to more examples are listed below:
- Example by Lawrence Chan in python/typing GH issue #1216
- Example by Yurii Karabas in python/typing GH issue #1216
- Example by āMardoxxā in stackoverflow
- Example by Randolf Scholz in python/typing GH issue #1216
- Example by Philipp Dowling in python/typing GH issue #1216
- Example by Randolf Scholz in python/typing GH issue #1216
Specification
Similar to how type transformations can be applied to non-variadic type variables, type transformations will now be applicable to variadic generics, so long as all transformations are performed before the unpack operation. For example, the following function using variadic type transformations:
def foo[*Ts](*args: *Type[Ts]) -> Tuple[*Ts]: ...
when called with a number of parameters (for this example, str, int, and float):
foo(str, int, float)
would be understood to have a return type of Tuple[str, int, float]. Without this type transformation, i.e.:
def foo[*Ts](*args: *Ts) -> Tuple[*Ts]: ...
and, when called with the same values as before, the return type would be understood to be Tuple[Type[str], Type[int], Type[float]], as expected given the above function definition.
More formally, for any parameterized type T, the following holds true:
-
If
Taccepts exactly one generic parameter, thenT[Ps](wherePsis a variadic generic) can be understood to contain the typesT[Psā],T[Psā], ā¦,T[Psā]. -
If
Taccepts more than one generic parameter, thenT[As, Bs, ..., Ns](whereAs,Bs, andNsare variadic generics) can be understood to contain the typesT[Asā, Bsā, ..., Nsā],T[Asā, Bsā, ..., Nsā], ā¦,T[Asā, Bsā, ..., Nsā]. Additionally, the length of all passed variadic generics (As,Bs, ā¦,Ns) must be equal. -
When both generics and variadic generics are mixed, all non-variadic generics will be reused as many times as needed, such that any given non-variadic generic
Xis equivalent to a variadic genericXsthat containsXas many times as is needed. That is, ifTaccepts both variadic generics and non-variadic generics, thenT[As, Bs, ..., Ns, X, ..., Z](whereAs,Bs, andNsare variadic generics, andXandZare non-variadic generics) can be understood to contain the typesT[Asā, Bsā, ..., Nsā, X, ..., Z],T[Asā, Bsā, ..., Nsā, X, ..., Z], ā¦,T[Asā, Bsā, ..., Nsā, X, ..., Z]
Backwards Compatibility
For backwards compatibility, the Map [3] object will be added to the typing_extensions library. The structure of how this type will be used is not concrete, so here are three options that I have seen in other peopleās examples:
NOTE: in all examples below, T is a generic class that accepts the required number of types, Us and Vs are both variadic generics, and X is a non-variadic generic.
-
- Single variadic generic usage:
Map[T[Us]] == T[Us] - Multiple variadic generic usage:
Map[T[Us, Vs]] == T[Us, Vs] - Mixed generic usage:
Map[T[Us, Vs, X]] == T[Us, Vs, X]
- Single variadic generic usage:
-
- Single variadic generic usage:
Map[T[Us], Us] == T[Us] - Multiple variadic generic usage:
Map[T[Us, Vs], Us, Vs] == T[Us, Vs] - Mixed generic usage:
Map[T[Us, Vs, X], Us, Vs, X] == T[Us, Vs, X]
- Single variadic generic usage:
-
- Single variadic generic usage:
Map[T[u], u in Us] == T[Us] - Multiple variadic generic usage:
Map[T[u, v], u in Us, v in Vs] == T[Us, Vs] - Mixed generic usage:
Map[T[u, v, X], u in Us, v in Vs] == T[Us, Vs, X]
- Single variadic generic usage:
Notes
Final Questions and Feedback
As this is a draft PEP discussion, all feedback and criticism is welcome. Please be kind when discussing your ideas with others. If possible, below is a list of aspects I would like feedback on. Additional feedback is welcome as well.
- Notes 1, 2 and 3.
- In a sub-section of the PEP 646 Specification, it is noted that
"[...]type variable tuples must always be used unpacked (that is, prefixed by the star operator)."Does this, by definition, make the specification of this PEP (at least when given the proposed syntax) impossible? - In the Backwards Compatibility section, which listed (or unlisted) option do you believe is the most viable and/or easy to use and understand?
- Is the formal specification at the end of the Specification section understandable?
Thank you for reading this far!