Using TypeVarTuple to describe Tuple of generic objects

Typing
1

Hello,

I’ve been using variadic generics (TypeVarTuple) with some success, but I’m running into a problem in the following situation.

Here is an attempt to explain, but I think the code example is much easier to understand:
I have a generic class that contains an item (of the type specified by the type variable).
I have a method that takes a sequence of objects of this class and returns a tuple of their items.
I want to express that the type of item 0 matches the type argument of element 0 in my sequence, etc.
But that doesn’t seem possible?

T = TypeVar("T")
TT = TypeVarTuple("TT")

@dataclasses.dataclass
class A(Generic[T]):
    item: T

# How to type this function?
def extract_all(aa: List[A[T]]) -> Tuple[*TT]:
    return tuple(a.item for a in aa)

Obviously, there is no link between T and TT, but I don’t see a way to express this.

More broadly, I would like to be able to express the mapping of a tuple of objects A[T] onto a tuple of objects B[T] where I can express that for corresponding objects the type arguments are the same.

Maybe with syntax like:

def mapper(aa: Tuple[A[T in TT]]) -> Tuple[B[T in TT]]:

Am I overlooking something and is this currently possible?

2 Likes
2

You don’t need a TypeVarTuple for this at all:

def extract_all(aa: List[A[T]]) -> Tuple[T, ...]:
    return tuple(a.item for a in aa)

At least as far as homogenous collections go, if you want to extract the type for each element, rather than end up a union, there’s currently no way to express this, because we have no map operations on TypeVarTuple.

Some variation of Map has been proposed several times in the past, but no concrete PEP has come out of it yet, with Map the heterogeneous case may look something like this:

def extract_all(*aa: *Map[A, Ts]) -> Tuple[*Ts]:
    return tuple(a.item for a in aa)

With a list as a parameter you won’t be able to get a heterogenous type anyways, it would have to be varargs or a tuple parameter.

3

Thank you for the informative reply.

I’m looking for the heterogeneous case indeed.
The Map construct looks promising, I hope it will make it into a PEP soon.

4

I’ve seen many people who hope that Map[] will be added but as far as I know nobody is actually working on a PEP. I’d encourage those who want to see this construct in the language to actually work on such a PEP. Create PEP for Map with bind: match Tuple of parameterized types with parameters from TypeVarTuple · Issue #1383 · python/typing · GitHub might be a good place to coordinate.

5

One issue I have with some Map proposals I’ve seen in the past (including the one in the early drafts of PEP 646) is that they’re very specialized and addresses only a single use case. I prefer the approach that typescript adopted here, which allows the concept to be applied more generally.

There has been some work to explore a more general mechanism: Proposal: KeyType and ElementType for TypedDicts (also, Map) · python/typing · Discussion #1412 · GitHub.

1 Like
6

The main issue I have with copying the TypeScript comprehension-like syntax, is that it doesn’t translate very well into runtime introspection, I feel like any proposal would need to yield an object structure that could easily be translated back to the original annotation.

While it’s certainly possible to do something using the locals from the Frame associated with the Generator, that at best gives you the name of the TypeVar you are mapping to, but not the two other parts of the expression, you may be able to collect that information by iterating over the generator and have things like KeyOf yield dummy introspection objects when iterated over, but it all feels very clunky, so I wonder if there’s a syntax that’s still just as powerful but lends itself better to runtime introspection, and also doesn’t risk creating weird generator objects in annotation contexts [1].

I feel like a functional-style Map combined with Unpack/* should be able to deliver the same possibilities without making runtime introspection hell. This could even support generics with multiple parameters, by making Map variadic and allowing things like *tuple[T, ...] to bind a scalar TypeVar in order to allow mapping over generics that have variadic as well as scalar type vars.

A mixed example could look like this:

def filter_choices[T, *Ts](
    *choices: *Map[tuple, *tuple[T, ...], *Ts],
    label_filter: Collection[T]
) -> tuple[*Ts]: ...

Where a choice is a tuple of a label and a value, and the label type is homogenous, but the value type is heterogenous.

The KeyOf and ValueOf part of the proposal feels orthogonal to the whole discussion, since they’re essentially just a way to extract a TypeVarTuple or tuple from a TypedDict, so as long as we can come up with something that works well with TypeVarTuple and tuple, it should also work well with ValueOf and KeyOf.

  1. or runtime overhead if __class_getitem__ is supposed to magically convert these to something easier to inspect ↩︎

7

My main desire for Map operator is it handles both typevartuple and paramspec. I occasionally see decorators that transform each argument both positional (typevartuple can handle) and keyword in a way that would best be described with Map.