What are the subtyping rules for tuple[T, ...]?

gwerbin · December 21, 2023, 1:16am

Another place where you find heterogeneously-typed tuples of statically-unknown size is in the “data frame” libraries Pandas and Polars.

As a concrete example, pandas.DataFrame.itertuples dynamically constructs a namedtuple and returns an iterator of those dynamically-constructed namedtuples:

github.com

pandas-dev/pandas/blob/a671b5a8bf5dd13fb19f0e88edc679bc9e15c673/pandas/core/frame.py#L1528-L1537


      
          if name is not None:
              # https://github.com/python/mypy/issues/9046
              # error: namedtuple() expects a string literal as the first argument
              itertuple = collections.namedtuple(  # type: ignore[misc]
                  name, fields, rename=True
              )
              return map(itertuple._make, zip(*arrays))
          
          # fallback to regular tuples
          return zip(*arrays)

Currently Pandas annotates the result as tuple[Any, ...].

I stumbled into this issue because I, as the programmer, happen to know exactly what to expect from that named tuple, so I wrote something like this:

import typing
import pandas as pd

if typing.TYPE_CHECKING:
    class WidgetRow(typing.NamedTuple):
        size: float
        color: str
        quantity: int

widgets = pd.read_parquet("widgets.parquet")

widget: WidgetRow
for widget in widgets.itertuples():
    typing.assert_type(widget.quantity, int)

And I was surprised to find that Mypy didn’t like it:

error: Incompatible types in assignment (expression has type "tuple[Any, ...]", variable has type "WidgetRow")  [assignment]
Found 1 error in 1 file (checked 1 source file)

So consider this a vote from a “regular user” that Python ought to provide some way to express this concept in type annotations, whatever that might be.