My understanding is that the way to make a derived generic type without actually resolving the specifics of the type variable is to simply inherit from the base class and use the same type variable as the base class.
This is a simplified example of doing that with a bound type variable:
from typing import Generic, Sequence, TypeVar
T = TypeVar("T", str, dict)
class BaseGeneric(Generic[T]):
def __init__(self, example: Sequence[T]):
self._example: Sequence[T] = example
class ChildGeneric(BaseGeneric[T]):
def __init__(self, example: Sequence[T]):
super().__init__(example)
However, mypy generates a pair of errors on the subclass call to the base class initialiser:
$ mypy example.py
example.py:11: error: Argument 1 to "__init__" of "BaseGeneric" has incompatible type "Sequence[str]"; expected "Sequence[T]" [arg-type]
example.py:11: error: Argument 1 to "__init__" of "BaseGeneric" has incompatible type "Sequence[dict[Any, Any]]"; expected "Sequence[T]" [arg-type]
Found 2 errors in 1 file (checked 1 source file)
Switching to an unbound type variable eliminates the errors. I couldn’t find anything obviously related on the mypy issue tracker, so is this a not-yet-reported bug in mypy’s handling of bound type variables, or is my understanding incorrect and I should be declaring these subclasses some other way?
I would expect the subclass to also specify Generic[T] as a base class in order to bind the typevar, but seem to still yield similar errors. Seems like a mypy bug to me, but maybe related to how __init__ is not considered part of LSP …
That was my first thought as well, but I looked that up back when I first encountered the issue and confirmed that you only need to do that when introducing new type vars that the base class doesn’t already accept.
I just ran across the comment I had in my code about it today, and came up with the simplified example to post (as well as doing the experiment to confirm that unbound type vars don’t run into the same confusion).
This suggestion at least let me find a workaround that keeps the type constraints and doesn’t rely on hitting the super().__init__() call with the # type: ignore hammer:
from typing import Generic, Sequence, TypeVar
T = TypeVar("T", str, dict)
class BaseGeneric(Generic[T]):
def __init__(self, example: Sequence[T]):
self._init_base(example)
def _init_base(self, example: Sequence[T]):
self._example: Sequence[T] = example
class ChildGeneric(BaseGeneric[T]):
def __init__(self, example: Sequence[T]):
self._init_base(example)
So yeah, it’s definitely some poor interaction between bound type variables and __init__ specifically (since an unbound type var doesn’t cause complaints, and neither does calling a base class method other than __init__).
Edit: Hmm, maybe the subclass should be using an unbound type var, and leaving it to the base class to impose the bound type var restriction?
Edit 2: Nope, that gets rejected when declaring the child class (with the unbound type var not being accepted as a valid type argument value for the base class)
This looks like a bug in mypy. The code sample should type check without error. It works fine in pyright.
I’ll note that your code sample does not use a “bound” type variable (i.e. one with an “upper bound”). Rather, it uses a type variable with value constraints.
I recommend against using value-constrained type variables if it can be avoided. There’s usually a better approach. Value-constrained type variables are not well defined in the typing spec currently, and I’m not optimistic that we’ll be able to fully spec them in a way that is consistent and composes well with other type features. If you use type variables with value constraints, you’re likely to see a variety of strange bugs and inconsistent behaviors. The one you’re experiencing here is just the tip of the ice berg.
If you’re interested in learning more about value-constrained type variables and how they’re handled in mypy vs pyright, I’ve included some details in the pyright documentation.
Here are some alternative approaches that (depending on your use case) can eliminate the need for value-constrained type variables:
Make your class non-generic and simply replace the type variable with a union.
Replace the value-constrained type variable with a “bound” type variable using a union of the supported types.
If the type variable is scoped to a function or method, use an overloaded signature instead.
Thanks for that explanation (and clarification for the right terminology).
Making the use case a little less vague, it comes from an AI LLM prediction API where the result() method may return dict or str based on whether the request (specified when creating the prediction object) was for a structured or unstructured prediction.
Of the alternatives you suggested, the type var bound to the union sounds like the best fit, so I’ll fix up this case to use that pattern instead.
After trying it again, I am rediscovering the problem I ran into with attempting to use a bound union instead of a value constraint.
With the value constraint, the child generic type is mostly treated as ChildGeneric[str] | ChildGeneric[dict], which then lets you declare function overloads that disambiguate between the two possible results.
By contrast, with the type variable bound to a union, the child generic type is mostly treated as ChildGeneric[str|dict], which means attempting to disambiguate via a function overload gives you “Overloaded function implementation cannot produce return type of signature 1” (or an error in type inference somewhere along the chain, depending on exactly how intervening variable type declarations are structured).
So even with its known flaws, I think the value constraint is the most accurate expression of the typing involved in this application (any given instance of the generic classes will be either working with structured data, or working with unstructured data, it isn’t permitted or supported to mix and match them as the union based typing suggests is allowed).