# Generic \`typing.ForwardRef\` to support generic recursive types

**URL:** <https://discuss.python.org/t/generic-typing-forwardref-to-support-generic-recursive-types/20168>\
**Category:** Ideas\
**Tags:** typing, stdlib\
**Created:** [October 21, 2022, 9:45am UTC](https://discuss.python.org/t/generic-typing-forwardref-to-support-generic-recursive-types/20168 "2022-10-21T09:45:20Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![XuehaiPan](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/xuehaipan/32/9029_2.png) [@XuehaiPan](https://discuss.python.org/u/XuehaiPan)\
**Post date:** [October 21, 2022, 9:45am UTC](https://discuss.python.org/t/generic-typing-forwardref-to-support-generic-recursive-types/20168/1 "2022-10-21T09:45:20Z")

</div>

Since v0.981 `mypy` supports recursive types and will be enabled by default since v0.990. E.g.:

```python
JSON = Union[Dict[str, 'JSON'], List['JSON'], str, int, float, bool, None]

```

Recursive types need to use `ForwardRef`s at the right-hand side to reference the type alias before assignment. The ref `ForwardRef('JSON')` is not generic parameterized.

I’m trying to annotate an arbitrarily nested container with a specific element type or parameterized by a typevar. For example:

```python
T = TypeVar('T')

PyTree = Union[
    T,
    Tuple['PyTree[T]', ...], # Tuple, NamedTuple
    List['PyTree[T]'],
    Dict[Any, 'PyTree[T]'], # Dict, OrderedDict, DefaultDict
]

```

since the typevar `T` is inside a string `'PyTree[T]'`, which will be converted to a `ForwardRef`. However, it will be never evaluated because it is not assigned to a variable (`PyTree[T]` is not a valid identifier either). So I get:

```python
>>> TreeOfInts = PyTree[int]
>>> TreeOfInts
typing.Union[int, typing.Tuple[ForwardRef('PyTree[T]'), ...], typing.List[ForwardRef('PyTree[T]')], typing.Dict[typing.Any, ForwardRef('PyTree[T]')]]

```

`Union` only substitutes the first `T` to `int` which not in `ForwardRef`s, while the remaining `ForwardRef`s are remained as `ForwardRef('PyTree[T]')`.

As an alternative, I can specify the `int` type as:

```python
>>> TreeOfInts = Union[int, Tuple['TreeOfInts', ...], List['TreeOfInts'], Dict[Any, 'TreeOfInts']]
>>> TreeOfInts
typing.Union[int, typing.Tuple[ForwardRef('TreeOfInts'), ...], typing.List[ForwardRef('TreeOfInts')], typing.Dict[typing.Any, ForwardRef('TreeOfInts')]]

```

But this approach is not considered because it is impossible to cover all element types for function annotations. E.g:

```python
def tree_leaves(
    tree: PyTree[T],
    is_leaf: Optional[Callable[[T], bool]] = None,
    *,
    none_is_leaf: bool = False,
) -> List[T]: ...

```

Is it possible to make `ForwardRef` generic? Or is there any other way to define a generic recursive type (for function annotations)?

References:

- [Support recursive types python/mypy#731](https://github.com/python/mypy/issues/731)
- [Recursive Generic Type Support python/mypy#13693](https://github.com/python/mypy/issues/13693)
- [Type checking for pytrees google/jax#3340](https://github.com/google/jax/issues/3340)
- [Better Type Annotation Support for PyTrees metaopt/optree#6](https://github.com/metaopt/optree/issues/6)

---

<div class="post-metadata">

**Author:** ![guido](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/guido/32/21_2.png) [@guido](https://discuss.python.org/u/guido)\
**Post date:** [October 21, 2022, 4:48pm UTC](https://discuss.python.org/t/generic-typing-forwardref-to-support-generic-recursive-types/20168/2 "2022-10-21T16:48:26Z")

</div>

This looks like an interesting feature proposal.

A better venue for discussing it is probably [typing-sig@python.org](mailto:typing-sig@python.org) (join the list at [Mailman 3 Info | typing-sig@python.org - python.org](https://mail.python.org/mailman3/lists/typing-sig.python.org/)).