# Generics in metadata of \`Annotated\`

**URL:** <https://discuss.python.org/t/generics-in-metadata-of-annotated/62059>\
**Category:** Typing\
**Tags:** typing\
**Created:** [August 26, 2024, 11:11pm UTC](https://discuss.python.org/t/generics-in-metadata-of-annotated/62059 "2024-08-26T23:11:44Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![basnijholt](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/basnijholt/32/16882_2.png) [@basnijholt](https://discuss.python.org/u/basnijholt)\
**Post date:** [August 26, 2024, 11:11pm UTC](https://discuss.python.org/t/generics-in-metadata-of-annotated/62059/1 "2024-08-26T23:11:44Z")

</div>

Hi folks,

Context:  
I am developing a feature for the [GitHub - pipefunc/pipefunc: Lightweight function pipeline (DAG) creation in pure Python for scientific workflows 🕸️🧪](https://github.com/pipefunc/pipefunc) package which allows to easily construct pipelines (workflows) from a set of callables. Now I would like to make sure that types between the pipeline functions match. It supports N-dimensional map-reduce operations where outputs are stored in a `numpy.ndarray(..., dtype=object)` or equivalently `np.ndarray[Any, np.dtype[object]]`.

Problem:  
I want to be able to additionally keep the type of the elements of the object array in the type hint.

I thought I could use `typing.Annotated` for this and store the type of the elements in my object array in the metadata. For example, for strings I would have `Annotated[np.ndarray[Any, np.dtype[object]], str]`. Now the problem is that this is quite verbose and would like to use generics to do:

```python
T = TypeVar("T")
Array = Annotated[np.ndarray[Any, np.dtype(object)],T]

```

and then have folks use it like:

```python
def f(x: Array[int]) -> int: ...

```

However, as I discovered, generics don’t work in `Annotated`’s metadata.

So I came up with:

```python
T = TypeVar("T")

class Array(Generic[T]):
    def __class_getitem__ (cls, item):
        return Annotated[np.ndarray[Any, np.dtype[object]], item]

```

But this doesn’t work for mypy (expected I guess) and additionally, it is giving me a lot of trouble with evaluating `ForwardRef`s, so therefore I am looking for advice on how to best implement this.

Related:

- [allow `typing.Annotated` to be generic in its metadata · Issue #1386 · python/typing · GitHub](https://github.com/python/typing/issues/1386)

---

<div class="post-metadata">

**Author:** ![mdrissi](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/mdrissi/32/5427_2.png) [@mdrissi](https://discuss.python.org/u/mdrissi)\
**Post date:** [August 27, 2024, 7:46am UTC](https://discuss.python.org/t/generics-in-metadata-of-annotated/62059/2 "2024-08-27T07:46:19Z")

</div>

```python
class Array(Generic[T], np.ndarray[Any, np.dtype(object)]):
  def __class_getitem__ (cls, item):
        return Annotated[np.ndarray[Any, np.dtype[object]], item]

```

I’d expect something like that to work. You can add some tricks with TYPE\_CHECKING if you need to make class\_getitem extra invisible for some reason. Metadata is something type checkers ignore so to encode `np.ndarray[Any, np.dtype[object]]` just have that part as a parent class.

As for forward refs issue that’s one of common challenges of any runtime type manipulation. I ended up having code that did tricks/hacks of looking at parent stack frames to heuristically determine reasonable globals/locals to evaluate it. In general though forward refs cause challenges for bunch of libraries and easiest answer is don’t try to support every edge case involving them and come with ways to allow users to work around them like registering certain forward refs for later to be included in eval context.

---

<div class="post-metadata">

**Author:** ![basnijholt](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/basnijholt/32/16882_2.png) [@basnijholt](https://discuss.python.org/u/basnijholt)\
**Post date:** [August 27, 2024, 6:35pm UTC](https://discuss.python.org/t/generics-in-metadata-of-annotated/62059/3 "2024-08-27T18:35:47Z")

</div>

Yes that works, thanks a lot @mdrissi!

For future reference, the full implementation is in [Add parameter and output annotations to PipelineFunction class by basnijholt · Pull Request #6 · pipefunc/pipefunc · GitHub](https://github.com/pipefunc/pipefunc/pull/6) where I also verify the metadata at runtime.