Finding edge cases for PEPs 484, 563, and 649 (type annotations)

PEPs
1

Previously, the steering council put out a call for someone to please step forward and help us gather information around type annotations, how they are declared, and to help us understand the problem space. Unfortunately no one came forward, and so we are now putting out a more concrete request of the general typing community to help us make a decision around PEP 484, PEP 563, and PEP 649.

Specificially, what we would like a list of cases where declaring type annotations do not work under (at least) one of the three (proposed) ways of declaring annotations:

  • Executed at runtime (PEP 484)
  • Strings, ala from __future__ import annotations (PEP 563)
  • Deferred evaluation (PEP 649)

For each edge case, we would like a small code sample and which of the three approaches it doesn’t work for. As an example, the following does not work for PEP 484 due to the lack of support for forward references:

class BinaryTree:
    value: Any | None
    lhs: BinaryTree | None
    rhs: BinaryTree | None

At the end of this, what we would like is to have a list of edge cases that fail for (at least) one of the approaches so we can understand the compromises we would be asking of the community if we were to accept any of them as the default/only solution to declaring type annotations. Once we have such a list we believe we can continue on with our discussions on the topic (although this by no means is meant to suggest it will directly lead to a decision about these PEPs).

7 Likes
2

Edge case: forward references

Referencing a class within its own type annotations.

class BinaryTree:
    value: Any | None
    lhs: BinaryTree | None
    rhs: BinaryTree | None

Fails for …

  • Executed at runtime (PEP 484)
1 Like
3

This variation also causes issues in PEP 649:

from dataclasses import dataclass

@dataclass
class User:
   name: str
   friends: list[User]

This one causes problems for PEP 649, because the @dataclass decorator accesses the class’s annotations before the name is bound. This is discussed in Recursive dataclasses · Issue #2 · larryhastings/co_annotations · GitHub.

Possible solutions under PEP 649 include:

4 Likes
4

A case where PEP 563 fails for runtime analysis:

def class_maker() -> list[type]:
    class SomeClass:
        def make_instance() -> SomeClass: ...
    return [SomeClass]

SomeClass is defined in a local namespace, so it’s inaccessible (by name) once the function returns. Inside the function, it is possible to fetch if the type checker is willing to use sys._getframe, but determining whether it should do so is itself tricky. This also applies to any local imports.

1 Like
5

Edge case: Resolve type forms without eval

from __future__ import annotations
from trycast import trycast  # pip3 install 'trycast>=0.6.1'
from typing import TypedDict

class Point2D(TypedDict):
    x: float
    y: float

class Circle(TypedDict):
    center: Point2D

maybe_circle = trycast(Circle, {"center": {"x": 0.0, "y": 0.0}}, eval=False)
if maybe_circle is not None:
    print("It's a Circle!")

The above raises: trycast.UnresolvedForwardRefError: trycast does not support checking against type form <class '__main__.Circle'> which contains a string-based forward reference. trycast() cannot resolve string type references because it was called with eval=False.

If you comment out the from __future__ import annotations line then no error is raised and "It's a Circle!" will be printed instead.

Fails for …

  • Strings, ala from __future__ import annotations (PEP 563)

Explanation

trycast() is a runtime type checker. Calling trycast(SomeType, some_value) does return some_value if it is assignable to SomeType (using PEP 484 assignability rules) and does return None if it is not.

trycast() works by inspecting the type form object it is passed at runtime to determine its shape. Then it checks the value it is passed to see whether it fits that shape.

The example above makes a call to trycast(Circle, a_value, eval=False):

  • The eval=False argument tells trycast() that it is not permitted to use the eval() function at all, which users may want to avoid for security or performance reasons.
  • When trycast() inspects the type form Circle, it sees it is a TypedDict. To determine the key & value types of a TypedDict, there are two strategies:
    • Since eval=False, Circle.__annotations__ is inspected. This will return {'center': ForwardRef('Point2D')}, which contains an unresolved stringified forward reference, which was stringified only because from __future__ import annotations was in effect. The only way to resolve a ForwardRef('Point2D') object is to use eval(), which is forbidden here because eval=False. So a UnresolvedForwardRefError is raised instead.
    • If we instead used eval=True, typing.get_type_hints(Circle) would be inspected instead. This function automatically expands ForwardRef(...) objects using eval().

In summary, if stringified type annotations become mandatory, it will also become mandatory for runtime users of type annotations to use eval() to resolve those annotations to actual type form objects. Yet some codebases do not want to support eval() for security or performance reasons, and thus those codebases will not be able to use type annotations at runtime under most circumstances.

3 Likes
6

Properly typing JSON seems to still be problematic:

1 Like
7

Can you please provide “a small code sample and which of the three approaches it doesn’t work for”, as per Brett’s request at the top of the thread?

8

Not sure if this message is for me or not, but no I can’t. The issue on GitHub was opened 6 years ago, some progress seems to have been made, but AFAICT the problem is not solved.

Brett is the person who opened the issue, so I guess he knows a lot more than me about the details.

9

The examples below are from running co_annotations on internal codebase that uses runtime introspection of type annotations for serializing and deserializing python objects from JSON. I would be happy to test larger portion of my codebase, but co_annotations branch uses an alpha version of 3.10 so a lot of dependencies I use lack wheels/compatibility (even including typing-extensions) with that alpha. If co_annotations can update to a release version of 3.10 I can try running all unit tests swapping annotations with co_annotations.

Failure Cases for PEP 649:

@dataclass
class NodeA:
    b: list[NodeB] = field(default_factory=list)

class NodeB:
    ...

This crashes with an error message of TypeError: 'b' is a field but has no type annotation. There is no recursive relationship between NodeA and NodeB. NodeB is defined later in file which is allowed with PEP 563 but does not work here with 649. The field is essential. If you change the code to this,

@dataclass
class NodeA:
    b: list[NodeB]

class NodeB:
    ...

then it works with 649.

The other PEP 649 error,

from __future__ import co_annotations

from typing import NamedTuple


class Person(NamedTuple):
    name: str
    age: int

This fails with an error message of TypeError: __co_annotations__ must be set to a callable or None. Not sure why NamedTuples are an issue. Looking at the error message I think this is a minor bug involving these two lines, 1 and 2.

I also encountered recursive data class issue mentioned above. Those are only 3 issues I found for code I could test.

I mainly have 649 failure cases, because I use existing 563 heavily so I already handle/workaround 563 weird cases. An example workaround I use is handling mutually recursive types which I think all options struggle with and unsure it’s even possible to support. If you have code like this,

def inspect_types(cls):
  annotations = get_type_hints(cls)
  ...

@inspect_types
class A:
  x: B

@inspect_types
class B:
  y: A

it fails at runtime for any option. I workaround this by having having annotation inspection be done lazily by decorator. Also there’s fallback workaround of just use decorator manually afterwards (A = inspect_types(A), B = inspect_types(B)). This lazy evaluation does require using sys.get_frame and keeping track of right namespace.

Properly typing JSON seems to still be problematic:

The JSON type issue is unrelated to this. pyright/pyre support json type. Runtime wise defining json type is simple. Issue there is mypy does not handle recursive type aliases, but that’s a mypy specific issue and runtime annotation semantics should not affect things.

10

The easy workaround for the “variation” cited by Jelle above is to skip the decorator syntax and call dataclass yourself, after the class is bound.

from dataclasses import dataclass

class User:
   name: str
   friends: list[User]
User = dataclass(User)

This works fine with PEP 649 active.

11

Are you asserting that these are unfixable edge cases with the design of PEP 649, or are these simply bugs you’re reporting in the wrong place?

12

mod1.py

from __future__ import annotations
from dataclasses import dataclass

class A: ...

@dataclass
class B:
    a: A

mod2.py

from dataclasses import dataclass
from typing import get_type_hints

from mod1 import B

@dataclass
class C(B):
    x: int

print(get_type_hints(B.__init__))  # works fine
print(get_type_hints(C.__init__))

Fails for PEP 563

Traceback (most recent call last):
  File "mod2.py", line 11, in <module>
    print(get_type_hints(C.__init__))
  File "/home/tmk/.conda/envs/py10/lib/python3.10/typing.py", line 1836, in get_type_hints
    value = _eval_type(value, globalns, localns)
  File "/home/tmk/.conda/envs/py10/lib/python3.10/typing.py", line 324, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "/home/tmk/.conda/envs/py10/lib/python3.10/typing.py", line 688, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'A' is not defined

Though you can make this work with vars(sys.modules[B.__module__]) passed to get_type_hints().

13

Larry, that feels like an unnecessary putdown.

1 Like
14

Brett can speak for himself, but the last note from him on that thread that I found said:

“I’m fine with tossing this whole idea out”

But anyway, the limitation there is that there’s no way (at least with MyPy?) to define recursive type – so that’s a limitation, but I don’t know that it’s the kind of Edge Case that is being talked about here.

15

My understanding of this topic is existing behavioral differences between the different peps. So I locally checked out 649 branch and ran portion of my tests to see differences. NamedTuple one looks like a minor bug. The dataclass one looks similar to the other dataclass issues. My intent here was mainly to document situations where runtime type behavior may change.

I will also clarify and say while I use runtime type inspection heavily to my knowledge both 563/649 edge cases can be handled with manual string quoting so there is always a fallback solution. As long as that works, I find it reasonable having a few edge cases need to do manual string escaping.

16

Then it seems you’ve misunderstood the topic. This is a call for people to describe what Brett called “edge cases”: situations where the technologies described in the PEPs don’t correctly handle a (valid) use case for annotations. Brett wants us to consider the design of each PEP, and what it does and doesn’t permit; “existing behavioral differences” suggests an examination of the current implementations of each of the PEPs, which is not the same thing. Bug reports, where the behavior is clearly not intended, are outside the intended scope of this discussion.

17

I don’t think there’s much of a distinction between the two. Design and implementation for this issue strongly overlap. Some of the solutions mentioned for certain bugs like suppressing name errors or lazy desciptors it is unclear to me whether that’s a design choice or an implementation detail.

As an explicit example are the dataclass examples bug reports or design issues? That’s unclear to me. The only example I listed I’d consider a likely implementation detail is NamedTuple one which I mentioned in first comment.

18

Assume PEP 649 is perfectly implemented for any example left here and there’s a scenario that does not work as coded. Workarounds are just illustrating how to deal with the issue today and are simply a way to help illustrate further why an edge case is problematic.

If you’re not sure about PEP 649 semantics, I’m sure @larry and other folks can clarify.

That’s covered by my opening edge case around recursive types.

19

When bugs in specific PEP implementations do come up here (natural), recognizing them for what they are (bugs out of spec vs the PEP) and linking to the place they are being tracked for the implementation would be good.

PEPs aren’t perfect, neither are implementations. Both may have potential issues to address that these conversations can reveal.

2 Likes
20

Edge case: Cross-module dataclass inheritance breaks get_type_hints

This is from Issue 45524: Cross-module dataclass inheritance breaks get_type_hints - Python tracker

If bar.py contains:

from __future__ import annotations

import foo
import dataclasses
import typing

@dataclasses.dataclass
class B(foo.A):
  pass

print(typing.get_type_hints(B.__init__))

And foo.py contains:

from __future__ import annotations

import collections
import dataclasses

@dataclasses.dataclass
class A:
  x: collections.OrderedDict

Then running python bar.py gives an error:

Traceback (most recent call last):
  File "...\bar.py", line 11, in <module>
    print(typing.get_type_hints(B.__init__))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\Lib\typing.py", line 2005, in get_type_hints
    value = _eval_type(value, globalns, localns)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\Lib\typing.py", line 336, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\Lib\typing.py", line 753, in _evaluate
    eval(self.__forward_code__, globalns, localns),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
NameError: name 'collections' is not defined

Fails for …

  • PEP 563. Works if the __future__ statements are removed.
  • Works with PEP 649.
3 Likes