Annotation string references in class scope in conformance tests

Typing
1

While trying to pass the conformance tests for Zuban, I have encountered a few fundamental issues, this is #4 of 4.

String reference annotations (forward references) in class scopes behave in weird ways in the conformance tests. I’m not saying it’s wrong, but it is not specified as far as I know.

The only part of the spec that mentions forward annotations is this:

The string literal should contain a valid Python expression (i.e., compile(lit, ‘’, ‘eval’) should be a valid code object) and it should evaluate without errors once the module has been fully loaded. The local and global namespace in which it is evaluated should be the same namespaces in which default arguments to the same function would be evaluated.

class ClassC:
    ...

class ClassD:
    ClassC: "ClassC"  # OK

    ClassF: "ClassF"  # E: circular reference

    str: "str" = ""  # OK

    def int(self) -> None:  # OK
        ...

    x: "int" = 0  # OK

    y: int = 0  # E: Refers to local int, which isn't a legal type expression

    def __init__(self) -> None:
        self.ClassC = ClassC()

assert_type(ClassD.str, str)
assert_type(ClassD.x, int)

Ideally a specification will arrive after or at least in conformance with PEP 749 (Deferred evaluations of annotations), but I haven’t gotten anything of value out of 3.14:

>>> class X:
...     def int(self) -> None: ...
...     x: "int" = 0
...     y: int = 0
...     
>>> import annotationlib
>>> annotationlib.get_annotations(X, format=annotationlib.Format.VALUE)
{'x': 'int', 'y': <function X.int at 0x75ce1e260300>}

The problem here for me is how “int” and “str” are resolved. Mypy/Zuban do this in a different way than Pyright. The conformance tests are currently mirroring Pyright’s behavior. I propose to allow both ways for now. My gut tells me that PEP 749 will sway into the direction of Zuban, but if it does not that’s also fine. I just don’t want to implement something that will be changed again. Maybe the PEP 749 implementors can have a word here as well (maybe @Jelle?), this might even have been implemented, I just don’t have an easy way to run the cpython master branch here.

If people agree with this proposal, I will add a PR to the typing repository and allow both behaviors in the conformance tests for now. Once the deferred evaluation of annotations is done, we can use that logic to implement that logic in the conformance tests.

2

I wrote these tests, and I agree that they overstep what’s in the spec currently. The typing spec is not very clear on this point, but I was trying to rely on runtime behaviors (the results of typing.get_type_hints) to inform the correct type checker behavior. The problem is that the runtime behavior is complex, not well documented, and has changed subtly over time.

I think we have a few options here:

  1. Leave the spec as is. This would leave the forward-declared type annotation resolution behavior under-specified. Update the conformance tests to reflect the fact that the behavior may vary across type checkers.
  2. Modify the spec to make it clear that type checkers should resolve forward-declared type annotations in the same manner as typing.get_type_hints at runtime. Update the conformance tests to reflect this.

Are there other options I’m missing?

@Jelle has significant expertise in this space, so I’d look to him for advice.

I’m fine with relaxing the conformance tests in the meantime. Thanks for offering to create a PR.

3

I feel string annotations should be evaluated in the same way as they would be evaluated if they were not strings in 3.14. This means that if the name is defined anywhere in the class scope, the definition in the class scope takes precedence. Therefore, most of the “OK” examples in the test you quote should be errors.

3 Likes
4

@Jelle Thanks for weighing in here. Do you think I should change the conformance tests in that way now or should we just wait until PEP 749 is fully implemented and handling these kinds of cases?

I’m happy to help with a PR to make this clearer in the spec, but I only want to improve the spec if we are in agreement that we want to change it in the direction you mentioned. Otherwise I can simply create a PR that allows both ways for now.

5

We should change the spec. PEP 749 is already fully implemented, but string annotations are here to stay for the foreseeable future: people writing 3.14+ code have very little reason to use them, but there’s lots of existing code using them and we should make sure type checkers interpret them in a consistent, sane way.

1 Like