PEP649 means that PEP563 will see *more* usage, not less, and will break runtime typecheckers

Typing
1

It seems PEP649 has been accepted! Hurrah, it’s great that string annotations might finally go the way of the dodo.

Unfortunately, until Python 3.12 is EOL, then approximately everyone is still going to writing code that is multi-version. And because PEP649 makes it possible to use bare forward references, then practically speaking that still means writing from __future__ import annotations everywhere, since on Python<=3.12, this is how you use bare forward references. Now we’re going to have stringified annotations everywhere!

So it seems that whilst the intention of PEP649 was to make runtime access to annotations better, it is in fact making things worse. At least for the time being.

…and this is making those of us who write runtime type checkers very sad. @leycec spotted this issue first, so we’ve recently started a discussion on the topic here. (It also touches on some other points, like speed, but I’ve highlighted above what I think is the most difficult concern for us to handle.)

I think us runtime-typing folks aren’t yet sure what might be an appropriate fix, but I wanted to make contact with the rest of you non-runtime-typing folks and start a parallel discussion over here.

1 Like
2

Apologies, I struggle to read text that leycec writes. Thanks for opening a discussion — if I understand correctly, the concern is something like:

  1. The PEP 649 behaviour is enabled by default in 3.13
  2. Someone writes a forward ref. When they run tests against 3.12, they get a NameError, so they add from __future__ import annotations to fix (or quote their annotations)
  3. If a user of that code is doing runtime type checking, and it’s one of the cases where eval works badly, that user is unhappy

Maybe I’m missing something obvious, but I’m confused how this is much worse than today. Step 2 plays out exactly the same as it would today. from __future__ import annotations is widely used enough that every runtime type checker needs to have basic support for it.

Anyway, I think it’s fairly likely that PEP 649 behaviour does not end up being enabled by default in 3.13. The SC at the time a) appeared unsure about that point and ended up polling, b) wanted it to land early in the lifecycle of 3.13. If it doesn’t land relatively soon, I wouldn’t be surprised if PEP 649 remains __future__ gated.

10 Likes
3

Interesting… thanks @hauntsaninja :slight_smile:

4

I’d just like to add that I maintain a (somewhat) runtime type checker, pyanalyze, and I am very excited about PEP 649, because it mostly makes the situation far better for runtime typing. Right now, when people use a forward reference in an annotation, they put it in a string (or they write from __future__ import annotation and automatically put everything in a string, which isn’t really any different). And when pyanalyze needs to resolve that annotation, it has to sort of guess at the right namespace in which to look up the name. Usually it’s easy (the function’s __module__), but sometimes it’s impossible (e.g., a type alias imported from another module). With PEP 649, people no longer have to stringify their annotations when writing forward references, and when pyanalyze accesses __annotations__, things will still work.

6 Likes
5

Actually, that’s not been my experience – in all my work I tend to take the opposite approach: given a stringified annotation (or even worse, a partially-stringified annotation), then I’ll tend to try typing.get_type_hints as a best-effort, and then either fail loudly or silently convert to Any, depending on what is most practical.

If so, would it be possible to backport that as a __future__ for earlier versions of Python as well?

Then we can at least tell everyone to just unconditionally add “always use from __future__ import pep649” and leave it at that, rather than the more-complicated
“”"
(a) if you’re on Python <=3.12 then don’t use from __future__ import annotations or everything will break, and please don’t use forward references either, and (b) if you’re on Python >=3.13 then do anything you like, and (c) if you want to write cross-version code then I’m afraid you’re stuck without forward references until Python 3.12 is EOL.
“”"

(Note that part (a) is what we already tell users, and it already comes up all the time.)

Jelle – I agree, I’m exciting about PEP649 as well! I’m aware of everything you’ve touched on. :slight_smile:

Whilst I’m here, I’ll copy-paste one of the more important messages from the runtime discussion thread I’ve linked:

  • To preserve compatibility, the new optional format parameter accepted by the inspect.get_annotations() and typing.get_type_hints() functions must default to FORWARDREF rather than VALUE. The current default of VALUE breaks compatibility by unpredictably raising unreadable NameError exceptions on accessing the __annotations__ dictionary, which prevents __annotations__ from being safely accessed, which (…waitforit) breaks compatibility.
  • It’s unclear how to safely modify type hints – or whether type hints even can be safely modified anymore. Pain points include:
    • How can two or more third-party packages safely modify __annotations__ and/or __annotate__() without destroying each other’s work?
    • The fact that __annotations__ and __annotate__() silently destroy each other is concerning. Ideally, one of these dunder attributes should be the definitive source of truth concerning annotations and not destroyable; the other dunder attribute should just be a cache that is destroyed and then silently recreated whenever the first dunder attribute is modified.
    • None of these dunder attributes should be destroyable, really. None values do not make sense for either __annotations__ and __annotate__(). Am I missing something? I’m probably missing something.

As a quick example of the kinds of things we don’t know the answer to right now, would something like this continue to be acceptable? Right now we’re not sure.

class MyClass:
    pass

foo = fn.__annotations__['foo']
if isinstance(foo, MyClass):
    foo.bar = baz

(note that in this case I own MyClass but not fn, so I’m being careful only to mutate the object that I own)

I’m sure all of these things can be understood, but right now those of us doing runtime stuff aren’t too sure how any of this is going to work, so it’s probably about time we reached out and changed that :slight_smile:

6

If so, would it be possible to backport that as a __future__ for earlier versions of Python as well?

Unless you have a time machine or the ability to control all computers, it is unfortunately impossible to backport PEP 649 to currently released versions of Python.

would something like this continue to be acceptable

By my read of the PEP this is mostly acceptable! The one quite marginal case is that if __annotate__ returns a dicts that refer to different objects, then mutations of objects within __annotations__ might not be visible if you try to inspect annotations with one of the other formats (because they will refer to different objects).

To preserve compatibility, the new optional format parameter accepted by the inspect.get_annotations() and typing.get_type_hints() functions must default to FORWARDREF rather than VALUE.

For the other quoted text there’s a lot that doesn’t line up with my reading of the PEP, e.g. it addresses the interactions between __annotations__ and __annotate__ pretty clearly; the thing the quoted text is complaining about is explicit design to keep basic semantics of __annotations__ backwards compatible. Most of the point of PEP 649 is that you won’t get NameError’s in many of the cases you formerly would, typing.get_type_hints is perfectly happy to give you NameError today, etc. But…

…I think that the best way to answer most of these questions is to have an implementation to play with. Unfortunately, we currently do not. This is presumably why the old SC wanted it to land early in the 3.13 lifecycle.

7

I suspect what was meant is “all bugfix-maintained major release series’ of Python,” (so that the next 3.11 and 3.12 release would contain the feature) which is the usual understanding of how backports would work, in the absence of a time machine :slight_smile:

On the purely technical level, I think this is somewhat plausible for 3.12, since the PEP 695 implementation already contained the key underlying “annotation scope” feature. I think it’s out of the question for 3.11, given the invasiveness of that feature.

But in terms of backport policy, it seems out of the question for either 3.11 or 3.12, because we backport bugfixes, we don’t backport new features. And there are good reasons for this, both because people rely on the stability of bugfix releases, and because specifying “Python 3.12” should be enough to clarify the feature-set; code written for 3.12.4 should not use features that don’t exist on 3.12.3.

But I also agree that all of this discussion is somewhat putting the cart before the horse, in the absence of a Python 3.13 implementation of PEP 649 proposed for merge.

11 Likes
8

Hello! I’m sitting at the PyCon sprints with @Jelle, who is now working on implementing PEP 649 for Python 3.14 (since it didn’t happen in time for Python 3.13.) We wanted to follow up on this thread to ensure the concerns raised here (and in the linked discussion at The First Annual Meeting of the Extraordinary League of Runtime Typers · beartype/beartype · Discussion #335 · GitHub) are at least addressed clearly.

I’ll attempt to summarize each concrete concern that I was able to glean from reading this thread and the linked GitHub discussion (in quotes to clarify that these are my attempted summary, not my own assertions), and offer my thoughts on each:

  1. “PEP 649 will increase usage of PEP 563 in the short term, because Python library authors will start expecting bare forward references to work, and the only way to make them work on older Python versions is PEP 563.”

This may be true; it’s hard to predict the extent of this effect. If someone wasn’t willing to use PEP 563 to get bare forward references in Python 3.12 or 3.13 today, it’s not clear why the addition of bare forward references support in 3.14 would suddenly change their mind.

It seems the only concrete proposal that’s been made to address this is to backport PEP 649 to older Python versions. I think there is zero chance of this, for good reasons.

If we introduce PEP 649 under a __future__ import, and don’t make it default until the oldest supported Python has that __future__ import, that would minimize this effect: bare forward-refs would continue to be an experimental opt-in-only feature until PEP 563 is no longer needed for them at all. But this comes at the cost of lengthening the whole transition period (a LOT – we wouldn’t be able to make PEP 649 the default behavior until 2029); everyone would need to consider three possible annotation semantics for the next several Python feature releases. Do we want to rip off the band-aid with more short-term pain, or spread the pain out over years to come? (Related recent discussion in PEP 649: Deferred evaluation of annotations, tentatively accepted - #44 by Jelle).

  1. “PEP 649 should default to FORWARDREFS format, rather than VALUES format, to avoid causing NameErrors in runtime-checking decorators when people start using bare forward-refs in their annotations.”

This is framed as a compatibility concern, but I think compatibility is precisely the reason we can’t do this. Existing code assumes that __annotations__ contains normal Python objects, without replacing unbound names with ForwardRef objects. Changing that behavior should be opt-in.

Existing code does not assume that it can use bare forward refs in annotations without a __future__ import. If someone tries to start using bare forward-refs in their annotations, and that breaks with a runtime-checking decorator, that’s no different from the behavior today, so it’s not a compatibility break.

The runtime checking decorators can (and should) request annotations explicitly using FORWARDREFS format (under PEP 649) if they want to enable bare forward refs.

  1. “PEP 649 FORWARDREFS mode will be slow, and there’s no built-in caching of the results.”

This is true. I think that we should provide memoization of FORWARDREFS-style annotations, but probably as a function in the inspect module, rather than built-in to the object model.

With the benefit of some experience and real-world use cases, I also expect that we can better optimize the FORWARDREFS mode.

  1. “Why weren’t we consulted?”

I don’t want to get too derailed on this meta-issue, but I do want to avoid it recurring in the future, if possible. I admit that I don’t really understand the implicit expectation here. There were lengthy public discussions of the details of PEP 649 on this forum over a period of months prior to its acceptance. Anyone who wants to influence the direction of Python is always welcome to keep an eye on the PEPs category (which is not too noisy) and participate in discussions here. It’s also fine not to do that, of course, but that’s a choice not to be involved in the decisions.

(There are also some questions about the behavior of modifying annotations at runtime; I’ll leave these for a separate post since this one is long enough already.)

10 Likes
9

Regarding runtime mutation of __annotations__ and __annotate__, let me attempt to summarize the design intent of PEP 649 (maybe Larry will chime in to correct me if I get it wrong):

The canonical runtime representation of the annotations is VALUES form, which is why that’s the only form stored permanently in __annotations__. FORWARDREFS is just an (ephemeral) partial rendition of VALUES, in the case where some names are not (yet) bound. If you have once successfully created the VALUES form, that means all used names have now been defined, and there’s no possible remaining need for FORWARDREFS; it would be identical to VALUES anyway.

This is why the sample implementation of inspect.get_annotations() implements FORWARDREFS by first checking if o.__annotations__ already exists, and just returning it if so, and only if not trying to call the __annotate__ method with fake-globals.

There aren’t rules about how or when you can modify __annotations__. Once it exists for an object, it’s canonical for that object, and you can freely change it however you like (again, just like today). (When and whether you should is a different question; out of scope here.)

If you set __annotate__ to a new value for an object, you are declaring your intent to totally replace all annotation information for that object, and so __annotations__ is then cleared as well (to be later likely repopulated by the return value of your new __annotate__ function.)

Setting __annotate__ to None is intended to avoid a split-brain situation if someone sets a new value for the __annotations__ dict, where __annotations__ is something custom but calling __annotate__ again gives the original values. I do personally think it’s a bit odd that in PEP 649 when you set __annotations__ to an entirely new value, that sets __annotate__ to None, but if you mutate __annotations__, that doesn’t. We attempt to avoid split-brain in the one case, but not in the other, even though the line between the cases is quite subtle (why should o.__annotations__.clear(); o.__annotations__.update(new) preserve __annotate__ while o.__annotations__ = new sets it to None?) Given that we can’t consistently prevent the possibility of split-brain, I think it would be more predictable not to try, and to never set __annotate__ to None.

2 Likes
10

Thank you for the response!

I think everything you’ve said makes sense to me. In particular I’ve very glad to hear that mutating __annotations__ will continue to be safe. (And moreover, that mutating the objects inside __annotations__, e.g. o.__annotations__["foo"].mutate(), which though terrifying, should also be okay.)

On the “being consulted” – no expectation here at all, this thread is just me (us?) checking in for exactly this purpose :slight_smile:

I think once there’s a draft implementation of this then we should try running the jaxtyping/beartype/typeguard/pydantic/etc test suits and see how they fare!

1 Like
11

There is now; see my post at PEP 649: Deferred evaluation of annotations, tentatively accepted - #60 by Jelle. I spent a bit of time trying to get pydantic running, but it doesn’t work with Python 3.14 for unrelated reasons, and I haven’t yet had time to look at the others. I’m tracking the implementation of PEP 649 in Implement PEP 649 · Issue #119180 · python/cpython · GitHub.

3 Likes