Issue with accessing annotations as documented in 3.14+

Python Help
1

The datamodel says I should expect annotations to be available as either object.__annotation__ or object.__annotate__,: 3. Data model — Python 3.14.0 documentation

This appears to be wrong, and I’m not sure if it’s a documentation error or a released python version error, but since the pep text matches the data model text…

>>> class A:
...     x: int
...
>>> dir(A)
['__annotate_func__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__firstlineno__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__static_attributes__', '__str__', '__subclasshook__', '__weakref__']

__annotate_func__ is available, isn’t documented in the datamodel, but appears to correspond with __annotate__'s documented behavior. This function raises NotImplemented for 3 (Forwardref) and 4 (String) (see documentation of values here)

which is significantly less compatible than was promised with existing codebases using from __future__ import annotations

Not sure what the path forward here is on this one.

2

Impacting an existing codebase, results in it silently ignoring the annotation when attempting to access as documented as if annotations are missing.

3

My understanding is that the intention is for annotationlib.get_annotate_from_class_namespace to be used if you need to retrieve the __annotate__ function from a class namespace, annotationlib.call_annotate_function to be used to call these retrieved functions and annotationlib.get_annotations to be used in most other cases.

I believe the presence of __annotate_func__ in the class namespace is considered an implementation detail that may change in future, so the function from annotationlib is intended to be the stable way to retrieve the function. You’re not supposed to access it directly.

If you have the object you’d use get_annotations(A, format=Format.FORWARDREF) if you only have the namespace, such as in a metaclass you need the combination of the other two functions. The implementation of Format.FORWARDREF and Format.STRING exists within annotationlib.call_annotate_function - the __annotate__ functions themselves don’t implement these formats.

4

That matches neither the datamodel documentation nor the PEP. If that behavior had been brought up in the pep, I would have had significantly more feedback, and I hope that’s incorrect.

1 Like
5

You should use annotationlib to get annotations.

The class method __annotate__ and the class property __annotations__ should still be accessible on the class object, but not necessarily in the class’s dict (because they’re properties).

They are also not listed in the class’s dir() though, which probably should be a bug.

6

PEP 749 says in the specification section that:

The .__annotate__ and .__annotations__ attributes on class objects should reliably return the annotate function and the annotations dictionary, respectively, even in the presence of custom metaclasses.

I understand that annotationslib exists, but in the case of a library that is already having to manually access the supported annotation locations for older python versions, it makes no sense to import it when there’s what is supposed to be a supported attribute right there.

1 Like
7

You can access the attribute, but not from the class namespace. There’s a related bug in older Python with metaclasses and accessing __annotations__ that this is intended to avoid.

PEP-749 covers this in annotations and metaclasses where it’s mentioned as:

We considered several solutions but landed on one where we store the __annotate__ and __annotations__ objects in the class dictionary, but under a different, internal-only name. This means that the class dictionary entries will not interfere with the descriptors defined on type.

This internal-only name is __annotate_func__.

I’ll note that I’d also like to be able to deal with annotations without importing annotationlib but it’s not really possible in 3.14.

8

I read that, but that doesn’t actually match what the PEP says here.

In fact, right below that, it’s a rejected alternative to tell users not to use __annotate__ / __annotations__, and the specification sections says that those should always reliably get the annotations, even in the presence of metaclasses.

1 Like
9

Yes, you can reliably use .__annotate__ or .__annotations__ but you can’t use ns.get("__annotate__") or ns.get("__annotations__") as the specification says:

Users should not access the class dictionary directly for accessing annotations or the annotate function; the data stored in the class dictionary is an implementation detail and its format may change in the future. If only the class namespace dictionary is available (e.g., while the class is being constructed), annotationlib.get_annotate_from_class_namespace may be used to retrieve the annotate function from the class dictionary.

You still need call_annotate_function if you need Format.STRING or Format.FORWARDREF output though as that is where those formats are implemented.

10

This whole thing feels like a mess of conflicting information. The pep as it was accepted, and datamodel where it is cannonically documented indicate that the annotate function takes those formats, why would a helper need to be called with the same format?

1 Like
11

The helper function contains the actual implementation of those formats. annotate functions are allowed to raise NotImplementedError for formats they don’t support, and the generated functions don’t support the STRING or FORWARDREF formats.

They support VALUE_WITH_FAKE_GLOBALS which is used to indicate that call_annotate_function can call them in the fake globals namespace used to implement the other formats.

Yes it’s convoluted, I’m not going to argue that.

12

I am confused what you are seeing as an issue here. If I run this example in Python 3.14 and then access A.__annotations__ I get the annotation dict. Yes, this is not inside of dir[1]

A.__annotate__ also works - it just also isn’t in the dir output. (IDK why __annotate_func__ is the internal backing function, but I am sure you could dig up the PRs for this). Either way, __annotate__ and __annotations__ are the publicly available interface.

  1. Before a.__annotations__ is accessed for the first time, afterwards it’s cached. ↩︎

2 Likes
13

Removing __annotate__ was a change in one of the pre-releases (looking at the date I think the first beta?). It was done to resolve this issue: [3.14] annotationlib - calling `get_annotations` on instances gets an unexpected error · Issue #132261 · python/cpython · GitHub

It has always been the case for 3.14 that the generated annotate functions don’t support Format.STRING or Format.FORWARDREF. I think Format.VALUE_WITH_FAKE_GLOBALS was added in a pre-release and changed the enum values (so you can’t rely on the actual integer values for the enum either).

None of this is to say I’m particularly happy with the state of the current implementation.

In cases where you have to create an __annotate__ function you either have to recreate any logic used to collect the annotations you plan to use within the new annotate function (see this implementation for dataclasses), or you give up and put VALUE or STRING annotations in __annotations__ and assume people already have tooling to deal with string annotations (what I did for my own dataclass-like).

I posted one idea as an attempt to try to improve this situation but there was no interest so I can only assume that needing to generate these is rare, and tools that should be doing so may not have got around to it yet. I know attrs currently just sticks FORWARDREF annotations in __annotations__ which is slightly better than what dataclasses was doing.

14

Really not a fan. Having to import a helper, and it not being in the class dict both impact good ways to try and handle multiple python versions.

It really feels worse than the old future annotation after all of the changes that pushed it further and further from what was originally proposed, but I’ll deal with it.

The documentation on this is also horrifically bad as it explains almost none of it. And if you need forward refs, you’re out of luck unless you actually do exactly what was listed as a rejected option in the pep, don’t directly use the annotate function, and use the helper function.

1 Like