Two polls on how to revise PEP 649

Howdy howdy. Let’s talk about PEP 649, Deferred Evaluation Of Annotations Using Descriptors. I think the PEP has an excellent chance of being accepted. However, the PEP needs to be revised, and there are two particular points on which I can’t quite make up my mind. I figured it was best to raise the issues here and see what the community thinks. To that end, I’ve added two polls to the end of this message. Please vote in both!


Let’s start with the simpler question. Should 649 specify that the __co_annotations__ attribute is a supported public interface? Or should it specify that __co_annotations__ is an internal implementation detail? I can see both sides.

On the one hand, Python tends to make these things public. And __co_annotations__ is a reasonable API, one that users would have no trouble working with.

On the other hand, I can’t contrive a scenario in which anyone would want to write their own __co_annotations__ function. For that to happen, there would have to be users who:

• create their own Python code objects and functions by hand, not using the compiler,
• which they’d want to annotate,
• and they’d want to examine the annotations of those objects at runtime,
• and those annotations would have to be lazily evaluated.

I don’t think such a user exists. (But if you are such a user, please! speak up!)

Another consideration: if we declare __co_annotations__ is a private implementation detail today, that gives us the flexibility to change it as needed later, allowing us to more easily accommodate unforeseen future needs. And of course, if we declare it private now, we could simply change our minds and make it public in the future.


Here’s the second, more complicated question. In order to accommodate users who want annotations as strings, we’re going to modify inspect.get_annotations and typing.get_type_hints (which I will collectively call library functions computing annotations) to explicitly support that feature. By default, both functions will function as they did in the initial 649 prototype: when the user asks for the annotations (or type hints) for a function that has a __co_annotations__ function, it’ll return the real values. We’ll add a new parameter to the library functions computing annotations that says “return the annotation values as strings”.

How will that work? Initially we planned to use Carl Meyer’s “Stringizer” proposal. In this approach, we’d rebind the __co_annotations__ code object to a custom empty __globals__ dict that supported calling a callback for every undefined symbol (e.g. collections.DefaultDict.__missing__). That callback function would return a new “Stringizer” object for every missing symbol, which would roughly reconstruct the original string of the annotation. The library function computing annotations would then simply call this rebound annotations function normally, using the real Python interpreter. Our rebound function would compute and return the annotations dict, but all the values in the dict would be “Stringizer” objects. The library function would then walk through the resulting dict, turning these “Stringizer” values into real strings, and finally return the dict with those strings. The result would be very similar to the stringized annotations produced by PEP 563’s from __future__ import annotations.

How does this “Stringizer” perform its magic? First, every “Stringizer” is created as a mock replacement for a symbol; we tell the object the name of the symbol it’s replacing. Then, the “Stringizer” implements every dunder method exposed by an object called when evaluating an expression, returning an object that represents performing that operation on that name.

Here’s a high-level hand-wave-y example of how this works. Let’s say you have an annotation that reads mymodule.MyType[Int], and you want to back-compute the string based on the bytecode. When running the bytecode, Python would first ask the custom globals dict for the symbol 'mymodule'. The custom globals dict would return a “Stringizer” that knew its name was 'mymodule'. Then, Python would evaluate the '.MyType', which means it would next call the __get_attribute__ method of our “Stringizer”. Our “Stringizer” would override that method and return a new second “Stringizer” object that knew its name was 'mymodule.MyType'. Third, Python would ask our fake globals for Int, which would return a third “Stringizer” that knew its name was 'Int'. Finally Python would evaluate the [Int], which would turn into a __getitem__ call on the second “Stringizer”, passing in the “Stringizer” named 'Int'. This would return a new fourth “Stringizer” that knew its name was 'mymodule.symbol[Int]'. Finally, Python would store this fourth “Stringizer” as the value in the annotations dict.

I’ve prototyped this, and I’m delighted to report that the “Stringizer” works fine. After all, all __co_annotations__ functions do is evaluate a bunch of expressions, then build a dict out of them and return it. And Python calls dunder methods to compute nearly everything in an expression, meaning the “Stringizer” approach really can handle almost everything.

But there’s an important exception, a part of the language where Python doesn’t call dunder methods when computing an expression: flow control. Although the Python interpreter does consult objects at certain points during flow control—asking an object for its true-ness or false-ness, or asking for an iterator over the object—the actual computation of the expression is done internally in the Python interpreter, out of the “Stringizer”'s control.

Examples of flow control used in expressions:
• Short-circuiting or
• Short-circuiting and
• Ternary operator (the if / then operator)
• Generator expressions
• List / dict / set comprehensions
• Iterable unpacking

The good news was, nobody did this stuff in type hints. So the “Stringizer” approach seemed like it should work great for the Python static typing community in practice.

But that’s changed with the acceptance of PEP 646, “Variadic Generics”. This adds TypeVarTuple objects to Python, which are designed to be unpacked in type hints using iterable unpacking. At the bytecode level this turns into a tiny loop, iterating over the values yielded by the TypeVarTuple being unpacked. This is the one sticking point that prevents the “Stringizer” from being viable.

What should 649 do about this? From a high level, it seems like there are two ways we could go.

First, there’s an approach I call “Hard-Code The Stringizer”. This rests on two assumptions, which I think are true, and which I’m hoping the community will confirm (or, regrettably, debunk). First, the only people who care about turning their annotations back into strings are people using annotations for type hints. And second, unpacking a TypeVarTuple is the only type of iteration we’ll ever see in an annotation. If those are both true, we can simply hard-code the “Stringizer” to assume that one use case. Unpacked TypeVarTuple objects simply return an “unpacked” form of themselves, which really just prints an asterisk in front of its name in its repr. We code the “Stringizer” so it assumes any call to __iter__ is unpacking a TypeVarTuple, and return an iterator that yields one of these objects. And if that’s good enough, we’re done, the “Stringizer” becomes viable again.

If “Hard-Code The Stringizer” won’t work, the next step is an approach I call “A Simple Custom Bytecode Interpreter”. And, yes, this involves writing a bytecode interpreter. Maybe that sounds awful—but I’ve prototyped it and it’s not really all that bad. The key insight is that __co_annotations__ functions generated by Python are simple: they evaluate a bunch of expressions, then build them into a dict, and finally just return that dict. Our custom bytecode interpreter would only have to support one statement, return; all the other work is done inside expressions. In practice it’s not that much more work than the “Stringizer” was in the first place; it does much the same thing, it just does its work based on bytecode dispatch rather than in dunder methods.

I prototyped this too. It works fine, and isn’t even that much longer than the “Stringizer” prototype. Performance is presumably much worse, but hopefully nobody is querying the stringized annotations of objects in performance-sensitive code.

But then again! Let’s quickly revisit my first question, about __co_annotations__ functions being public or private. If we allow users to write their own __co_annotations__ functions, they could potentially set __co_annotations__ to any Python function they like, and the function could use any Python statement they wanted. If the function they write uses bytecodes my simple custom bytecode interpreter doesn’t implement, it would simply fail, presumably noisily.

This suggests a possible third approach, “A Full Custom Bytecode Interpreter”, where our bytecode interpreter would implement every bytecode and actually do a better job of simulating running the __co_annotations__ function. But I assert this isn’t viable in practice. My first two proposals rely on the fact that __co_annotations__ functions don’t do any real work. They’re predictable and simple. So creating mock objects for every symbol works fine when computing the stringized annotations dict. But if you substitute your own arbitrary __co_annotations__ function, our hypothetical full custom bytecode interpreter wouldn’t know which values used by the function should be “Stringizer” objects and which have to be real objects. It just won’t work.

Final random notes:

  • In addition to flow control, the “Stringizer” approach also can’t correctly handle an annotation that uses the walrus operator. However, this has already been declared illegal in annotation expressions. So it’s not a concern.
  • In the original PEP 649, once a __co_annotations__ function was called, it was discarded. This was pleasingly sanitary; either an object had an __annotations__ dict set, or a __co_annotations__ function, or neither—but never both at the same time. Now that we’re going to support this “Stringizer” or “custom bytecode interpreter” approach for back-computing strings, objects will retain their __co_annotations__ functions even after they’ve been called, in case the user later requests their stringized form. This could cause a weird situation: if the user modified the annotations dict, then asked for the stringized version, they might be surprised to see that the two no longer match. Bad news: this isn’t fixable. At best, we could notice that the annotations dict was modified and throw an error, but even that seems like it would be too expensive. I think the best option here is to simply document the behavior and categorize it as “consenting adults” stuff. If you’re the sort of person who modifies annotations dicts… good luck!
  • If you’re thinking “o-ho! the custom bytecode interpreter approach is made way more complicated by the specializing adaptive interpreter work!”, happily no it isn’t. The specializing adaptive bytecode work is all done in secret by the Python interpreter, and you won’t see those funny specialized bytecodes if you examine the bytecode normally (co_code on the code object). So the “custom bytecode interpreter” doesn’t need to worry about them.
  • The library functions computing annotations will also support an alternate “mixed” mode, in which the fake globals dictionary returns real values for names that are bound, and “Stringizer” objects for unbound names. This should work perfectly for use cases like dataclass, where all it cares about is whether or not an annotation is an instance of InitVar or ClassVar. In this case, unbound values, presumably from modules bound by if TYPE_CHECKING, will be Stringizers, but they’ll get wrapped with real InitVar and ClassVar objects. This allows dataclass to do real isinstance tests, rather than manually parsing PEP 563 stringized annotations. It’s great!

Poll 1: should __co_annotations__ be public or private?

  • __co_annotations__ should be an unsupported internal implementation detail, at least for now.
  • __co_annotations__ should be an supported public API.

0 voters


Poll 2: what approach should we use to stringize __co_annotations__ functions?

  • “Hard-Code The Stringizer”
  • “A Simple Custom Bytecode Interpreter”
  • “A Full Custom Bytecode Interpreter”

0 voters

2 Likes

Oh, I meant to add my vote. My guess is “Hard Code The Stringizer” will work fine, and I… really don’t know whether __co_annotations__ should be considered public or private. As a rule stuff like this in Python should be public. There are definitely undocumented internal implementation details in Python… that people discover and use anyway. So it seems like there isn’t that much difference. I guess I’m +0 on making it private and -0 on making it public; after all, we could always change our mind later.

1 Like

Should 649 specify that the __co_annotations__ attribute is a supported public interface? Or should it specify that __co_annotations__ is an internal implementation detail?

Personally, I don’t mind either solution, but I’d like to point out that “internal implementation details” that are unprotected are prone to become features that people rely on.

3 Likes

By private, do you mean invisible (inaccessible) or read-only (like some other things are)?

Neither. I mean private as in “documented only to say that this is an internal implementation detail, and it is unsupported and may change without warning”. Maybe “private” isn’t the best word for that, but, well, that’s what I meant.

2 Likes

Regarding the Stringizer proposal, the solutions you have prototyped seem to be a fair amount of work to maintain going into the future.

Would it be viable to have the compiler grab the annotations as strings and store them somewhere for later use? Similar to PEP 563 (Postponed Evaluation of Annotations). Then stringifying an annotation turns from a complex interpretation process to a fast lookup.

You meantioned two modes for getting the annotations as strings, I’m not sure if this helps the “mixed” mode. You might need a full Stringizer to deal with the mixed mode, in which case storing the annotation strings is of little value.

Silly question: you say that Stringizer may include a byte-code interpreter. Isn’t that eval, which can operate on code objects as well as strings?

2 Likes

The “Stringizer” wouldn’t need much upkeep. The “bytecode interpreter” version would presumably need more, as the bytecodes themselves change. But most of the time Python bytecode is pretty stable. For a baseline, take a look at the log for Lib/dis.py and notice how infrequent it’s changed to reflect bytecode changes.

It’s possible, but it seems wasteful. The percentage of Python programs that will ever examine stringized annotations at runtime is quite small. Meanwhile we’d be storing the annotations in two different formats.

Also, as I stated in the original post, I don’t think anyone will examine stringized annotations in performance-sensitive code. While speed improvements are always a nice-to-have, in this case they aren’t actually all that important. So this proposal improves speed for non-performance-critical code at the cost of a lot of space. I don’t think that’s a worthwhile tradeoff.

Note that this approach could allow for an improvement in fidelity. The implementation of PEP 563 doesn’t preserve the original text from the .py file, it reconstructs the annotation strings by walking the ADT. This is already a slightly imprecise reproduction of the original annotation expression; for example it doesn’t preserve whitespace or extraneous parentheses. I don’t recall complaints about this aspect of PEP 563, so it seems like everyone was okay with it. Admittedly, reconstructing the annotation strings from bytecode is less authentic still; for example, some optimizations may already have been applied. (If the original annotation had a static expression, like 2 **16, I believe that by the time the “Stringizer” or custom bytecode interpreter saw it would have been optimized into 65536.) I’m pretty sure this too is fine. But, for what it’s worth, it’s possible some brainy egghead on the parser team could separate out the original text of the annotations and write that as the pre-calculated stringized form in the .pyc file, which would result in the highest fidelity annotation strings. Again, I don’t think anyone actually cares.

So in summary, I doubt we should go this route.

I’ve heard there’s a proposal to add lazy-loading of data from .pyc files. If we could store the strings in the .pyc file and only load them on demand. That would make this approach more viable–disk space is pretty cheap these days.

Finally, for what it’s worth, I proposed this at some point in the last year or two, calling it “the great compromise” or something. Nobody was interested. And now that the “Stringizer” approach is here, I’ve lost interest in it too. I don’t think we need to bother with storing the strings, reconstructing them works fine.

Correct, the pre-calculated stringized annotations you proposed above would not be useful for “mixed mode”. For that we need to execute the bytecode. (Or pseduo-execute it in a custom bytecode interpreter.)

First, I’ve been using “Stringizer” specifically as the name for the class that implements all the dunder methods. I don’t call the bytecode-interpreter approach “Stringizer”. Admittedly it is stringizing the values. But, just so we have convenient names for specific approaches, let’s not refer to the bytecode interpreter as “Stringizer” too.

eval computes real values. The bytecode interpreter computes fake stringized values. Although they operate on the same object–the bytecode–they do different things when evaluating it. If I have to go with the “bytecode interpreter” approach, eval won’t work for me.

Of course, the “Stringizer” approach is using real eval, more or less. It actually rebinds the code object to a fresh function object then runs that, which was better than using eval for some obscure reason I’ve temporarily forgotten.

1 Like

I’m trying to stay out of this discussion, but here I feel a word of caution is needed:

That may have been the case up to Python 3.9, maybe even for 3.10, but in 3.11 we changed a lot of instructions (due to the specializing interpreter), and 3.12 looks like it might break that record (we keep finding ways to improve upon 3.11). I don’t expect we’ll be done in 3.12 either. Now, you may not need the full bytecode interpreter for your needs, but as a general rule, this guidance is about as out of date as Moore’s Law.

3 Likes

I was messing around with bytecode (translating to a register-based VM) in the space between 3.9 & 3.10. While the changes might have seemed modest, when I tried to make the leap to 3.10, it broke me and I gave up. I can’t even imagine how much drift there was in 3.11. I’ve never gone back to look at it.

I guess stable is in the eye of the beholder. Or something like that.

1 Like

Wouldn’t Cython apply? Cython typically wants its C-compiled functions to look like plain Python functions as much as possible. cc @scoder

1 Like

Since you’re soliciting responses from people who haven’t spent months thinking about this: +1 to just store the strings.

Python already stores function and variable names, and docstrings.
Even if we decide the initial implementation needs to include lazy-loading introspection/debug data from pycs, work/maintenance-wise that seems (very roughly) similar to a bytecode decompiler, and it would help other use cases.

2 Likes

I wouldn’t know, I’ve never used Cython. My hunch is, Cython functions don’t need lazily evaluated annotations, because in order to compile to native code, they need to know all the types in advance. So they won’t have the problem of “this type hasn’t been declared yet / is imported in an if TYPE_CHECKING block”. I hope a Cython person can interject and speak more authoritatively.

I assume that by "just store the strings", you mean "simply also store the strings", not "only store the strings". The latter would be equivalent to PEP 563, which we’re working towards deprecating.

So, that’s an interesting idea. I think it would work fine; my two questions are “where should I get the strings?”, and “are we okay with spending the additional disk space on it?”. I feel like the additional disk space would be okay but I’d like to other folks to chime in.

As far as my first question goes, I’d say “preferably straight out of the parser”. But it occurs to me that this could potentially result in some strange annotation strings:

def my_fn(a: list[
#  line comment here
int]):

If we preserve everything, this annotation string would have newlines and a line comment. Maybe, if we get the strings straight out of the parser, we should transform them a little? Like, reconstitute the annotation from the token stream, but remove all comments, and convert all whitespace strings into a single space.

As far as “it’d be roughly similar to a bytecode decompiler”, I don’t think I agree. The bytecode decompiler would be written in Python; the lazy loader and tap into the parser would all be written in C. I’ve already written the bytecode decompiler and I didn’t think it was a big deal. The lazy loader and the tap into the parser seems a lot more complicated to me. Of course, the “Stringizer” beats both, as it’d be written in Python with little or no dependency on internal implementation details. So it might be portable to PyPy without much trouble.

I’d like to know what the community thinks of the “Also Store The Strings And Write A Lazy Loader” approach. I tried to add it to the poll, but Discuss won’t let me. Should I add a new poll? Should folks who prefer that approach post a message with a “+1 to Also Store The Strings And Write A Lazy Loader”?

2 Likes

Cython doesn’t need typing information to compile to native code. By default it will generate code using the abstract C API, though it may try to infer the types of some variables and/or take reasonable shortcuts.

So, Cython functions could need lazily evaluated annotations just for the same reason Python functions do.

1 Like

Huh, okay.

But consider this: Cython is already completely dependent on internal implementation details of CPython. So maybe it wouldn’t be much of an impediment to them if we declared __co_annotations__ an internal implementation detail. They’d use it anyway :wink:

1 Like

I don’t know enough about the on-disk representation to know how feasible this is, but I would imagine that a significant proportion of type annotations are primitive types, or containers of primitive types. As such, I would assume that there would be a large benefit in interning annotation strings.

1 Like

Quite probably, but that doesn’t sound like a terrific argument either. Especially as other similar tools (pybind11 perhaps) might want to support lazy annotations as well.

1 Like

Could you please be specific? What is the large benefit you have in mind?

I notic that both the tools you’ve suggested so far build native extensions, rather than Python code objects. So in both cases you’re talking about consumers of the C API, not the Python API. Would it make sense for __co_annotations__ to have a supported C API, but designate its Python API as “internal and unsupported”?

1 Like

Oh, I’m not hugely worried about this. My point was that if people are worried about the cost of storing the stringified annotations and the code objects, then we can potentially mitigate that with a lookup table.

Oh. Well, it probably should have been obvious from the name, but I hadn’t noticed that __co_annotations__ was an attribute on the code object (rather than the function itself) :slight_smile:

Then, yes, what you’re suggesting probably makes sense.

1 Like

You had it right the first time: __co_annotations__ is an attribute of the function object. The name __co_annotations__ is originally because it was a (reference to a) code object; I changed it to a (reference to a) function object less than a week before posting the first version of the PEP. I kept the name because frankly no better name suggested itself. There’s a section about “Bikeshedding the name” in the PEP but IIRC nobody has suggested a different name, much less a name I liked better.

Does that change your opinion?

1 Like