thx for the detailed example – it’s a great proof for showing the Adaptive Intrepter’s fast performance when handling basic types.
But in practice, most Python code uses complex types:* Optional[T], list[T], dict[K, V].
The adaptive interpreter can’t specialize on these because it only sees the runtime type (e.g., list), not the type arguments (list[str] vs list[int]).
That’s the gap I’m trying to fill with type hints.
My point isn’t that people should write this code - it’s that people have written code like this. A change that suddenly breaks working code is a serious thing. How do you justify breaking backwards compatibility like that?
Not to say that it can’t be justified, but it’s an important hill to climb for any idea like this.
I understand the backward compatibility argument, but I’d push back: if someone writes name: str and passes an int, the code is already wrong — it’s just not being caught. The type checker would flag it, but not everyone runs a type checker. Making the runtime optionally enforce the annotation (via a flag or decorator) wouldn’t break default behavior, but it would give users who want correctness a way to get it.
Are you still talking about your earlier example? Wouldn’t the existing “fallback” mentioned by Jesper keep it working?
I’m wandering about that is Dispatch Inline Caches (IC) good enough.
There’s issues with the type system that conflict with wanting to do this even if it weren’t more complex to pass in hints for the smallest of possible gains, like that the type system treats float in annotations as if you had written float | int, or that the type system currently makes incorrect assumptions about __hash__
despite how often this topic seems to come up, the combination of the type system having holes and the specializing interpreter doing it’s work relatively quickly from accurate information at runtime keeps this from being something I’d even consider pursuing in more detail.
3 Likes
In CPython, this approach would not work well because Python is a dynamically typed language. If you assume a value always has the same type, you would need to perform runtime checks continuously to verify that assumption. For hot code paths, this is acceptable. CPython’s adaptive specializing interpreter can observe execution, identify stable type patterns, and replace generic bytecode with specialized bytecode. The cost of these checks is small compared to the performance gains because the code runs frequently. However, for cold code paths (functions or sections of code that are rarely executed) the overhead becomes much harder to justify. You would still pay the cost of the runtime checks, but there may not be enough executions to recover that cost through specialization. In some cases, the code could even become slower than the normal generic implementation.
Julia addresses this differently through multiple dispatch and JIT compilation. When a function is called, Julia specializes the function for the concrete argument types and compiles optimized machine code. After the first compilation, later calls with the same argument types can directly execute the previously generated optimized machine code.
Bringing a similar model to CPython would introduce significant complexity and would fundamentally change its design philosophy.
There have been different tries with different languages to achieve this but I don’t think it will ever work in Python for the arguments mentioned above.
However there is currently work done on Spy(Lang) which seems to conceptually hit the sweet spot. But it’s in its early stages so time will tell if it’ll mature enough or if it’ll stay in the niche like e.g. nim.