Use function annotations to improve bytecode specialization?

Ideas
1

CPython 3.11 introduced adaptive bytecode specialization. It can specialize operations after observing runtime behavior.

For example:

def add(x, y):
    return x + y

Initially uses BINARY_OP, then after enough executions it may become an integer addition specialization.

But what about:

def add(x: int, y: int) -> int:
    return x + y

Here the compiler already has information that both arguments are expected to be integers.

Currently, this information seems to be ignored by CPython’s optimizer. The interpreter still starts from the generic BINARY_OP and needs to rediscover the types at runtime.

Would it make sense to use annotations as an optimization input?

For example, emit a more specialized operation directly:

LOAD_FAST x
LOAD_FAST y
BINARY_ADD_INT
RETURN_VALUE

instead of:

LOAD_FAST x
LOAD_FAST y
BINARY_OP
RETURN_VALUE

I understand that annotations are not runtime-enforced in Python today. This does not necessarily require changing that. Similar to static languages, this could be an optional optimization based on the programmer-provided contract.

The main question is whether CPython should make use of existing typing information instead of throwing it away during compilation and performing type discovery again at runtime.

2

If the compiler can infer the exact types at compile time, it can also infer whether __add__ *will fall back to __radd__. In that case, the runtime if check for radd is unnecessary. The generated code can call the corresponding C function directly.

3

That would break dynamic addition / deletion of attributes though…

class mytype:
     def __add__(self, other: object) -> mytype: ...

# Implementation is irrelevant 

m1 = mytype()

m1 + 1 # Uses __add__

# Might be called after some n calls of the previous behavior, in an if-case, ...
del mytype.__add__

m1 + 1 # Should use int.__radd__, raises an error
2 Likes
4

I think the language allows for the identifier int to have other values than the type builtins.int.
Reading

def add(x: int, y: int) -> int:
    return x + y

doesn’t yet tell that the arguments are both integers.

5

Type annotations are not enforced at runtime, by design. The following is perfectly valid code:

def add(x: int, y: int) -> int:
    return x + y

print(add("foo", "bar")) # "foobar"

Optimizing the bytecode in advance would break this snippet. That’s a massive breaking change that has been discussed several times but I seriously doubt will ever happen.

5 Likes
6

True, and if you really want it, there’s Cython, although it uses its own types.

(disclaimer: I don’t have hands-on experience with Cython, but my understanding is it uses the annotations for faster code, just like OP proposed)

1 Like
7

I really, really wish we could do this, but unfortunately & has already said, this would be a massive breaking change. Sure, my own very personal opinion is that dynamic attribute setting is a massive code smell anyways, etc… but python is designed at the core to support this. The only option would be to have a config option to activate these types of optimizations, but I’m pretty sure this has already been discussed and would be some huge work anyway. If you do need to speed up python with guaranteed types, I can’t recommend enough trying rust compiled modules with pyo3. Rust is very approachable when coming from python, despite it’s reputation, and you will see a lot of perf improvements.

You can also try mypc, it’s closer than what you are thinking (using type hints infos to compile code), altough using mypy has a type checker is meh in my experience (VS basedpyright for example)

Finally, cython, has this has already been recommended. However developper experience is much more clunky, you lose memory safety (guaranteed by both python and rust) once you go low level with ctypes, and good luck making it work seamlessly with type checker/linter/formatter/LSP.

2 Likes
8

That would be the job of something like mypyc, or Nuitka (to a limited extent).

3 Likes
9

The existing optimization from BINARY_OP to BINARY_ADD_INT is heuristic, based on some kind of runtime statistics, anyway, no? I’m assuming that it in any case we can never be 100% sure that BINARY_ADD_INT will always be called with ints, so it has to have some kind of error-detection and fallback already? Maybe it’d be ok to use the type-hints and start out with BINARY_ADD_INT, and 99% of the time it will be fine, and in the remaining 1%, where the programmer has decided to do something “funny”, we rely on the (presumed) existing error detection and fallback?

1 Like
10

The heuristic you refer to is already sophisticated, and in my view, more reliable than type hints, since those can always be wrong (for example, I once saw an official typeshed stub having an annotation of a dictionary subscripted with dictionary keys) or abused by downstream. Ironic.

1 Like