PEP 744: JIT Compilation

[brandtbucher]

Mostly echoing @mdroettboom’s excellent answers:

@malemburg

Thank you for the write-up, Brandt. This is an exciting development

A few questions:

• LLVM: I am a bit confused about the statements related to LLVM. In the “Rational” section you write using LLVM would incur significant overhead and a heavy dependency. Then in the “Support” section you write that some platforms may not receive JIT support due to LLVM not having good support for them. Could you clarify whether you are using LLVM of not ? Other comments in the PEP suggest that you are using LLVM. If that’s the case, how do you manage to keep the overhead low ? (In that case, it may also make sense to change/update the “Rational” section to avoid such questions)

LLVM is used at build time to compile individual micro-op instructions into mostly-opaque blobs of machine code. These blobs are dumped into a header file, which is used to build CPython itself.

At runtime, we compile a sequence of micro-ops by “simply” copying the machine code for each one, almost verbatim. So instead of firing up LLVM, it’s “just” a memcpy and a couple of loops to fix up parts of the code. Implementation here, in C.

The PEP could probably be updated to make the run-time/build-time distinction clearer.

@malemburg

• Patents: Are you aware of any patents on the copy-and-patch approach or other advanced techniques you are using in the JIT, which may encumber the use of a JIT enabled CPython ?

I’m not aware of any patents on the approaches used here.

@malemburg

• RAM usage: Does the JIT limit itself to using a certain percentage of available RAM (as a measure against DOS style attacks or bugs in the code) ? How quickly are less used JIT compiled parts garbage collected ? Can the JIT GC be controlled programmatically (e.g. by sys module calls) ?

No, when their refcount hits zero, and no.

…at least, that’s the state right now. The current memory allocation/freeing scheme can be summed up as “I got it working a while back and haven’t touched it since”. All of the points you make above are things I want to do, but simply haven’t had the time to work on yet.

One main thing to keep in mind is that I’d like to avoid introducing any “official” JIT APIs (or encouraging users to tweak them) while it’s still experimental. I could definitely see tunable controls for enabling/disabling the JIT, controlling max memory, and more control over GC of cold traces in the future, though, exposed through the sys module, environment variables, etc.

The good news is that all of the things you ask for should be easy to do. We already maintain a doubly-linked list of all JIT code (for unrelated correctness reasons) and have ways to turn the JIT on and off at runtime (for testing).

@malemburg

• Dependencies: Assuming you are using LLVM, is it still possible to build CPython without JIT support on platforms where LLVM is not available/supported ?

Yep, and it always will be (even on platforms where LLVM is available).