PEP 836: JIT Go Brrr: The Path to a Supported JIT Compiler for CPython

[brandtbucher]

Thanks for your comments, Thomas.

@thomas

However, this really feels like (at least) two PEPs, and I think it should be split up. I think I understand you want to show that the criteria for supported status are attainable even if we aren’t particularly close right now, but honestly, I don’t think that should matter for the simple goal of setting acceptable criteria. I also don’t think it’s a good idea to mix the two fairly disparate discussions, about the criteria and about how to get there. Fundamentally, the discussion about the minimum expectations for a supported JIT is distinct from how to get there, especially as we have different proposals for how to get there.

@thomas

The timeline given in the PEP is, I think, a little out of place. It combines the performance goals with the implementation paths for the new JIT ideas, and I can’t help but point out that past predictions for performance progress have rarely panned out precisely as predicted. I don’t think we should expect a firm timeline for JIT development. It’s just not how development works in an open-source project, especially not one that cares about backward compatibility .

The timeline and method JIT proposal are providing what the SC asked for, and we put a lot of thought and time into forming a cohesive PEP around them because they were explicitly requested.

From the SC’s post (extracting the two relevant bullets):

@pablogsal

We would like the PEP to address at least the following points:

• Clear, measurable success metrics and timelines. What the project is aiming for and by when, for example performance targets, platform coverage, and memory overhead.

• Whether the current JIT architecture is considered stable, or likely to change further.

Also in the same post:

@pablogsal

While the intent is not to call for competing proposals, we believe that now is a good time to discuss and propose alternative proposals as well.

We are providing a concrete proposal for an alternative implementation that a lot of people have suggested over the years, with clear, measurable success metrics and timelines for that alternative implementation, as well as setting expectations about how stable and unstable the different parts of the current JIT architecture are in light of that transition.

So I think it would be inappropriate for the PEP to not include these. Further, shifting to the alternative design directly affects the timeline/maintainability/etc., so a separate PEP seems like it makes the process more complicated and opaque, not less. Here, we have a way to present a complete story.

(And if the SC feels differently, it can totally say “We accept the PEP, but we want to stick with tracing because of X, Y, and Z.”)

@thomas

Using a stack based JIT makes sense if you look at it from the view point of a CPython developer familiar with the stack based interpreter who needs to learn about the JIT. I do not think that this is important. We should not design our architecture to be most understandable to current Core team members. Instead, we should focus on making whatever design we end up with understandable, and understood by the Core team. We need to make sure enough people work on it and spend time with it.

I think this is a misunderstanding.

Firstly, I want to re-emphasize that the uop IR is what’s currently in main, we are not introducing a new IR with this PEP. The main reason why we like the existing uop IR so much and don’t want to change it is because it’s literally how the base interpreter itself is written. It’s not just “understandable”, it’s what’s actually used today to define every single bytecode instruction. Furthermore, lowering is trivial (and mostly generated), keeping the Python frame in a sane state is easy (because the IR operates on the exact same stack as the bytecode)… there are many nice things about sharing definitions between the two parts of the runtime, and a concatenative, stack-based IR is a very straightforward way to do that in our experience.

So I would argue that yes, this is already understood well by anyone who works in the interpreter loop. More importantly, is it maintainable by a small team due to deriving from the interpreter. If you’re interested, take a look at What’s new in Python 3.15 — Python 3.16.0a0 documentation; I count 14 contributors to the JIT optimizer, of which only four are core devs. I think this is pretty strong evidence that the current JIT IR can be understood and is accessible to other members outside the core team.

@thomas

We actually discussed this notion briefly among people working on CinderX, and @mpage and @kddnewton pointed out that attempting to design a JIT to be more easily understood by Core team members makes it less easily understood by others, and makes it harder to apply literature and lessons learned from other JITs to CPython’s.

That’s a huge part of why the method JIT is part of the proposal. There are a lot more people familiar with whole-function compiler architectures than tracing ones. It turns out the uops/stack also have nice SSA properties if you’re careful, and so tons of relevant theory is directly applicable to the bytecode-fragment IR we already have.