An announcement from the Steering Council regarding the JIT project

Given that the SC considers the vague new-feature permission of informational PEP 744 to have ended with 3.15, it is reasonable to not want more, unapproved, features added to the future 3.16. On the other hand, I think Mark’s concern about ‘momentum’ and volunteer retainment is reasonable. An implementing at least some of a PEPs promises after an initial draft is often part of getting it approved.

Would they be easier to handle on a temporary ‘jit’ branch of main on python/cpython rather than on a personal cpython fork?

I am aware that we now restrict cpython branches to those release-related, but this might be special-enough for an (temporary) exception.

5 Likes

I have contributed some code to JIT, mainly related to uop coverage. Currently, the plan for JIT is outlined in an overall direction at JIT Planning for 3.15 and 3.16 · Issue #139038 · python/cpython · GitHub . Since JIT is still in a rapid development stage, I think it is feasible to formulate specific short term plans and long term goals. However, for 3.15, the “must have” Free-threading support has not been completed. Currently, JIT is intentionally designed to be an easy-to-maintain mode, using many generators, but this also leads to the situation where only a few lines of changes ultimately result in thousands of lines of generated modifications, such as gh-148211: decompose `_SHUFFLE_3_LOAD_CONST_INLINE_BORROW` in JIT (GH… · python/cpython@2b6a137 · GitHub . Personally, I do not want this advantage to become a burden of fork merge conflicts. Perhaps during this period, there are clear and minor optimizations (that do not bring obvious visible performance improvements), such as Better uop coverage in the JIT optimizer · Issue #131798 · python/cpython · GitHub , which are acceptable? This is also the part that is easiest to implement and attract new volunteers, and it can maintain the level of activity.

1 Like

To make the current situation clear from the SC’s perspective, especially for new volunteers: continuing any kind of activity in a forked repository is, of course, totally fine.

I have also really enjoyed collaborating with the JIT team over the past year. However, I think we are now at the point where a PEP is needed, and this is a time when we all need to be a little patient.

5 Likes

This thread of discussion has made me realize that PEP 744 never left Draft status, so I wonder to what extent this permission existed in the first place. Describing it as an informal consensus among the Core team that got (partially) written down in that PEP, perhaps?

The low-hanging fruit for the PEP that the SC is calling for seems like the list of criteria laid out in PEP 744 for becoming non-experimental:

  1. It provides a meaningful performance improvement for at least one popular platform (realistically, on the order of 5%).
  2. It can be built, distributed, and deployed with minimal disruption.
  3. The Steering Council, upon request, has determined that it would provide more value to the community if enabled than if disabled (considering tradeoffs such as maintenance burden, memory usage, or the feasibility of alternate designs).

In light of this thread I think (3) should be struck out because the SC is (I think rightly) delegating this back to the community, but (1) and (2) are reasonable criteria that I think we should have some clear answers to after 2 years of development.

I don’t know about (2) since the changes have been mostly transparent to me, but my understanding of (1) is that it’s a mixed bag: sometimes performance gains are well above this, and sometimes performance is made worse. The most thorough analysis I’m personally aware of are Thomas’s benchmarks which is current as of Nov 2025. The Effect of JIT plots for Python 3.13 and Python 3.14 paint a fairly grim picture for the test systems in that benchmarking, on average. But you subject matter experts may know of other benchmarks that should be part of the discussion (e.g. have things meaningfully changed with what’s on-deck for Python 3.15?).

You may also have opinions about refining the PEP’s broad “meaning performance improvement” into something more measurable; I personally think the geometric mean given there is good-enough.

1 Like

Thomas’s benchmark is severely outdated by now. Check out https://www.doesjitgobrrr.com/. The JIT does achieve >5% geomean (in fact, > 10% in some cases) on systems.

12 Likes

The main case where I think a grace period may be warranted is any currently in flight PRs (especially those from external contributors). (I am assuming there are some open PRs, since that concern was raised above, but I haven’t specifically checked
how many there are myself)

Saying “no new PRs on this topic until a PEP is approved” is one thing, shifting the goal posts for already in progress changes is something else (it may even sometimes be an appropriate step to take, but having the main branch JIT include a few merged PRs that aren’t present in Python 3.15 isn’t going to meaningfully affect the PEP discussions, but abruptly halting pending PR reviews has an immediate negative effect on the contributor experience)

6 Likes

I believe most of the currently open PRs are about increasing uop coverage for the JIT: Better uop coverage in the JIT optimizer · Issue #131798 · python/cpython · GitHub (6 PRs are opened)
The remaining PRs with the JIT label are currently six in total. (In fact, two of them are mine, and one of those is not going to be merged.) There may also be some PRs classified as JIT-related that I am not aware of.
(There are also some PRs that are already stale, but I won’t comment on the status of individual PRs here.)

Even if we adjust the policy to allow PRs that were already opened before the statement was published, there are still some details we would need to work through. For example, should all currently open PRs be allowed, or only some of them? There may also be PRs where the boundary is unclear, especially if they could be classified as new features, so some of these details may need further discussion.

So, I still think our main focus should be on writing the PEP. Involving those contributors in the PEP process would also be a good experience for them as part of CPython’s development process.

3 Likes

A post was split to a new topic: How can I contribute to the JIT?

4 posts were split to a new topic: Feedback on doesjitgobrrr.com

Thanks everyone for the feedback so far.

@markshannon and I have discussed the next steps. We want to move quickly and aim to have a first complete PEP draft this week.

Before publishing it publicly, we plan to share the draft with a small set of targeted reviewers, only to get early feedback on whether the PEP is clear, complete, and properly addresses the Steering Council’s request.

After that, we plan to publish the draft on discuss.python.org for wider community review.

Bear with us while we work on the PEP. Watch this space.

29 Likes

2c from someone far removed from core or JIT development: I feel that the project needs to be larger in terms of number of people involved, possibly with more time to spare and/or corporate backing, and opportunities for wider community to contribute.

I’m not sure if the human aspect of the project belongs to the PEP as such, though I’d like that to be stated somewhere.

2 Likes

In recent times, or at least compared to last year, the JIT has definitely picked up quite a few new contributors. But yeah, community support is definitely very important and I think that has been the way since the start.

For those who wish to continue contributing to the JIT:

Bugfixes and security fixes may of course continue as normal;

There are plenty of bugs to be fixed

4 Likes

PEP Talk

Hi,

The following is co-written with Aaron Patterson but written mostly in the first person. It represents only our personal viewpoints, not those of our employer or other CRuby developers.

My name is Max. I work on ZJIT at Shopify. I used to work on Cinder at Facebook and before that on Skybison at Facebook. I thought it would be helpful to give perspective about how the YJIT/ZJIT team addressed the following concerns from the CRuby perspective (though keep in mind that CRuby’s development process isn’t as formal as CPython’s, so we don’t have exactly the equivalent of PEPs):

Maintenance is a real concern for a subsystem of this size and complexity, and the PEP should set out a clear plan for how the JIT will be sustained and maintained over the long term and how it will impact maintainers and contributors that don’t directly contribute to the JIT.

Maintenance can have multiple facets. For example the host language choice (in our case Rust baked inside of a C codebase), the compiler design (YJIT is LBBV, ZJIT is a more traditional SSA CFG method based compiler).

Here is some brief history and some of the things we’ve found so far:

  • Initial versions of UJIT/YJIT were written in C
  • Writing a compiler in C is hard™
  • Rust makes it easier to avoid UB etc in the compiler itself
  • Rust code is easier to organize than C
  • Rust has nice data structures that you have to DIY in C
  • Rust is a maintenance burden on the core CRuby team because they are mostly familiar with C and Ruby
  • Rust is a maintenance burden because of the requirement that CRuby be buildable and packageable on stock Debian; we cannot use Cargo to install dependencies or build the JITs.
    • This has knock-on effects such as some of the implementation details being slightly less idiomatic than in the wild
  • YJIT used a newer-research-based architecture (LBBV) whereas ZJIT intentionally moved toward a more standard/well-known SSA-CFG-based architecture to encourage more community contribution and ownership
  • CRuby configure/build complexity. CRuby has the requirement that it only depends on a C compiler, so we’ve had to introduce build complexity in order to make YJIT/ZJIT optional

We’ve found the awkward build requirements to be not quite a “one time cost”, but very close to it—more O(1) than O(n). We’re not frequently mucking with build configuration things while doing JIT development. Additionally, since the JIT is something that can optionally be enabled at compile time, it forced us to design the JIT integration such that it could be removed from the codebase at any time (since it can be removed via build flags).

Despite some of the negative points listed above, we’ve found the added complexity to be worthwhile. We’ve found the Rust code to be easier to maintain, and the more “traditional” design of ZJIT to attract more external contributors.

Last, there is corporate sponsorship. I have seen that corporate sponsorship can accelerate and focus a JIT project (maybe makes it more cathedral than bazaar) and also can run the risk of abandonment or slowdown if the corporate sponsor loses interest. It cuts both ways.

How it will keep compatibility with existing CPython features and tooling. How the JIT interacts with, and what it guarantees for, the capabilities CPython already supports. This includes things such as free-threading, profilers, and debuggers, but the PEP should treat this broadly and with nuance rather than as a fixed checklist.

We haven’t explicitly documented this for YJIT/ZJIT, but the informal requirement is that there is no difference between observed interpreter behavior and behavior when the JIT compiler is enabled. In some cases the interpreter plays fast and loose with things like frame pushes (e.g. what is visible in a backtrace, or a profiler), and sometimes the JIT compiler does too, but usually in the same ways.

Other compatibility things we have noticed:

  • Non-public C API stuff – the ZJIT team has decided not to care to keep exact (accidental?) interpreter compatibility if C extensions do naughty things
  • We may delete allocations that are potentially countable by Ruby-visible allocator APIs
  • We currently keep the same exact backtraces as the interpreter but may in the future choose to occasionally elide a frame (this will have to be discussed and formalized)
  • We support sampling profilers and tracing and all manner of strange reflection for local variables
  • Bytecode instructions are the API boundary between VM and JITs and and we have had some friction there: some Ruby tests assert that methods compile, or don’t side-exit into the interpreter, etc. So sometimes the JIT team plays catch-up re-enabling tests and following up with an implementation of the opcode, and there is the possibility that this complication has a (small?) chilling effect on the development of new or existing opcodes in the interpreter.

Additionally, the way we have structured YJIT and ZJIT is that they can choose to opt-in or opt-out for potentially frustrating combinations of features such as JIT+Ractor or JIT+Box; when the VM enables/makes use of a feature that the JIT does not yet support, it bails out of existing code and does not attempt to compile if it has interactions with new code. This lets us have a more gradual development process rather than tackle everything head-on. I think there is at least a little bit of this kind of implementation strategy in the existing CPython JIT.

Clear, measurable success metrics and timelines. What the project is aiming for and by when, for example performance targets, platform coverage, and memory overhead.

Prior to YJIT, CRuby had MJIT which emitted C and shelled out to a C compiler to get platform-native code. MJIT committers introduced this style of JIT because they initially had a requirement that the JIT should support most/all target architectures; relying on an existing, mature C compiler helped here. They found that trying to write a very platform-generic compiler by relying on existing tools is very difficult and eventually wound it down.

I can’t say what is best for CPython, but Shopify initially developed YJIT in a fork of CRuby and when the team was able to demonstrate a substantial speed improvement it was a much easier case for upstreaming to CRuby (including getting Matz, the BDFL, to relax the all-architecture requirements). ZJIT followed on YJIT’s coattails because there was trust in the team.

Relationship to other JIT compilers. Whether the design is intended to provide general infrastructure that other efforts could build on, and whether it is expected to be compatible or incompatible with third-party JIT implementations such as CinderX, Numba, and PyTorch or any other 3rd party JIT.

We don’t have a public API for building a 3rd party JIT, but CRuby does have a small set of well-understood touchpoints inside the VM that allow JITs to hook in: method call counters, method invalidation callbacks, redefinition of “constants”, tracing APIs (TracePoint), etc. This allows both YJIT and ZJIT to both intercept frequently called methods and bail out when other code causes JIT-unfavorable conditions.

Additionally, CRuby has support for building a VM with both YJIT and ZJIT compiled in, but only allows one to ever be enabled in a given process. Perhaps this is a temporary limitation; the JIT developers have not discussed it further.

CPython has at least some support for compiler infrastructure: CinderX works as a native extension. Other runtimes offer support for fuller interfaces, such as OpenJDK’s JVMCI (C1+C2+Graal[1]).


Thanks for reading this long post. Hopefully it is somewhat illuminating. I am happy to answer questions and field comments.

Best,

Max & Aaron

cc @corona10, @kj0, @mpage


  1. Though maybe they are removing JVMCI now? ↩︎

37 Likes

Apologies for hijacking the thread but I want to give as much visibility as possible. We are collecting community perspectives to help shape the upcoming PEP for the CPython JIT.

Have your say here: Community perspectives on the JIT: experiences, expectations, and concerns

6 Likes

To create a cross link both ways, PEP 836 has been posted on the PEPs category.

4 Likes