Hi @kddnewton, thanks for taking the time to share:
I agree with almost everything you said (regarding memory, performance floors, etc.)
I disagree however with the point that a tracing jit requires lots of changes to support methods for the current state of CPython. I agree that advanced versions of them diverge quite a bit. However, the current tracing JIT implementation in cpython is low-medium stage on the path to a full-blown tracing jit like say LuaJIT. We have not implemented the usual complex things like inter-trace optimizations, recompilation, unboxing, etc. This means the complexity to switch at this point is not high. It will be high if we had a more mature tracing runtime, but we do not. IMO, tracing and method runtimes only start diverging in implementation (though not conceptually) a lot when we implement the more mature optimizations. That’s why we put forth the PEP as it is for CPython right now.
Also I think some of the design difference points mentioned for a method JIT apply to a tracing jit in cpython as well?
- Control-flow merges: You need this for loop peeling for LICM in tracing jits https://dl.acm.org/doi/10.1145/2384577.2384586
- Guard placement: We already do this for the upstream tracing JIT.
- Deopt: We also already do this for the upstream tracing JIT. This will also need to be done once we start doing serious optimizations regardless if it’s tracing or method?
- Multi-entry resume/entry: We need this anyways to support generators. In tracing it’s an n-way branch keyed on instruction pointer for generator yields/resumes in a tracing JIT. The current tracing JIT cannot do that efficiently. I tried (again, I implemented the current trace recorder), and unless we implement self-modifying code or allow growing dispatch tables, the naiive implementation is just too slow. So there’s really not much difference here.
- Branch history: This is done in the CPython interpreter, not the tracing JIT. So it works for both.
- More advanced ICs: This I agree with. However, our IC infrastructure in CPython is unique in that we already have runtime specializers that can profile data at specialization time (Python/specialize.c). Instrumenting them to save extra info isn’t hard. So it’s not the same as in other runtimes.
In short, additional things you mentioned that a method JIT requires is already all conceptually required by a mature tracing one in CPython, though the actual implementation can differ quite a bit. I appreciate your expertise in CinderX and other JITs. However, the method frontend proposal sits in a unique spot not because of tracing vs method, but that the nuances in CPython make it easy to switch at this point of time (I can’t say it’s will be the same in the future!)
IMO, PEPs are only required IMO if we change a fundamental or specified part of CPython. The current tracing JIT is neither a core part of CPython nor is it specified in the first place. I don’t understand why the burden is on a (genuinely, for our current state at least) minor change to produce two PEPs, when the current design has no community consensus nor formal approval to begin with. If anything, applying the same formal process in CPython means that the current upstream JIT must have two PEPs (one justifying its implementation, one criteria), same as any other implementation that is experimental. However, I don’t think the original SC announcement requested for that, and I don’t want to put the burden of producing two PEPs on the current upstream JIT as well.