Status Update about Interpreter Isolation

tl;dr We’re really close to having proper isolation between interpreters.

Hi All,

Over the last few years, a number of contributors[1] (including myself) have made a variety of changes in the code base to improve runtime/interpreter startup[2] and interpreter isolation. Unsurprisingly, my personal motivation has been primarily facilitating a per-interpreter GIL, but there are at least several good reasons for this class of improvement.

By “isolation” I’m talking about sharing any mutable data between interpreters. This especially applies to objects[3]. A small amount of shared mutable data is necessarily process-global, and interpreters will continue sharing that safely by using one or more locks (currently the GIL) to guard mutation. However, any other state (AKA mutable data) must be strictly tied to each interpreter. All this is covered in detail in PEP 684.

The biggest part, by far, of isolating interpreters has been eliminating C global variables (at least ones that aren’t const or effectively const). Since this part of the effort started in earnest ~5 years ago, we’ve gone from several thousand unsupported globals to, in the last few months, zero[4]. I genuinely appreciate the people that have made this happen!

One of the most complex parts of interpreter isolation has been (surprise, surprise) sorting out extension modules. It has required finding an effective and efficient way to isolate them to each interpreter. This work was started quite a few years ago by Martin von Löwis, but the bulk of the solutions have come from @encukou, who has put in a substantial amount of time and effort.

Relatedly, there has also been a lot of great work done in the last 3-4 years by quite a few people to port stdlib extension modules to multi-phase init (which isolates them). We still have a dozen or so modules left to be ported[5]. Help with wrapping up porting the remaining modules would be very appreciated! (I’d be glad to collaborate/instruct. Just ping me.)

Overall, we’re currently really close to reaching proper isolation between interpreters, to the point that a per-interpreter GIL is realistic for 3.12. Furthermore, in the process of all this we’ve ended up finding (and fixing) a bunch of bugs hidden in the runtime. We’ve exposed some useful capability through the C-API, and achieved (or facilitated) a number of performance improvements. We’ve advanced the effort to determine the future of the C-API, including by helping expose the value of explicit runtime contexts. Finally, it’s much easier to identify and reason about CPython’s runtime state now. IMHO, the runtime is in a better place as a result of this work.

Again, thanks to everyone that’s helped!

-eric

[1] It’s hard to identify everyone involved, but one key person was @vstinner. Victor had a hand in a lot of this work and it would have taken much, much longer without him.
[2] The bulk of the improvements to startup have been due, directly or indirectly, to the work @ncoghlan did around 2017(?), which were followed up extensively by @vstinner. Without those improvements, interpreter isolation would have been much harder to reason about and track, and thus much more difficult.
[3] Even otherwise effectively immutable objects (e.g. None) can’t be shared due to refcounts, at least not without object “immortality”. (See PEP 683.)
[4] This doesn’t count the objects that we will be making immortal or the handful of extension modules we still need to port to multi-phase init.
[5] I’m not counting the various test modules, which don’t impact real usage, nor builtin modules, which currently don’t have any unsupported globals (once we have immortal objects). In fact, I’d like to see all the remaining builtin modules ported to multi-phase init, since the machinery for single-phase init modules is a bit complex (making it harder to debug isolation issues in builtin modules), but that isn’t essential at the moment.

31 Likes

Very cool to see how close this is to becoming a reality! It’s admirable and impressive the way that you have been able to keep chipping away at the many component parts of this problem, finding ways to break it up into pieces that are acceptable improvements in their own right rather than needing to be a single “all at once” major integration effort.

One very early influence on this work folks may (or may not!) be surprised by is @brettcannon. He drove the initial development of the modern import system, setting the foundation for both the improvements to interpreter start up (as you can draw a direct line from my frustrations when working on the start up fixes needed to get the main module to play nice with the modern import system through my original design work on ways to make the interpreter start up phase easier to manage and on to @vstinner’s iterative efforts to make concrete improvements inspired by those ideas), and the multi-phase initialisation that enables improved module support (thanks to @encukou and everyone that has contributed module migrations over time!)

8 Likes

This may be off topic here, but: What exactly is Interpreter Isolation? I searched around a bit and I still couldn’t guess what it implies.

1 Like

Sorry that wasn’t more clear:

Here we’re talking about the capability of CPython to run multiple interpreter runtimes simultaneously in a single process. See the C-API docs for more info.

The objective is to isolate the state of each interpreter, so state doesn’t leak between them. There are a number of benefits to doing so.

1 Like

Looks like there are some PEPs folks can read about this too:

Looks like a guy named Eric authored them. These do a great job summarizing the work to do and the inspiration for it. Kudos!

7 Likes

Thanks, these are really interesting resources!

Thank you, Eric, Victor, Nick, Martin, Erlend, Dong-hee Na, Hai Shi, Mohamed Koubaa, and everyone else involved! This was faster than I thought it would take :‍)


Python 1.5 (1998)'s docs for Py_NewInterpreter include this caveat:

Because of the way extensions are shared between (sub-)interpreters, some extensions may not work properly […] It is possible to insert objects created in one sub-interpreter into a namespace of another sub-interpreter […] (XXX This is a hard-to-fix bug that will be addressed in a future release.)

The XXX moved into Python’s various bug tracking systems, but the rest of the note is still there today.
Calling it “hard-to-fix bug” was an understatement, but, after 25+ years, we have the fix on the CPython side, and are ready for the ecosystem to do their part (here’s a guide!) as we sand off the remaining rough edges.
Given how the ecosystem has grown, it won’t be quick – perhaps it’ll be another decade before we can remove the caveat. What’s important is that Python will keep working throughout the transition.

8 Likes