PEP 554: Multiple Interpreters in the Stdlib

PEPs
26

New PEP Version

I’ve made one last update to PEP 554. Given the low level of discussion the last few times, I’m hopeful this is the last round before I submit the proposal to the Steering Council.

You can find the updated PEP at PEP 554 – Multiple Interpreters in the Stdlib | peps.python.org and I’ve included the text below.

The most notable changes:

  • added channels back in
  • added Interpreter.set_main_attrs() (and Interpreter.get_main_attr())
  • renamed Interpreter.run() to Interpreter.exec()

I decided against adding any new C-API to the proposal though. I’m going to do that in a separate PEP.

Note that the changes are in response to real world experience I had using the proposed API, writing some concurrency benchmarks. I’m happy with where things are at.

Also note that I’m still trying to limit the amount of functionality in the proposal. My goal continues to be a minimal foundation to which we can add functionality as needed.

Open Questions

  • will is be too confusing that interp.exec() runs in the current thread?

With the change of name from interp.run(), I’m hoping it’s less of an issue. I expect that we should be able to provide enough clarity through the docs to avoid confusion.

  • should we add pickling fallbacks right now for interp.exec(), and/or
    Interpreter.set_main_attrs() and Interpreter.get_main_attr()?

This is something that might reduce friction for folks using the new module. It certainly isn’t necessary and runs counter to my “minimal” goal. However, practicality beats purity. If it’s clear that we’ll end up adding something like this, I’m not opposed to doing so immediately. It doesn’t have to be in the PEP though.

  • similarly, should we some a limited set of functions in interp.exec()?

From a usability standpoint, passing a string to interp.exec() is a bit clunky, even if it does stay true to the basic functionality. It isn’t actually that hard to support functions that aren’t closures and don’t take arguments. I’ve tried it out and even with those restrictions its a big improvement. This doesn’t strictly need to be part of the PEP, though.

  • rename Interpreter.close() to Interpreter.destroy()?

“close” matches the same function on a variety of other similar-ish classes, but isn’t nearly as clear as “destroy”. At the moment I’m leaning toward “close”, but I’m open to persuasion.

  • drop Interpreter.get_main_attr(), since we have channels?

interp.get_main_attr() could be an attractive nuisance, a clunky solution for getting data back from an interpreter. In the absence of alternatives, it’s probably fine as a low-level tool. However, the PEP currently includes channels, which are a much more elegant approach. I’m leaning toward dropping Interpreter.get_main_attr() if channels stick around.

  • should channels be its own PEP?

I just don’t see the value in that. The capability is relatively small and straight-forward, but useful enough to pay for itself. It doesn’t seem that controversial as (effectively) an appendage to PEP 554. The extra functionality would hardly be enough to warrant it’s own proposal. I’d rather it not make a separate PEP for channels.

4 Likes