PEP 741: Python Configuration C API (second version)

I’m collecting feedback from users: see previous comments.

There are use cases to embed Python with the stable ABI. The main use case is to select which Python version is select which Python version is used when running an application, without having to rebuild the application.

Reading the current configuration at runtime was asked multiple times by Marc-André Lemburg.

Marc-André Lemburg wrote (emphasis is mine):

I also could not find a public PyConfig API for reading the current config. Why is that ?
There’s lots of useful information in the PyConfig struct, which extensions could use as well and won’t be available elsewhere anymore once the global vars are gone.

Marc-André Lemburg also wrote (emphasis is mine):

Marc-André also asked to have a way to modify them at runtime, but this feature is not part of the PEP scope. When it was discussed later, it was said that it’s better to modify Python attributes such as sys.path, rather than the “configuration” (such as config.module_search_paths).

Marc-André wants to update config.verbose but again, it’s not part of the PEP scope. I didn’t see other people asking for such feature so far. By the way, sys.flags.verbose is read-only.

PyInitConfig API can be used for that, no? It’s an opaque structure, we have a lot of freedom to decide what it contains and how it behaves.

If you want PyInitConfig_SetInt() to behave differently depending if you initialize the Python main interpreter, or a sub-interpreter, we can have different “Create” functions, such as:

  • PyInitConfig_CreatePythonInterpreter().
  • PyInitConfig_CreateIsolatedInterpreter().

For example, we can say that setting a preconfiguration option, such as allocator, would fail with an error.

PyConfig_Get() does not only read PyInterpeterState.config, but also sys attributes: there is no 1to1 mapping. It’s specified in the latest version of PEP 741 (PyConfig_Get() spec).

Many PyInterpeterState.config members are used at runtime and there is no other copy. Even if the implementation changes to remove that member, PyConfig_Get() would still remain relevant, no?

What would be your use case for that? With no concrete use case, there risk is that it would only waste memory. Which API would give access to it?

Currently, we don’t keep copies of PyConfig.module_search_paths or sys.path when they are modified.

PyInitConfig is opaque and so is a better API, since some members can be removed/ignored for sub-interpreters.

Note: PEP 741 “PyInitConfig” API does not deprecate PEP 587 PyConfig API.