PEP 741: Python Configuration C API

PEPs
38

I think making the configuration structure opaque and using an API to set/get configuration by name is a welcome simplification:

  • it’s a smaller API for language bindings like PyO3 to wrap and re-expose, and
  • it’s easier for people to support multiple Python versions to embed into their application; no need to conditionally compile structure field access, can just use normal error handling if configuration values are not available for a specific version at runtime.

I’m going to stay away from the question of whether users should be vendoring their own copy of Python or distributing apps which are expected to work with a system Python etc. On the PyO3 issue tracker I’ve had users wanting to do both of these things, for their own reasons. I think the bullets above apply regardless of distribution option.

Effect on the Stable ABI

It seems to me that there are two main new concepts which this PEP would add to the stable ABI:

  • The two steps “preinitialize” and “initialize”, and I wish it didn’t (I’ll cover that more below).

  • Programmatic control of the configuration variables. This would be helpful, as using the stable ABI’s Py_InitializeEx already allows for a lot of configuration to be read as environment variables. Being able to use the “isolated config” from PEP 587, for example, would be great; at the moment setting a bad PYTHONHOME value can probably break most Rust applications built with a Python embedded via PyO3.

Preinitialization :frowning:

I like how this API merges PyConfig and PyPreConfig of PEP 587 into an opaque structure.

Py_PreInitializeFromInitConfig is missing from the “Specification” bullet list even though it is described down in the full documentation. That got me briefly excited, I think that having multiple steps to initialize from the one configuration object is more complex than it might need to be.

  • Is it an error to modify the PyInitConfig after preinitialization?

  • Could we have APIs like PyInitConfig_AddInittab or PyInitConfig_AddInittabHook, which are used to control the few operations legal between preinitialization and full intialization?

  • Is there anything besides editing the inittab which is expected to be done between the two phases of initialization?

I would love it if it’s possible to adjust the PEP to remove the preinitialization step and add some extra configuration for the inittab (and any other similar bits which would be needed).

I think then this would be a straightforward replacement for Py_InitializeEx where the embedding user gets more control to set the configuration they care about.

Alternative Runtimes

This API might also be very interesting to alternative Python runtimes like PyPy and GraalPy. I suspect that supporting PEP 587 would be difficult due to those runtimes likely needing very different configuration variables, but this API might be possible.

(and a bikeshed, sorry…)

As a final bikeshed, it was a bit surprising to me that a structure called PyInitConfig contains error state. I don’t readily have a better name though, PyInitializer maybe?

4 Likes