The Steering Council is having a tough time evaluating PEP 741 (Python Configuration C API). We’re not sure how to proceed given where we are (a big proposal that apparently failed to reach consensus) and what the SC’s mandate is (find consensus for proposals), and we think we need some help moving this along. In particular we think it’s important to understand why the discussions haven’t led to consensus around this proposal, and what that means for the proposal itself.
We’d like to hear more about the disagreements and concerns that other participants in the discussion have, in particular users who are trying to use the stable ABI in embedded Python situations, as well as C API WG members. We’re not sure in what form that would be most productive, though. It could be here on discuss.python.org, or in private email to steering-council@python.org, or in a video meeting (during our office hours or separately scheduled). We’ll leave that up to whoever wants to participate.
Here are some of the concerns we have, which are not necessarily questions we expect anyone to have answers for:
It’s a big proposal, even if the basic premise (provide a Limited API/Stable ABI version of the PyConfig APIs for embedded interpreter configuration) is simple. It’s adding a lot of functions, with different error handling than other parts of the C API, and some with different memory management (e.g. PyInitConfig_GetStrList
and PyInitConfig_FreeStrList
). It’s also trying to cater to a lot of use-cases, even if the basic use-case is simple (embedding using the Stable ABI). On top of that, this is all by necessity targeted to the set-in-concrete API and ABI, which makes mistakes very difficult to correct.
The SC isn’t really set up to evaluate the technical merits of a proposal like this one. It’s not really the SC’s job, and it’s not why individual SC members get elected. Instead, we have to rely on, and try to find, community consensus for proposals. This is true even when, like now, individual SC members have technical insight and opinions on the proposal’s subject matter.
The API design is fairly complex, given all the different types involved (Int, Str, StrLocale, WStr, StrList, WStrList) and all the different options that can be set. It’s not clear that the use-cases listed in the PEP need all the complexity the PEP proposes. At the same time, it’s not clear that the PEP is enough to satisfy all the use-cases, either.
To what extent is embedding with the Stable ABI a realistic target? Is this C API really the only thing that prevents embedders from using the Stable ABI? The PEP’s testimonials seem to be lauding the stable ABI itself for extension modules, which doesn’t really seem to apply to this C API: embedders can still use extension modules that use the Stable ABI, after all.
Could we not do with a simpler API (e.g. only allow utf-8 strings as values, convert them to whatever Python needs internally)? Do we need to expose all the settings right away? Do we need this API to cater to all user requests?
Is there a way we could test-drive this API, or alternatives to it, without adding it to the Limited API/Stable ABI straight away? For example, a separate C library that doesn’t use the stable ABI itself but exposes the new APIs? (Perhaps as a separate library per major Python version, or as one library with weak symbols and copious runtime version checks.)
So, please, if you have opinions on this PEP you think the SC should be aware of, or ideas on how the SC should evaluate it, let us know. (For what it’s worth, we will continue to debate the PEP in the meantime.)