# PEP 741: Python Configuration C API

**URL:** <https://discuss.python.org/t/pep-741-python-configuration-c-api/43637>\
**Category:** PEPs\
**Created:** [January 19, 2024, 4:53pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637 "2024-01-19T16:53:03Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [January 19, 2024, 4:53pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/1 "2024-01-19T16:53:03Z")

</div>

Read the [PEP 741: Python Configuration C API](https://peps.python.org/pep-0741/).

## Abstract

Add a C API to the limited C API to configure the Python preinitialization and initialization, and to get the current configuration. It can be used with the stable ABI.

Add `sys.get_config(name)` function to get the current value of a configuration option.

Allow setting custom configuration options, not used by Python but by third-party code. Options are referred to by their name as a string.

[PEP 587](https://pep-previews--3625.org.readthedocs.build/pep-0587/) “Python Initialization Configuration” unified all the ways to configure the Python **initialization**. This PEP unifies also the configuration of the Python **preinitialization** and the Python **initialization** in a single API.

## Out of scope: set config at runtime

There is no API to _set_ a configuration option while Python is running. Only to set the _initialization_ configuration. Some people discussed it, but I didn’t see a clear willingness to have this feature.

Technically, it can be implemented. If we decide to add such API, I would suggest to make most options read-only, and add callbacks on others to validate values (reject invalid values) and execute code when an option is modified (ex: update/invalidate caches).

For example, do you want to be able to change `bytes_warning` option at runtime? I’m not sure that the Python ecosystem is ready to see this option changing between two lines of code.

It may be interesting to have a **Python API** (like `sys.set_config()`) to set some options at Python **startup**.

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [January 19, 2024, 6:24pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/2 "2024-01-19T18:24:59Z")

</div>

I think this is a good idea and the API looks generally well thought.

I would suggest a few changes:

- Instead of `PyInitConfig_SetStr` taking a locale-encoded string, I would suggest `PyInitConfig_SetUtf8` taking a UTF8 string (probably the preferred choice for most people?) and `PyInitConfig_SetLocaleStr` taking a locale-encoded string;
- `PyInitConfig_Exception` would probably be less confusingly named `PyInitConfig_HasError` (we aren’t talking about proper Python exceptions, are we?);
- `PyInitConfig_GetError` could probably return a `const char*` directly, rather than have a separate `int` return code that only duplicates the information.

Also, I do not understand what a config exit code is.

I’m not sure the `allow_custom_options` special config entry is really useful. Instead, you could arrange that values starting with some well-known prefix such as `X-` or `vendor.` are freely accepted.

Finally, it seems that `PyConfig_Get` and `PyConfig_GetInt` will only work if interpreter initialization was successful, since they set an exception on error? This means it’s not possible to inspect the current configuration while building it.

---

<div class="post-metadata">

**Author:** ![ronaldoussoren](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ronaldoussoren/32/61_2.png) [@ronaldoussoren](https://discuss.python.org/u/ronaldoussoren)\
**Post date:** [January 19, 2024, 8:57pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/3 "2024-01-19T20:57:52Z")

</div>

Are the configuration options themselves also a stable API?

The pep allows setting custom options. IMHO it would be better to specify that all names without a colon are reserved for CPython configuration and will be ignored as custom option. This would avoid breakage when adding new options in future Python versions.

---

<div class="post-metadata">

**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\
**Post date:** [January 22, 2024, 2:57pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/4 "2024-01-22T14:57:10Z")

</div>

Thank you for writing it down as a PEP!

For the rationale: Note that the PyO3 project _can_ target limited C API, bit this is not [(yet)](https://github.com/PyO3/pyo3/issues/2865) the default.

The “Deprecated legacy API” section lists API that was deprecated in Python 3.8, and then says it was deprecated in 3.11, 3.12 and 3.13. Which is correct?

What is the rationale for custom configuration options?

I see `PyPreConfig` and `PyConfig` are merged into `PyInitConfig`. Does that make the previous two obsolete? Can we do anything to avoid having so many different structs?  
Would it be good to add `PyInitConfig_GetPyConfig` and `PyInitConfig_GetPreConfig` so that version-specific code can cooperate with limited-API code?  
Or would it make sense to add `preconfig` and `status` fields to `PyConfig`, and use _that_ instead of `PyInitConfig` (i.e. make it opaque in the limited API)?

What does it mean to “Get current configuration”? I assume that for _some_ keys it returns the value used for config, but for others it returns a value from elsewhere (e.g. in `sys.argv`). Is that right? And if so, is it more _useful_ than always returning the value used to initialize Python? (IMO, having `PyConfig_Get` only query info stored in a runtime-wide `PyInitConfig` struct would make it quite a bit clearer.)

The `PyConfig_Get` API is a tiny part of a Mapping protocol. Why should we not expose e.g. the set of available keys? (IMO, a dump of the config would be quite useful in an error report, but you’d need `iter` for that.)

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [January 23, 2024, 3:11pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/5 "2024-01-23T15:11:36Z")

</div>

> [@pitrou](#):
>
> Instead of `PyInitConfig_SetStr` taking a locale-encoded string, I would suggest `PyInitConfig_SetUtf8` taking a UTF8 string (probably the preferred choice for most people?) and `PyInitConfig_SetLocaleStr` taking a locale-encoded string;

I’m fine with having separated API for UTF-8 encoded and locale-encoded strings. In that case, I would prefer UTF-8 to be the default as `PyInitConfig_SetStr`, and `PyInitConfig_SetLocaleStr` for the locale-encoded string.

> [@pitrou](#):
>
> - `PyInitConfig_Exception` would probably be less confusingly named `PyInitConfig_HasError` (we aren’t talking about proper Python exceptions, are we?);
> - `PyInitConfig_GetError` could probably return a `const char*` directly, rather than have a separate `int` return code that only duplicates the information.

Implementating PEP 587 “PyConfig API” was a huge piece of work and I left the code to handle a few actions in the “initialization” for backward compatibility. For example, if you can Python with `python --help`, `Py_InitializeFromConfig()` will return “an exception” and the exit code set to `0`. I need to investigate how doable it would be to be able to **move** this code into `Py_RunMain()` to make the API easier to use.

> [@pitrou](#):
>
> I’m not sure the `allow_custom_options` special config entry is really useful. Instead, you could arrange that values starting with some well-known prefix such as `X-` or `vendor.` are freely accepted.

The problem is to catch typos in configuration option names. If you set `module_search_path` instead of `module_search_paths`, it will be silently ignored. Maybe we can do the opposite and reserve a prefix to all Python options, such as `py:`? For example, set `py:verbose`? Any name not string with `py:` would be a custom option.

> [@pitrou](#):
>
> Finally, it seems that `PyConfig_Get` and `PyConfig_GetInt` will only work if interpreter initialization was successful, since they set an exception on error? This means it’s not possible to inspect the current configuration while building it.

You can only call these functions _after_ Python is initialized. Is it an issue?

If you configure the Python initialization, you have a `PyInitConfig` structure that you can use to _set_ options (but you cannot _get_ options from it).

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [January 23, 2024, 3:24pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/6 "2024-01-23T15:24:27Z")

</div>

> [@ronaldoussoren](#):
>
> Are the configuration options themselves also a stable API?

Adding new options is fine, but I’m not sure if removing options would be acceptable? Should we support deprecated/removed options forever? Or can we just fail with an error?

I don’t recall examples of _removed_ configuration options. My hope is that `legacy_windows_fs_encoding` and `legacy_windows_stdio` options will go away at some point, but I’m not sure about it.

> [@encukou](#):
>
> For the rationale: Note that the PyO3 project _can_ target limited C API, bit this is not [(yet)](https://github.com/PyO3/pyo3/issues/2865) the default.

I wasn’t sure about that, I tried to explain that the limited API is optional. I will try to rephrase that.

> [@encukou](#):
>
> The “Deprecated legacy API” section lists API that was deprecated in Python 3.8, and then says it was deprecated in 3.11, 3.12 and 3.13. Which is correct?

The latter. Apparently, my phrasing is confusing. I mean that since PEP 587 was implemented in Python 3.8, some APIs started to be deprecated.

> [@encukou](#):
>
> I see `PyPreConfig` and `PyConfig` are merged into `PyInitConfig`. Does that make the previous two obsolete?

The Backward Compatibility section says that PEP 587 PyConfig API is still supported, there is no plan to deprecate it. For me, it’s almost a different use case (different constraints).

> [@encukou](#):
>
> Can we do anything to avoid having so many different structs?

PyInitConfig is an opaque structure, it’s different. I don’t see how we can reduce the number of structures without affecting the backward compatibility.

> [@encukou](#):
>
> Would it be good to add `PyInitConfig_GetPyConfig` and `PyInitConfig_GetPreConfig` so that version-specific code can cooperate with limited-API code?

That would be incompatible with the limited C API, since PyConfig and PyPreConfig members are not part of the limited C API. I would prefer to avoid that.

> [@encukou](#):
>
> Or would it make sense to add `preconfig` and `status` fields to `PyConfig`, and use _that_ instead of `PyInitConfig` (i.e. make it opaque in the limited API)?

`PyConfig` is part of `PyInterpreterState`, whereas `PyPreConfig` is part of `PyRuntimeState`. The `PyStatus` is only used to report failures to the `Py_InitializeFromConfig()` caller, it shouldn’t be stored at runtime. Also, changing these structures might affect the backward compatibility.

> [@encukou](#):
>
> What does it mean to “Get current configuration”? I assume that for _some_ keys it returns the value used for config, but for others it returns a value from elsewhere (e.g. in `sys.argv`). Is that right?

When `PyConfig` members are only used to _initialize_ configuration options, `PyConfig_Get()` gets the _runtime_ configuration option value, not the `PyInterpreterState.config` value which was used for initialization.

For example, `sys.path` is always different than `PyConfig.module_search_paths`, since the `site` module is executed after `sys.path` is initialized from `PyConfig.module_search_paths`. `PyConfig_Get("module_search_paths")` gets `sys.path`.

Some options are not copied outside PyConfig, such as `PyConfig.tracemalloc`. Some options are copied but cannot be modified at runtime.

> [@encukou](#):
>
> And if so, is it more _useful_ than always returning the value used to initialize Python?

I think that it’s more useful to use what the user gets/sees at runtime, rather than what was used to initialize Python. That’s why it’s called `PyConfig_Get()` and not `PyInitConfig_Get()`.

> [@encukou](#):
>
> The `PyConfig_Get` API is a tiny part of a Mapping protocol. Why should we not expose e.g. the set of available keys?

I prefer to write the smallest API, unless a strong use case to justify adding more options.

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [January 23, 2024, 3:32pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/7 "2024-01-23T15:32:59Z")

</div>

> [@vstinner](#):
>
> I’m not sure if removing options would be acceptable? Should we support deprecated/removed options forever? Or can we just fail with an error?

I’m sure we can deprecate options like we do with any other API that doesn’t have a purpose anymore. We can then remove them after a few releases if we want to.

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [January 23, 2024, 3:39pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/8 "2024-01-23T15:39:07Z")

</div>

> [@vstinner](#):
>
> You can only call these functions _after_ Python is initialized. Is it an issue?

I imagine it might be nice to inspect existing configuration before trying to modify it. In general, it seems weird and error-prone to have a getter API that requires more precondition than the comparable setter API 🙂

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [January 24, 2024, 4:42pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/9 "2024-01-24T16:42:57Z")

</div>

My initial concern with this proposal is that I don’t believe there’s a real benefit to having initialization in the _limited_ API.

While theoretically an embedder could use the limited API, they really ought to know the version of Python that they are using, and very likely need to know it at compile time.

> Note that the APIs being discussed are only useful for _embedders_. Every other time we’ve had this discussion this point gets missed, so let me elevate it up front.

Certainly on Windows, every scenario where you would embed CPython is best done with your own private copy of it, and not by searching the user’s machine for an install. If for whatever reason you _must_ use the user’s existing install, you should launch Python with your own script and import an extension module (also known as running out-of-process). The security implications of loading arbitrary code (with network access!) into your own process are terrifying, and we should _not_ encourage it. Also, generally people want to load an existing install to get the existing 3rd party modules, but chances are the search paths will be wrong due to the excessively complicated system we have for calculating it.

I want to see a rationale for why we need version-independent embedding before we commit to a version-independent initialization API, regardless of how the API looks. That rationale isn’t in the PEP yet.

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [January 24, 2024, 5:00pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/10 "2024-01-24T17:00:48Z")

</div>

> [@steve.dower](#):
>
> My initial concern with this proposal is that I don’t believe there’s a real benefit to having initialization in the _limited_ API.

Adding an API to the limited C API was requested by different users:

- [FR: Allow private runtime config to enable extending without breaking the `PyConfig` ABI - #9 by ronaldoussoren](https://discuss.python.org/t/fr-allow-private-runtime-config-to-enable-extending-without-breaking-the-pyconfig-abi/18004/9)
- [FR: Allow private runtime config to enable extending without breaking the `PyConfig` ABI - #10 by gpshead](https://discuss.python.org/t/fr-allow-private-runtime-config-to-enable-extending-without-breaking-the-pyconfig-abi/18004/10)
- [FR: Allow private runtime config to enable extending without breaking the `PyConfig` ABI - #11 by LostTime76](https://discuss.python.org/t/fr-allow-private-runtime-config-to-enable-extending-without-breaking-the-pyconfig-abi/18004/11)
- [[C API] PEP 741: No limited C API to customize Python initialization (PyConfig, PEP 587) · Issue #107954 · python/cpython · GitHub](https://github.com/python/cpython/issues/107954#issuecomment-1893988614)

> [@steve.dower](#):
>
> I want to see a rationale for why we need version-independent embedding before we commit to a version-independent initialization API, regardless of how the API looks. That rationale isn’t in the PEP yet.

Sure, I will try to summarize the rationale for adding such API to the limited C API.

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [January 24, 2024, 5:22pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/11 "2024-01-24T17:22:15Z")

</div>

> [@vstinner](#):
>
> Adding an API to the limited C API was requested by different users

No, the first two requested struct stability within a major release, and the third wants an initialization API that doesn’t rely on C structures. The fourth points out that closing the issue about a limited API implies that there is now a limited API, which is not the case, but doesn’t provide any motivation or rationale for actually wanting it.

There _are_ actual user requests on Discourse and GitHub if you want to find them. Probably the easiest way to find them is to look for my replies explaining why they will be happier if they embed their own copy of Python rather than doing what they think they want to do 😉

Summarising “don’t break structure between `3.x.y` and `3.x.(y+1)`” doesn’t justify the proposal to not change the API between `3.x` and `3.(x+N)`.

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [January 25, 2024, 12:46am UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/12 "2024-01-25T00:46:50Z")

</div>

Getting further into the PEP, I’d really like to get away from `PreInitialize` and `Initialize` methods.

With a lot of the cleanup that @eric.snow (and others) have done for subinterpreters, we should be getting very close to having a separation between “initialize runtime” and “initialize the first interpreter”. This I believe makes more sense, and is less built around how libc works.

We may still be a release away from actually having that separation, but we can see it coming, and I would much prefer not to formalise the old nomenclature forever in the meantime.

* * *

The work that is currently done in PreInitialize is to enable the program to correctly parse locale strings. I believe that should be done in the host application (i.e. `python.c`) rather than in the runtime (i.e. `libpython.so`). In practice, we may still provide a helper function from libpython to do the parsing for standard variables, but embedders should be able to simply omit the global locale changes we make, and simply skip the environment parsing we do. But essentially, I want `PreInitialize` to become redundant.

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [January 28, 2024, 1:55pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/13 "2024-01-28T13:55:12Z")

</div>

> [@steve.dower](#):
>
> The work that is currently done in PreInitialize is to enable the program to correctly parse locale strings. I believe that should be done in the host application (i.e. `python.c`) rather than in the runtime (i.e. `libpython.so`).

That’s why there are two configurations:

- “Python” configuration is to write a program which is almost like “python” but with minor differences, so configure locales and C stdio the same way. It’s to write an “application”.
- “Isolated” configuration leaves the current process unchanged, don’t touch locales for example: it’s to embed Python in an existing application.

Obviously, people are doing things in the between, that’s why there are configuration options 🙂

---

<div class="post-metadata">

**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\
**Post date:** [January 29, 2024, 8:08am UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/14 "2024-01-29T08:08:45Z")

</div>

> [@steve.dower](#):
>
> Certainly on Windows, every scenario where you would embed CPython is best done with your own private copy of it, and not by searching the user’s machine for an install.

That’s not my experience on Linux: I want the system Gimp, OBS or Tiled to default to the system Python – or, preferably, let me configure which Python to use.  
With things like Flatpack, things are moving toward what you describe. But I don’t think we got to the point where Python should drop support for the “old way”. (And I’d still prefer being able to _assemble_ self-contained sandboxes by linking rather than re-building, but I’m not quite in a position to steer toward that.)

> [@steve.dower](#):
>
> If for whatever reason you _must_ use the user’s existing install, you should launch Python with your own script and import an extension module (also known as running out-of-process).

Thinking about the GUI apps I mentioned: running scripts out-of-process gets quite complicated if the script wants to show a clickable button or live visualisation. It’s not _impossible_, but the necessary architecture is usually more expensive to maintain than, say, wrapping C++ objects.

> [@steve.dower](#):
>
> The security implications of loading arbitrary code (with network access!) into your own process are terrifying, and we should _not_ encourage it.

I generally agree that we should do less of that. But how terrifying this is does depend on your security model. (IMO, it’s not worse than grabbing random stuff off PyPI.)

> [@steve.dower](#):
>
> Also, generally people want to load an existing install to get the existing 3rd party modules, but chances are the search paths will be wrong due to the excessively complicated system we have for calculating it.

Yeah, but it was _possible_ to get it working. And if somebody did, I can imagine them being upset when Python removes the API – especially without a full replacement.

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [January 29, 2024, 1:28pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/15 "2024-01-29T13:28:44Z")

</div>

> [@encukou](#):
>
> That’s not my experience on Linux: I want the system Gimp, OBS or Tiled to default to the system Python – or, preferably, let me configure which Python to use.

As a user, that’s your preference. From the POV of the developers trying to support that, it’s usually a nightmare that they come to regret. If we were better at compatibility between versions, maybe, but the rate that we add behaviour changes (not just API changes) means that they’ll spend way too much time trying to reproduce differences reported by users, and eventually they’ll prefer to have just had a fixed version for their integration.

I think we’re better off helping them integrate a single Python runtime more tightly into their app, and provide ways to then help users do what they want to do, such as installing additional packages or connecting into IDEs. This isn’t hypothetical either, I’ve actually worked with products who do this and have often tried both ways.

> [@encukou](#):
>
> Thinking about the GUI apps I mentioned: running scripts out-of-process gets quite complicated if the script wants to show a clickable button or live visualisation. It’s not _impossible_, but the necessary architecture is usually more expensive to maintain than, say, wrapping C++ objects.

Out-of-proc is definitely more complicated up-front, and requires a bit more imagination when it comes to architecture. But it also quickly gets outweighed by the fact that your IPC machinery is going to be much easier to debug and maintain across multiple/arbitrary Python versions than an in-proc integration. Again, I’ve done all these approaches before, so I’m not hypothesising - I’ve seen the results of both approaches.

> [@encukou](#):
>
> But how terrifying this is does depend on your security model. (IMO, it’s not worse than grabbing random stuff off PyPI.)

“Distributing an app that loads arbitrary DLLs” isn’t even the same people who would be grabbing random stuff off PyPI, so maybe it’s no worse from the POV of a developer, but it’s 100% different from the POV of a sysadmin deciding whether to allow your app into their network or not (they probably don’t allow PyPI either, which is fairly common).

An OBS that can load DLLs from a user’s download folder and implicitly grant them access to the stored login credentials for a social media account isn’t going to be popular with IT departments. We don’t want that to happen because we told them they should load whatever Python they can find.

> [@encukou](#):
>
> Yeah, but it was _possible_ to get it working. And if somebody did, I can imagine them being upset when Python removes the API – especially without a full replacement.

You were lucky to get it working 😃 I don’t think even I could’ve made it work on purpose back when I was deep inside getpath all the time. Far easier to just launch the existing install, print out sys.path and inject it into the version you’ve got (and hope they’re compatible, but you were already hoping that so you’re no worse off).

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [January 29, 2024, 1:30pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/16 "2024-01-29T13:30:43Z")

</div>

> [@vstinner](#):
>
> That’s why there are two configurations:
> 
> - “Python” configuration is to write a program which is almost like “python” but with minor differences, so configure locales and C stdio the same way. It’s to write an “application”.
> - “Isolated” configuration leaves the current process unchanged, don’t touch locales for example: it’s to embed Python in an existing application.

I know that’s why it was _done_ like that, but I don’t think it should’ve been (and opposed it at the time, though my idea of how it _should_ be is more fleshed out now).

`libpython` should only offer the “isolated” configuration. `python.c` should implement the “Python” configuration, and make that code easy for embedders to copy and adapt.

---

<div class="post-metadata">

**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\
**Post date:** [January 29, 2024, 2:49pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/17 "2024-01-29T14:49:25Z")

</div>

Well, our experiences differ. IMO, `/usr/lib64/libpython3.so` is not quite an arbitrary DLL. No Downloads folder needed. And if an app can stick to stable ABI, there aren’t really that many behaviour changes to handle – especially if the Python scripting support is aimed at people familiar with Python. Of course, the situation changes as the integration matures, but IMO “bring your own Python” _is_ a good first step to add Python scripting – and it’s also a good option to provide for advanced users.  
Forcing app devs to become redistributors of CPython is very, very heavy.

Anyway: Your ideas aren’t bad, but IMO this should all be designed, communicated, and tried out _before_ we remove the API to do it “the old way”.  
That’s not the situation we’re in here. If we want to try this way, we should revert the removals first.

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [January 29, 2024, 2:58pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/18 "2024-01-29T14:58:54Z")

</div>

> [@encukou](#):
>
> Your ideas aren’t bad, but IMO this should all be designed, communicated, and tried out _before_ we remove the API to do it “the old way”.  
> That’s not the situation we’re in here. If we want to try this way, we should revert the removals first.

I thought we were discussing adding stuff to the limited API? Yes absolutely, if anything has been removed, bring it back.

We ought to design properly before adding anything to the limited API, and I’d argue we should design properly before changing anything that isn’t a blatant bug in the current API. Embedders deserve some stability, and we should make the next change a significant improvement for them.

> [@encukou](#):
>
> IMO, `/usr/lib64/libpython3.so` is not quite an arbitrary DLL.

That’s fair, but you also said you want users to customise it. Selecting between system-installed versions is fine, IMHO, but that’s a far more limited scenario. And even then, I’d still contend that developers will have a happier time if they restrict it to a single version and avoid picking up all the various site-packages that will be there.

---

<div class="post-metadata">

**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\
**Post date:** [January 29, 2024, 3:13pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/19 "2024-01-29T15:13:20Z")

</div>

> [@steve.dower](#):
>
> I thought we were discussing adding stuff to the limited API? Yes absolutely, if anything has been removed, bring it back.

+1  
@vstinner Do you agree?

> [@steve.dower](#):
>
> And even then, I’d still contend that developers will have a happier time if they restrict it to a single version and avoid picking up all the various site-packages that will be there.

Except if they _want_ to play with a newer Python version – perhaps to catch the new bugs.  
Yes, loading random DLLs shouldn’t be the default, but it’s useful if you know what you’re doing. We should make it harder to exploit than, say, `import ctypes`, but not prevent it entirely.

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [January 29, 2024, 5:43pm UTC](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637/20 "2024-01-29T17:43:39Z")

</div>

> [@encukou](#):
>
> Except if they _want_ to play with a newer Python version – perhaps to catch the new bugs.

I mean, the _developers_ of the app can do whatever they like. And if you’re looking for the kind of crashes that will happen when you swap out a CPython version without changing the app, I’m prepared to consider you a developer of the app.

The scenario is that the users are merely downloading, installing and using an app, and expect it to be reliable. OBS is a great example - the vast majority of users will never be “developers” of it, and probably most users of Python in it won’t be Python developers either, but will have copy-pasted something that they needed to work.

[Next page](https://discuss.python.org/t/pep-741-python-configuration-c-api/43637.md?page=2)
