@doko42 suggests putting it into python3.14-config --configdir (e.g. /usr/lib/python3.14/config-3.14-x86_64-linux-gnu)
(Although python3.x-config is only available when python3.x-dev is installed.)
Yes, because you configure it that way. Is there anything preventing you from not doing so?
2 Likes
That shouldn’t matter here, the point of this file is to be found without launching the interpreter, and to be referenceable by a user/tool (e.g. by directory name) without being implicitly dependent on $PATH or similar.
I’m not sure how that best works out in your case, but it shouldn’t depend on inspecting runtime state. Think of it as needing to locate this file in order to inspect runtime state (but hopefully the state is in the file itself).
1 Like
In practice we still wouldn’t install anything in there. So you’re suggesting that we change our sysconfig just to help find this one file. I don’t think that’s the best direction to be taking.
Right, that makes sense.
We’re just looking for somewhere well-known within the Python install.
The tool analysing this still needs to find the Python install. As well as finding this file within it.
The obvious approaches I can see are for us to have:
- A
build-detailsdirectory, containing all the relevant.jsonfiles for this install, with the multiarch tuple in their names. - A possible filename extensions (like C extensions)
build-details-*.json - Put
build-detailsinto theconfigdirectory (but then you need to find that).
Also, I note that these two keys only take a single value, but our multi-arch approach means that we have 2 values for each:
abi.extension_suffix: ['.cpython-314-x86_64-linux-gnu.so', '.so']abi.stable_abi_suffix: ['.abi3.so', '.abi3-x86_64-linux-gnu.so'](See GH-122931)
It’s not obvious from the text, which of the two values you’d want represented there. Probably the more specific one? But as long as GH-122931 languishes as PR, .abi3-x86_64-linux-gnu.so is the preferred local install path, but not something that should be published in a wheel.
Do you really expect all tools that consume platform-specific files to learn about Debian-specific multiarch paths? From what I gather from a quick read over the docs, when other static files have arch-specific content (e.g., generated headers, pkg-config files) they can be put in places with target in the name but there’s a symlinking approach to pick the right one. Can’t that work here?
I guess the bit I’m missing is how the end user that uses multiarch installs selects between them. I’d hope that’s some step that, when taken, allows updating the symlink that the generic name points to.
1 Like
Simple tools don’t need to. They can be passed an install path, and respect it.
In some cases, we can do hacks like install to a non-multiarch path, and move files around or rename them later.
But yes, all of the complex build tools that need fingers deep in the pie, need to learn the basics of the environments they operate on. Python has supported our multiarch layout for a decade already.
Right, you don’t select between them. There can be two architectures of a library installed concurrently. However, there can only be one architecture of a well-known binary installed at a time (there is only one /usr/bin/python3.14).
Let’s talk about some concretely useful examples:
If we install a foreign architecture arm64 binary that embeds Python 3.14, it will depend on libpython3.14:arm64, which depends on libpython3.14-stdlib:arm64 (the current home of build-details.json).
If we imagine a cross-building situation, where we’re trying to build a Python C extension for arm64 on an amd64 host, libpython3.14-dev:arm64 is installed. But python3.14:amd64 is also installed, for the build tools. These both depend on libpython3.14-stdlib, so you’ll get two of those co-installed.
In our cross-building situation, the amd64 build tools can find out about the arm64 Python, by setting the _PYTHON_SYSCONFIGDATA_NAME environment variable. This apparently works very successfully.
One approach to the problem we’re facing here is to move build-details.json from the libpython3.14-stdlib package into the python3.14-minimal package that provides /usr/bin/python3.14. There is only one of those installed at a time. Whether that’s the right thing to do, or not, really depends on what tools are consuming this file.
Pros: No conflicts.
Cons: Any build-details.json that you find may represent the build host’s native Python, rather than the build target.
2 Likes
I think the ideal assumption to work from is that a user specifies where to look, and then the file is to provide all the other details. So a possible guideline is “given the path to the python executable, a tool can locate the build-details.json for it” (for *nix systems this seems to make more sense than getting the prefix, which is what we’d likely expect for Windows,).
Again, I don’t know what the right answer is here, so I’m trying to help make sure the question is correct/in line with our intent, so that it’s easier for the actual experts to answer.
In light of your reply above that was posted while I was first typing, I think it probably means moving the build details into the python3.14-minimal package.
The one that a build tool should generate by default (unless the user specifically overrides the suffix). I suspect in both of these cases, it would be the first value you listed.
1 Like
Given that cross-compilation is a main use case for this file, I think you want to have both. So probably allowing build-details-*.json is better than moving to -minimal. This is of course then more work for every build tool to support, but it seems okay for this to take longer to surface and fix in multiple tools - it just means regular cross-compilation support without hacks will take longer for Debian than for other distros.
For non cross compilation purposes, you only want the native arch file, so also having build-details.json in -minimal may also be good. Then for build tools they’d have to check for the multiarch flavor first, and the plainly named one after.
Not really, the build tool needs to be told which one it should target (unless it’s a tool that targets “everything”, in which case, they can add additional search logic).
If build tools are going to say “give me the path to the JSON file I should use to build with” then it’s no extra work. If they’re just going with “give me the sys.prefix” or something like that, then it’s probably impossible to answer the question already.
3 Likes
Yep, this file points you to all the other paths you need. So if the interface is to point to it, then its location on the filesystem is non-critical.
In the typical (native) case, both files would be present on the filesystem: build-details.json and build-details-x86_64-linux-gnu.json. So the tool needs to know what it wants.
I guess, with this approach, the build tool would look at build-details.json when doing native work, and examine all available files to find the one matching the target architecture, when cross-compiling.
This might solve the issue for multi-arch installs, but not for multiple build configurations.
On Fedora, the debug & optimized builds share platlib, and are distinguished by flags in the .cpython-314d-x86_64-linux-gnu.so suffix.
I retract this proposal, our nomenclature on the path names is historic, and a bit confusing.
What the PEP means is that the build-details.json file should go to sysconfig.get_path(‘stdlib‘)/. The “platform“ in “platform independent“ really means, the system directory, as opposed to platstdlib, which is a leftover from the old virtual environments, and refers to a standard library directory specific to PYTHONHOME, and as a byproduct also virtual enviroments.
Like I said, this is confusing, the stdlib and platstdlib paths differ as such:
stdlib:/usr/lib/python3.13platstdlib:~/.virtualenvs/example/lib/python3.13/site-packages
Here’s my proposal to the Debian maintainers.
On multiarch installs, as described at Multiarch/TheCaseForMultiarch - Debian Wiki , use /usr/lib/aarch64-linux-gnu for eg.
This can be done by configuring CPython with --with-platlibdir=lib/aarch64-linux-gnu.
Partly, they are also distinguished by flags in the prefix subdir name, eg. libpython3.14 vs libpython3.14t.
Currently there’s a discrepancy, which I believe only affects debug builds. They will have a different subdir for headers, but not for stdlib/purelib/etc.
IMO, these different behaviors introduce unnecessary complexity, and we should adopt a consistent approach. But let’s discuss that in Adopt a consistent POSIX layout · Issue #142265 · python/cpython · GitHub.
Yes, we do that in Debian too. In fact, even PyPy is sharing the same tree.
(Although we stopped building debug builds since the pydebug interpreter started being able to import the regular builds).
You’re essentially asking us to move our entire Python install. Yes, that would technically solve the problem, but at the cost of a lot of work moving things around. And of course some unnecessary disk duplication in multi-arch co-installs.
That means we can put the build-details.json in the place defined in the PEP, but I don’t know how it helps a user to point to the correct build-details.json.
Is the complexity unnecessary?
Should distros change what they’re doing to make PEP 793 work, or should PEP 793 change to make it work with distros?
Shouldn’t build-details.json be part of -dev package, similar to python3.14-configor python-3.14.pc?
Since it points to /usr/lib/x86_64-linux-gnu/libpython3.14.a and includes that only available in -dev.