We now have a working implementation of the main use-case of this PEP — Prototype support for PEP 739 by FFY00 · Pull Request #13945 · mesonbuild/meson · GitHub.
@pf_moore, I believe the PEP is now ready for review.
We now have a working implementation of the main use-case of this PEP — Prototype support for PEP 739 by FFY00 · Pull Request #13945 · mesonbuild/meson · GitHub.
@pf_moore, I believe the PEP is now ready for review.
I’ll take a look in the next week or so. However, one thing struck me immediately on skimming the PEP - it’s not clear what a “Python installation” is in this context. If I’m looking at a virtual environment, is that an “installation”, or am I expected to look at the pyvenv.cfg
file to find the base Python interpreter, and then that is the “installation”? The existence of the base_interpreter
field suggests the former, but I couldn’t find anything saying that explicitly.
Also, I’m still uncomfortable with the fact that very little is said about how to find the file. Putting it in <stdlib>/build-details.json
is not required, and installations are allowed to put the file elsewhere if they want. The Meson implementation seems to suggest it’s up to the user to find the file and pass it to meson - is that the intended approach? If so, it should probably be described in the PEP.
Sorry if this is covered in the discussion here. It’s been a while since I read it and I haven’t re-read it yet, so if it is answered there, that’s fine. But I do think the PEP should include it as well, as it should be possible to read the PEP without reference to the discussion here.
Good point, I will clarify that. A better description might be “Python distribution”.
No, implementations should place the file in <stdlib>/build-details.json
, unless there are technical limitations that prevent it. When such limitations exist, it is up to the implementations to decide if and how to make this file available. Maybe I should make that clearer.
The part about the file possibly being provided in another location is stated to make it clear that in the future we may also provide this file in a different location with better discoverability.
The stdlib
was chosen as the install location as it is reasonably easy to find, however if we are trying to discover the Python installation present on the system without running the interpreter, it isn’t a great location to have the config file. To tackle this use-case, I envisioned that in the future we may do something like standardize locations where config files can be installed (e.g. /usr/share/python/installations
. ~/.local/python/installations
), allowing tools to quickly scan them and discover the Python installations on the system. This is out of scope for the PEP, but I wanted to make it clear that the file config might be provided in extra locations.
Yeah, I agree, I will try to clarify the PEP’s intentions.
FWIW, I’m currently working on this to be able to replace the installer on Windows, and while I tried to follow this PEP it turned out to not be of any benefit. So I ended up creating an __install__.json
in sys.prefix
that contains the info I need (which is basically “what tags should launch this and which executable; what Start menu shortcuts and registry keys should be created; etc.”). This is an internal implementation detail though, so it could merge into build-details.json
later, but I wanted to mention that I found there wasn’t enough useful info in there that I felt the need to make use of it. (There’s a long post in Ideas right now if you want more details, or I’ll hopefully be posting the PEP properly later today.)
Which other information do you need?
Including a list of supported wheel tags in build-details.json
is something we can consider, but I don’t want to do it in this first format version.
Am I still waiting for an update to the PEP here? There’s no rush if you’re busy, but I don’t want to be the one delaying things because I missed that an update had been done…
No, I pushed a large-ish update last week.
There were two recent changes in PEP 739: some (hopefully final) updates by FFY00 · Pull Request #4216 · python/peps · GitHub and PEP 739: some more (hopefully) final updates by FFY00 · Pull Request #4220 · python/peps · GitHub, perhaps?
A
Yes, exactly. I hope those changes were enough to address Paul’s worries, and am open to more direct/quicker feedback rounds to get it in shape. I’d like to have this resolved sooner rather than later, so that we can ship it ASAP to start testing and integrating it into tooling, ensuring that whenever 3.14 gets a final release, things are already at least somewhat mature.
Thanks. Apologies for not spotting the new version, I don’t typically see notifications from the PEPs repo. The revision has addressed my concerns, thanks for that.
With this latest update, I’m pleased to say that the PEP is approved. Thanks to everyone who participated in the discussion, and congratulations to @FFY00.
One note - the “human readable” version of the JSON schema used in the PEP is very much appreciated. It’s far easier to read than the raw schema definitions I’ve occasionally seen elsewhere. I assume a tool was used to produce it (if you did it by hand, my appreciation is even greater!) but the results were definitely worthwhile.
One minor editorial comment that I hadn’t spotted earlier:
abi.extension_suffix
- states that the entry will be missing if the implementation doesn’t support extensions, but says required=True
. I assume required
should be False
.I’m happy for the position on this field to be clarified without affecting my approval of the PEP.
No worries, I will explicitly ping you next time.
Yes, it’s rendered with a script I wrote. It is available in this gist (archived it just in case), which is mentioned as a comment in the rST source. I intended to commit it to the repo, but the PEP editors told me we should leave it out, so I uploaded it as a gist and added a comment with a link to it.
Oh, yes! Thanks for noticing. It is a mistake, I’ll correct it when updating the PEP status.
Even better - my thanks to you for taking the time to write such a script
Is there a plan to make this information also available through a Python API?
Yes, likely in sysconfig
, but I am not sure yet how exactly the API would look like. Probably, the API would either be containers with the same information as the file, or — and this was my initial plan for the new syconfig
API — a richer, more Pythonic API, that could load any arbitrary build-details.json
file. The goal with the fancier API was to be able to have packaging build backends use the new API for Python introspection, which would then make it seamless to support cross-compilation — they just use the build-details.json
file from the target.
One thing I would like to expose regardless, is a way to ask the Python installation for its build-details.json
file, if available.
Any possibility of getting this usable for docs builds? I just went through a bit of work trying to figure out “How do I get the Docs
make doctest
to use my newly-built Python”, be really nice if it could rely on the static definition to find it. (ex. Add a new method to a Python builtin which contains a doctest
, doctest
will fail unless built with the new Python build). Today that information has to be hand-passed via make variables. Would also be nice dogfooding / “does this provide the benefit expected” without exposing externally APIs.
Yes, starting to move away from relying on make variables is precisely one of my motivations behind this PEP and the new sysconfig
API work.
Congratulations and thanks for all the hard work! Very exciting!